[go: up one dir, main page]
Page MenuHomePhabricator
Create Task
Maniphest T67264

File storage uses (title,timestamp) as a unique key, but this is not unique
Open, HighPublic
Actions

Description

FileRepo::findFile appears to use the (title,timestamp) pair as a unique key to identify a particular version of a file. However, this is not necessarily unique. See for example https://commons.wikimedia.org/wiki/File:Treppe_22_22_test_upload.jpg where two versions were both uploaded at 2014-05-05T13:00:59Z.

For proper operation, the API requires some ordered unique identifier for file revisions. I imagine there are other places in the code that also require unique identifiers.

This appears to depend on bug 15441 (for the oldimage table), although I'm not familiar enough with the FileBackend code to know for sure whether that's generally the case or if it only applies to certain backends.

Version: 1.23.0
Severity: normal

Details

Reference
bz65264

Related Objects
Search...

Event Timeline

• bzimport raised the priority of this task from to High.Nov 22 2014, 3:24 AM
• bzimport added a project: MediaWiki-File-management.
• bzimport set Reference to bz65264.
• bzimport added a subscriber: Unknown Object (MLST).
Anomie created this task.May 13 2014, 4:33 PM
gerritbot added a comment.May 13 2014, 10:04 PM

Change 133178 had a related patch set uploaded by Aaron Schulz:
Made LocalFile avoid duplicate (name,timestamp) pairs

https://gerrit.wikimedia.org/r/133178

gerritbot added a comment.May 13 2014, 10:21 PM

Change 133178 merged by jenkins-bot:
Made LocalFile avoid duplicate (name,timestamp) pairs

https://gerrit.wikimedia.org/r/133178

Platonides added a comment.May 14 2014, 12:01 AM

This is probably a duplicate of some older bug.

Anomie added a comment.May 14 2014, 2:49 PM

So change 133178 fixed it going forward, which is good. Although I left a comment there.

We'll also need to do database cleanup for situations where this already exists, such as https://commons.wikimedia.org/wiki/File:Treppe_22_22_test_upload.jpg, to completely fix the bug. I also wonder if it's worth making the oi_name_timestamp index unique, although that would only detect the problem after the fact since the row for the current version isn't added to oldimage until a new version becomes current.

• Gilles added a project: Multimedia.Nov 24 2014, 3:33 PM
Anomie mentioned this in T67251: Query-continue for imageinfo returns timestamp which is ambiguous if multiple files were uploaded within the same second under the same file name.Feb 13 2015, 7:07 PM
Jdforrester-WMF moved this task from Untriaged to Backlog on the Multimedia board.Sep 4 2015, 6:46 PM
Restricted Application added subscribers: Steinsplitter, Matanya, Aklapper. · View Herald TranscriptSep 4 2015, 6:46 PM
MarkTraceur unsubscribed.Jan 27 2017, 10:36 PM
Restricted Application added a project: Commons. · View Herald TranscriptJan 27 2017, 10:36 PM
Krinkle edited subtasks, added: T28741: Migrate file tables to a modern layout (image/oldimage; file/file_revision; add primary keys); removed: T17441: Some tables lack unique or primary keys, may allow confusing duplicate data.Apr 12 2017, 8:55 PM
Liuxinyu970226 subscribed.Nov 26 2017, 11:20 PM
Anomie mentioned this in T192866: Some DjVu files have too much metadata to fit in their database column.Apr 25 2018, 4:06 PM
Aklapper removed subscribers: Anomie, • Fabrice_Florin, • wikibugs-l-list.Oct 16 2020, 5:01 PM
Krinkle mentioned this in T299764: Document media recovery use case proposals and decide their priority.Jun 3 2022, 6:53 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits