In T292220#7738863, @lbowmaker wrote:

@Clarakosi - ok to move this one to done?

The changes were included as part of algorithm v2 changes: https://gitlab.wikimedia.org/gmodena/ImageMatching/-/merge_requests/33

In T296425#7550682, @ssastry wrote:

@Clarakosi @Pchelolo Is it possible to purge storage only for some wikis vs. all wikis?

Has been merged into the ImageMatching repo on GitLab: https://gitlab.wikimedia.org/gmodena/ImageMatching/-/merge_requests/33

@EBernhardson The updated data can be found under clarakosi.search_imagerec. Please let me know if you have any problems accessing the table.

I've updated the task to move the remaining work to T280585.

In T293808#7449413, @Eevans wrote:

...
Looking at the Superset link referenced earlier in that thread, is it fair to assume it is canonical, and imagerec_prod.tar.bz2 is out of date?

In T293808#7449369, @lbowmaker wrote:

@Clarakosi @gmodena - Are you able to answer the first question above? Seems like there isn't a good reason to store page title? Not sure if there was any reasoning to it in the original requirements.

Moved model and dataset versioning to T292740

In T285816#7272823, @EBernhardson wrote:

Did a test run, this looks to fit in and process as expected. Once we have a final dataset we should be able to ship it.

In T285816#7214472, @dcausse wrote:

[...]

• use a hive table with a schema similar to what we already read (wikiid, page_id, page_namespace, recommendation_type + date partition) to inject to the search indices. One unscheduled airflow dag could be created to fetch a specific partition that you would have populated. Transforming this to be more regular will then just be a matter of scheduling this dag.

Analysis can be found here: https://github.com/clarakosi/ImageMatching/tree/metrics-templae/dataset_metrics/dataset_template#tools-and-approaches-for-generating-dataset-metrics

Reevaluate estimation in a week.

The algorithm notebook and pipeline have been updated to filter out placeholder images using @AikoChou changes.

We've implemented the filtering for disambiguation, lists, and year pages. Below you can find the early rough metrics for this filtering.

In T277557#6956618, @Cparle wrote:

Is this the final ticket for removing disambiguation pages from the results of the api?

PR at https://github.com/mirrys/ImageMatching/pull/9

The RESTbase change we spoke of has been merged but still needs to be deployed: https://github.com/wikimedia/restbase/pull/1288

Created a PR for the data metrics change: https://github.com/mirrys/ImageMatching/pull/8

@Clarakosi I see you did https://github.com/wikimedia/oauth2-server/commits/update_v9.0.0-alpha... Is that actually ready to go? :)

Moving this task to the review column for PM review/signoff before marking it as resolved

@kevinbazira thanks for your work on this!

The patch from T239523 should fix this but moving this to waiting for deployment until we can confirm

In T120242#6741183, @Milimetric wrote:

@Clarakosi: I think @Ottomata meant to ping you above, adding here.

Marking as resolved but please feel free to reopen if we missed anything

Thanks, @Ammarpad!