As a potential guide, I counted how often a given article-type appears as a link. For example, the most common article-type for links in enwiki is the Wikidata-item Q5 (human) with more than 38M occurrences (i.e. biographies). There are only 639 links to articles that are an instance of "century" (Q578) at rank 9068.

https://docs.google.com/spreadsheets/d/1Q2sSDgX1_QlOOzxxRjdMHUaDkd50HWiVjH4MYXh7VWw/edit#gid=1556337929

I dug a bit deeper into the wikidata-ontology to identify article-types related to the calendar such as years or centuries.

The following list are Wikidata-items which characterize Wikipedia-articles related to calendar events, that is the example-article is an instance of the article-type.

• Day: Q573
  • https://en.wikipedia.org/wiki/Saint_Patrick%27s_Day
• Year: Q577
  • https://en.wikipedia.org/wiki/1900
• Decade: Q39911
  • https://en.wikipedia.org/wiki/2010s
• Century: Q578
  • https://en.wikipedia.org/wiki/20th_century
• Millenium: Q36507
  • https://en.wikipedia.org/wiki/2nd_millennium
• Calendar day of a given year Q47150325
  • https://de.wikipedia.org/wiki/Februar_2008
• Calendar month of a given year Q47018478
  • https://en.wikipedia.org/wiki/March_1936
• Calendar year: Q3186692
  • https://en.wikipedia.org/wiki/2015

If we wanted to filter links to articles related to calendar events, this would be the relevant items to filter.

@MGerlach -- thank you for creating this task. Could you imagine these types of filters being configured downstream at the wiki-level? For instance, if one community did want to link to dates, and another did not? Or does it need to happen upstream at model training?

cc @Urbanecm_WMF regarding T274520: Move Growth configuration to on-wiki JSON file

In T279434#6977396, @MMiller_WMF wrote:

@MGerlach -- thank you for creating this task. Could you imagine these types of filters being configured downstream at the wiki-level? For instance, if one community did want to link to dates, and another did not? Or does it need to happen upstream at model training?

@MMiller_WMF : at the moment, this filter is defined globally for all wikis in the training pipeline (it is set manually here). We take a simple approach by just removing links matching the filter from the set of candidate-links before training so they will never be considered. I think we could move this filter further downstream, if needed. This would likely require some engineering work since we then need to create and move an additional table such that we can do the lookups for the filter when generating the recommendations (e.g. is the recommended link an instance of century "Q578"). However, the advantage would be that, with this approach, each wiki could then define their own custom filter in a straightforward way by simply providing a list of entity-types that they dont want to link to (e.g. extending or shortening the list on "calendar-related articles" T279434#6977310).

All dates of the year like "12 May" have to be filtered. They are subclasses of Q14795564.

In T279434#7083447, @geraki wrote:

All dates of the year like "12 May" have to be filtered. They are subclasses of Q14795564.

@geraki thanks for this catch.

@kostajh if you think dates are an issue, I can submit a patch to update the list of instances to be included in the corresponding code that filters the anchor-dictionary. However, this would require us to retrain the models to take effect in the API, so we might want to defer to later.

For reference, we currently filter the following instances as qids:

• Wikimedia disambiguation page (Q4167410)
• Wikimedia list article (Q13406463)
• Wikimedia pages related to time/dates:
  • Year (Q577)
  • Calendar year (Q3186692)

Regarding the time/date-pages, we would want to expand this list to include the following qids:

• Day: (Q573)
• Year: (Q577)
• Decade: (Q39911)
• Century: (Q578)
• Millenium: (Q36507)
• Calendar day of a given year: (Q47150325)
• Calendar month of a given year: (Q47018478)
• Calendar year: (Q3186692)
• Calendar date: (Q205892)
• Point in time with respect to recurrent timeframe: (Q14795564)

In T279434#7093628, @MGerlach wrote:

In T279434#7083447, @geraki wrote:

All dates of the year like "12 May" have to be filtered. They are subclasses of Q14795564.

@geraki thanks for this catch.

@kostajh if you think dates are an issue, I can submit a patch to update the list of instances to be included in the corresponding code that filters the anchor-dictionary. However, this would require us to retrain the models to take effect in the API, so we might want to defer to later.

For reference, we currently filter the following instances as qids:

• Wikimedia disambiguation page (Q4167410)
• Wikimedia list article (Q13406463)
• Wikimedia pages related to time/dates:
  • Year (Q577)
  • Calendar year (Q3186692)

Regarding the time/date-pages, we would want to expand this list to include the following qids:

• Day: (Q573)
• Year: (Q577)
• Decade: (Q39911)
• Century: (Q578)
• Millenium: (Q36507)
• Calendar day of a given year: (Q47150325)
• Calendar month of a given year: (Q47018478)
• Calendar year: (Q3186692)
• Calendar date: (Q205892)
• Point in time with respect to recurrent timeframe: (Q14795564)

cc @MMiller_WMF, should we have another pass at updating the time/date pages?

Some examples of suggested articles on days of the year:

1. এরোমাঙ্গা সেনসেই: suggesting ১০ নভেম্বর (November 10), ৯ এপ্রিল (April 9), ২৫ জুন (June 25)
2. মার্গারিটা সালাস: suggesting ৩০ নভেম্বর (November 30), ৭ নভেম্বর (November 7)
3. ডালিয়া গ্রাইবস্কেইট: suggesting ১ মার্চ (March 1)
4. ইপ্সিতা পাটি: suggesting ১৮ জুন (June 18)

Figure from example 2:

@MGerlach @MMiller_WMF, here are some examples from bnwiki. :)

In T279434#7112518, @Ankan_WMF wrote:

Some examples of suggested articles on days of the year:

1. এরোমাঙ্গা সেনসেই: suggesting ১০ নভেম্বর (November 10), ৯ এপ্রিল (April 9), ২৫ জুন (June 25)
2. মার্গারিটা সালাস: suggesting ৩০ নভেম্বর (November 30), ৭ নভেম্বর (November 7)
3. ডালিয়া গ্রাইবস্কেইট: suggesting ১ মার্চ (March 1)
4. ইপ্সিতা পাটি: suggesting ১৮ জুন (June 18)

Figure from example 2:

I can provide more examples if needed.

Thanks for these examples. All examples are instances of "Point in time with respect to recurrent timeframe: (Q14795564)". Thus updating the filter for the anchor-dictionary as suggested above would remove those recommendations from the updated model.

@MMiller_WMF it sounds like we should update the list of filters in the mwaddlink tool and update the datasets. That process took about a week of processing time last time around. It will then take some weeks for the older recommendations to cycle out of the MediaWiki cache, or we could write a script to remove them and generate new ones. Let me know how you'd like to proceed, please.

Hello @MGerlach, some suggestions are for dates in cswiki. Here are some examples:

• in Let Korean Air 858, "27. července" was suggested to link to 27. červenec (an article about date), "12. listopadu" was suggested to link to "12. listopad" (an article about date) and "18. listopadu" was suggested to link to "18. listopad"
• in "Letiště Edinburgh", "60. let" was suggested to link to "1960-1969" (an article about a decade) and "20. století" was suggested to link to "20. století" (an article about a century)
• in "Eduard Pagáč", 8. května" was suggested to link to "8. květen" (an article about a date) and "12. března" was suggested to link to "12. březen" (an article about a date).

Could those be excluded, please? Thanks!

In T279434#7124800, @Urbanecm_WMF wrote:

Hello @MGerlach, some suggestions are for dates in cswiki. Here are some examples:

• in Let Korean Air 858, "27. července" was suggested to link to 27. červenec (an article about date), "12. listopadu" was suggested to link to "12. listopad" (an article about date) and "18. listopadu" was suggested to link to "18. listopad"
• in "Letiště Edinburgh", "60. let" was suggested to link to "1960-1969" (an article about a decade) and "20. století" was suggested to link to "20. století" (an article about a century)
• in "Eduard Pagáč", 8. května" was suggested to link to "8. květen" (an article about a date) and "12. března" was suggested to link to "12. březen" (an article about a date).

Could those be excluded, please? Thanks!

@Urbanecm_WMF thank you for pointing out those examples. All cases you mention should be excluded with the update suggested in T279434#7093628 since the wikidata-items of the articles recommended as links are all instances of i) Point in time with respect to recurrent timeframe: (Q14795564), or ii) Decade: (Q39911), or iii) Century: (Q578). Though, the update has not yet been deployed.

@MMiller_WMF I think the next step here is to make sure we have a consolidated list of all the Wikidata items we want to exclude, then retrain the models and rebuild datasets for all the wikis. Last time around, the rebuild process took a lot longer than we hoped due to various technical snags; in theory it should be smoother this time around. But it will still take several days to a week due to the time needed for retraining and rebuilding datasets for each wiki.

Because this blocks us deploying to more wikis (T284481), we want to prioritize it. @kostajh -- is this something that Growth engineers do, or that @MGerlach does?

In T279434#7140075, @MMiller_WMF wrote:

Because this blocks us deploying to more wikis (T284481), we want to prioritize it. @kostajh -- is this something that Growth engineers do, or that @MGerlach does?

@MMiller_WMF I will update the algorithm to include the suggested changes and re-run the model for the pilot wikis this week. work on other wikis will be captured in T284481

A possible Wikidata item to consider for rejection is any article defined as events in a specific year or time period (Q18340514). These links are tricky to use, even when being an experienced user: you have an article about a given train station. Should the construction date link to the city or the railway?

Based on the suggestions in T279434#7143095 T279434#7093628 T284666 I am adding the following entity-types to the filter (all links that are an instance of these entities are removed from the set of candidates):

• day, example: Saint Patrick's Day
• decade, example: 2010s
• century, example: 20th century
• millenium, example: 2nd millennium
• calendar day of a given year, example: Portal:Current_events/2006_February_4
• calendar month of a given year, example: August 1936
• calendar date, example: Ides of March
• point in time with respect to recurrent timeframe, example: January 1
• events in a specific year or time period, example 1881 in baseball
• Unit of length, example: meter
• Unit of time, example: day
• Unit of volume, example: litre
• Unit of mass, example: kilogram
• Unit of energy, example: joule
• Unit of information, example: byte
• Currency, example: US Dollar
• Parts-per notation, example: percent

I am starting to re-run the models and update the datasets. Let me know if you want to add further items or remove some from this list.

In T279434#7151066, @MGerlach wrote:

Based on the suggestions in T279434#7143095 T279434#7093628 T284666 I am adding the following entity-types to the filter (all links that are an instance of these entities are removed from the set of candidates):

• day, example: Saint Patrick's Day
• decade, example: 2010s
• century, example: 20th century
• millenium, example: 2nd millennium
• calendar day of a given year, example: Portal:Current_events/2006_February_4
• calendar month of a given year, example: August 1936
• calendar date, example: Ides of March
• point in time with respect to recurrent timeframe, example: January 1
• events in a specific year or time period, example 1881 in baseball
• Unit of length, example: meter
• Unit of time, example: day
• Unit of volume, example: litre
• Unit of mass, example: kilogram
• Unit of energy, example: joule
• Unit of information, example: byte
• Currency, example: US Dollar
• Parts-per notation, example: percent

I am starting to re-run the models and update the datasets. Let me know if you want to add further items or remove some from this list.

@MGerlach sounds good to me, but please don't run ./publish-datasets.sh just yet, I'd like to be able to do some sample queries for @MMiller_WMF and our ambassadors before we publish the new datasets.

@kostajh ack.

Change 699395 had a related patch set uploaded (by Kosta Harlan; author: MGerlach):

[research/mwaddlink@main] Update the filter for removing candidate links

https://gerrit.wikimedia.org/r/699395

I trained the model with the updated filter for the 4 initial wikis (ar,bn,cs,vi), the 7 new wikis mentioned in T284481, and enwiki (for comparison). The backtesting evaluation suggests that performance is ok (precision is similar, recall decreases slightly); see detailed results on the project-page on meta.

The data/model is available on the stat-machines and can be tested locally. We should test the link recommendations of the new model for some of the examples mentioned by the users, e.g. T279434#7124800 or T284666.

Change 699395 merged by jenkins-bot:

[research/mwaddlink@main] Update the filter for removing candidate links

https://gerrit.wikimedia.org/r/699395

In T279434#7124800, @Urbanecm_WMF wrote:

Hello @MGerlach, some suggestions are for dates in cswiki. Here are some examples:

• in Let Korean Air 858, "27. července" was suggested to link to 27. červenec (an article about date), "12. listopadu" was suggested to link to "12. listopad" (an article about date) and "18. listopadu" was suggested to link to "18. listopad"

Before:

{
  "links": [
    {
      "context_after": " 1953.\n\nAg",
      "context_before": " z ",
      "link_index": 0,
      "link_target": "27. červenec",
      "link_text": "27. července",
      "match_index": 0,
      "score": 0.7580083012580872,
      "wikitext_offset": 942
    },
    {
      "context_after": " 1987 vyce",
      "context_before": "\nDne ",
      "link_index": 1,
      "link_target": "12. listopad",
      "link_text": "12. listopadu",
      "match_index": 0,
      "score": 0.6342907547950745,
      "wikitext_offset": 2337
    },
    {
      "context_after": " agenti od",
      "context_before": " dní. Dne ",
      "link_index": 2,
      "link_target": "18. listopad",
      "link_text": "18. listopadu",
      "match_index": 0,
      "score": 0.5812450647354126,
      "wikitext_offset": 2608
    },
    {
      "context_after": " úřadující",
      "context_before": "okorejský ",
      "link_index": 3,
      "link_target": "Diplomat",
      "link_text": "diplomat",
      "match_index": 0,
      "score": 0.6413889527320862,
      "wikitext_offset": 5291
    },
    {
      "context_after": " 1987 byla",
      "context_before": " z ",
      "link_index": 4,
      "link_target": "15. prosinec",
      "link_text": "15. prosince",
      "match_index": 0,
      "score": 0.6673643589019775,
      "wikitext_offset": 5551
    },
    {
      "context_after": ". Poté, co",
      "context_before": "hudobou a ",
      "link_index": 5,
      "link_target": "Korupce",
      "link_text": "korupcí",
      "match_index": 0,
      "score": 0.5933520197868347,
      "wikitext_offset": 6364
    },
    {
      "context_after": ", o samotn",
      "context_before": ") a v ",
      "link_index": 6,
      "link_target": "Evropa",
      "link_text": "Evropě",
      "match_index": 0,
      "score": 0.5208941102027893,
      "wikitext_offset": 10353
    }
  ],
}

After:

{
    "links": [
        {
            "context_after": " \u00fa\u0159aduj\u00edc\u00ed",
            "context_before": "okorejsk\u00fd ",
            "link_index": 0,
            "link_target": "Diplomat",
            "link_text": "diplomat",
            "match_index": 0,
            "score": 0.6219052076339722,
            "wikitext_offset": 5291
        },
        {
            "context_after": ". Pot\u00e9, co",
            "context_before": "hudobou a ",
            "link_index": 1,
            "link_target": "Korupce",
            "link_text": "korupc\u00ed",
            "match_index": 0,
            "score": 0.5970041751861572,
            "wikitext_offset": 6364
        },
        {
            "context_after": ", o samotn",
            "context_before": ") a v ",
            "link_index": 2,
            "link_target": "Evropa",
            "link_text": "Evrop\u011b",
            "match_index": 0,
            "score": 0.5348489880561829,
            "wikitext_offset": 10353
        }
    ],
}

• in "Letiště Edinburgh", "60. let" was suggested to link to "1960-1969" (an article about a decade) and "20. století" was suggested to link to "20. století" (an article about a century)
• in "Eduard Pagáč", 8. května" was suggested to link to "8. květen" (an article about a date) and "12. března" was suggested to link to "12. březen" (an article about a date).

Could those be excluded, please? Thanks!

The datasets are being imported in production, and new suggestions can be checked in a day or two on https://api.wikimedia.org/service/linkrecommendation/apidocs/