Wikidata pays a lot of tribute to authority control, linking to all kinds of datasets and databases with various IDs. The holy grail of every GLAM worker Sum of All People, with links to their Works is coming about!

But we’re just at the start of a lot of work in that direction. The purpose of this project is to try and coordinate such work.

I know a few things (ViafBot, Mix-n-Match) and I'd like to help with some things, but I don't know what others are doing. --Vladimir Alexiev (talk) 08:08, 27 January 2015 (UTC)[reply]

VIAF information has been moved to Wikidata:VIAF.

The report Name Data Sources for Semantic Enrichment shows that when it comes to name data sources, maybe the two that matter are VIAF and Wikidata.

• Their name coverage is fairly orthogonal: VIAF has more name variations and permutations, Wikidata has more translations (Venn diagram of names for Cranach).
• VIAF is much bigger: 35M persons/orgs. Wikidata has 2.7M persons and maybe 1M orgs.
• Only 0.5M of Wikidata persons/orgs are coreferenced to VIAF, with maybe another 0.5M coreferenced to other datasets, either VIAF-constituent (eg GND) or non-constituent (eg RKDartists). So coreferenced part between the two is still quite small (30%) and a lot of work remains!
• A lot can be gained by leveraging coreferencing across VIAF and Wikidata: finding errors in Authority files, finding merge candidates in Wikidata, promulgating identifiers...
• Wikidata has great tools for crowd-sourced coreferencing.

Please comment!

RKDartists is an important Authority that does not yet participate in VIAF. There are already 21760 RKDartist id's on Wikidata. These could be imported to VIAF for free!

The BM has several thesauri that are not co-referenced to anything in the world. I think they'd see it as a major win if the community helps them to co-reference.

• British Museum person or institution ID (P1711): 176461 persons, 21511 matched
• Wikidata:Property_proposal/British_Museum_place: 45883 places
• Wikidata:Property_proposal/British_Museum_thesauri: 28 more thesauri with 26804 entries

This could be followed by importing the 2.5M cultural objects of the BM.

ULAN does record possible matches and mismatches in their editorial system: ULAN Artists Whose Identity May be Associated or Confused With Another (608 pairs).

Looks like this:

Here's to proper coreferencing! --Vladimir Alexiev (talk) 18:07, 12 March 2015 (UTC)[reply]

We should match persons to persons, not disambiguation pages to persons or other disambiguation pages.

Wikipedias, GND and RKD all have disambiguation pages (in GND they are called "undifferentiated names"). 13 Feb 2015:

• Wrote to Jane
• Wrote to Magnus: Filter out Disambiguation entries and Un-notable Persons

Do you agree with my reasoning:

• Jane said "any match is better than none"
• I countered "A correct match is better than none"
• the only way to make sure it's correct is to examine more data about the person, which will necessarily lead you to a real person page.
• Look at the ULAN data above: that's good data that gives you some basis for decision. A name alone does not.

--Vladimir Alexiev (talk) 19:34, 12 March 2015 (UTC)[reply]

 Support @Vladimir Alexiev, Randykitty, Ghuron: A GND Tn (Thesaurus name = undifferentiated) is not a stable disambiguation page. A Tn is a placeholder. It can be deleted, it can be upgraded into a Tp (Thesaurus person), or changed into a redirect. Works connected with a Tn will be checked by the library or archive who owns them and afterwards might be delinked. The database Online GND (OGND) includes only Tp numbers. --Kolja21 (talk) 00:04, 28 March 2015 (UTC)[reply]

AAT is a crucial thesaurus in cultural heritage.

• It has 40k concepts, see http://vocab.getty.edu/sparql

    select (count(*) as ?c) {
      ?x a skos:Concept; skos:inScheme aat: }

• Of them only 363 AAT are coreferenced, or under 1%, see http://tools.wmflabs.org/autolist/autolist1.html?q=CLAIM%5B1014%5D

I think that's BAD. I'm sure going to need that coref for the Europeana Food and Drink Classification Scheme that will be based on Wikidata and AAT:

• presentation
• report

Update: the AAT-Wordnet coreference described below is brought into Wikidata. AAT is actively coreferenced on Mix-n-Match: 12985 (32%) matched, 3293 (8%) awaiting confirmation, 1553 (3.8%) confirmed no-matches, and 22543 (55.7%) awaiting matching. So it's way better than 2 years ago. Help coreference this pivot thesaurus that is of immense importance for Cultural Heritage! --Vladimir Alexiev (talk) 15:04, 19 September 2017 (UTC)[reply]

The Wikidata coref tool Mix-n-Match has mostly been used for people until now. But I hope it can be used for concepts as well.

I made an export that includes AAT URL, preferred English label (without qualifier), parents (ascendants to root) and scope note (description). Could also add alternative labels, and labels in other languages (Dutch, Spanish, Chinese).

select ?id (str(?lab) as ?label) ?parents (str(?scopeNote) as ?note) {
  ?x a gvp:Concept; dc:identifier ?id; gvp:prefLabelGVP/gvp:term ?lab;
     gvp:parentString ?parents.
  optional {?x skos:scopeNote [dct:language gvp_lang:en; rdf:value ?scopeNote]}
}

I saved as XML then converted to TDV: aat.rar.

    rset --results tsv aat.xml > aat.tdv

Also see https://meta.wikimedia.org/wiki/Talk:Mix%27n%27match#Coreference_AAT !!!

Mix-n-Match has good automatic matching, but that works for people.

So let's check what other vocabs that are coref to AAT may be coref to Wikidata: According to Michiel Hildebrand's famous CH LOD diagram:

• Wordnet. No such prop in Wikidata
• I'd guess Wiktionary is coref to Wordnet, but Wikidata got no site links to Wiktionary
• RKD Concepts. There's prop "RKDartists" and "RKDimages" but none for concepts
• Rijksmuseum Concepts. There's "Rijksmonument" but none for concepts
• Joconde: aha! There's Joconde work ID (P347), and it has 2275 instances, so that's better. Joconde is 18% coref to Wikidata but I don't know how much to AAT, maybe I can gain 1k here.
  • Looked at the results: nope, Joconde are all paintings, not concepts
• Bibliopolis: never heard of it, and nope
• SVCN: never heard of it, and nope

Then it dawns on me.

• BabelNet corefs Wordnet and Wikipedia! In fact, it corefs to Wikidata, see http://babelnet.org/stats. (Very useful stat: http://babelnet.org/stats#Numberofpolysemousandmonosemouswords)
• Some months ago they didn't have RDF access. But they do now: http://babelnet.org/download (not download, but API: good enough)
• On the web view of chipotle in Sources I see Wikidata
• On the RDF view http://babelnet.org/rdf/page/s00018522n I don't see Wikidata but I see DBpedia, so that's ok

Ok, so off to look for that AAT-Wordnet coref. - Why yes, it's part of http://semanticweb.cs.vu.nl/europeana/skos/browse/ - I got a file from somewhere that says

<aat_wordnet20_mappings>
  a void:Linkset;
    dcterms:title "AAT-Wordnet 2.0 mappings by Anna Tordai (baseline)" ;
    lib:source <http://semanticweb.cs.vu.nl/lod/getty/aat/> ;
    void:dataDump <bl_aat_wn.rdf> , <bl_norm_aat_wn.rdf> , <bl_sing_aat_wn.rdf> .

(Note: you can get those files from URLs like: http://semanticweb.cs.vu.nl/europeana/api/export_graph?graph=http://semanticweb.cs.vu.nl/lod/getty/aat/bl_sing_aat_wn.rdf&mimetype=text/plain&format=turtle)

These are called "baseline" (i.e. mostly literal matches). A quick conversion to Turtle and a line count:

$ wc -l bl*
   2300 bl_aat_wn.ttl
   4369 bl_norm_aat_wn.ttl
   4303 bl_sing_aat_wn.ttl
  10972 total

Run a query at http://semanticweb.cs.vu.nl/europeana/user/query (specify entailment=None or else!):

prefix getty:  <http://purl.org/vocabularies/getty/> 
prefix aat:  <http://purl.org/vocabularies/getty/aat/> 
select * {?x skos:inScheme getty:aat; skos:closeMatch ?y}

It returns 4592 (see below why).

There is significant overlap between the files:

$ cat bl* | sort| uniq | wc -l
4596
$ cat bl* |cut -d " " -f 1 | sort| uniq | wc -l
4581

The following AAT concepts have 2 matches:

aat:bleachers
aat:boxcars
aat:cleavers
aat:feudalism
aat:groats
aat:jackstraws
aat:lats
aat:leotards
aat:morocco
aat:ninepins
aat:quoits
aat:shekels
aat:stairs

We need to reconcile them manually, eg

aat:bleachers  skos:closeMatch  <http://www.w3.org/2006/03/wn/wn20/instances/synset-bleacher-noun-1> .
aat:bleachers  skos:closeMatch  <http://www.w3.org/2006/03/wn/wn20/instances/synset-bleachers-noun-1> .

The AAT definition is:

• aat:bleachers vp:descriptiveNote "Use for benchlike tiered seating for spectators at, for example, outdoor sporting events, usually without weather or sun protection, affording less advantageous views than grandstands; may also be used for similarly constructed, often telescoping, indoor seating."@en .
• Inspection at Wordnet 3.1 shows that the second one is right.

That's 4.6k matches, or 11% of AAT.

The coref looks like this:

    aat:wrought_iron skos:closeMatch <http://www.w3.org/2006/03/wn/wn20/instances/synset-wrought_iron-noun-1> .

And there's another file aat.ttl with rep like:

aat:wrought_iron aat:parentPreferred aat:iron_alloy .
aat:wrought_iron vp:id "300011012" .
aat:wrought_iron vp:labelPreferred "wrought iron"@en .
aat:wrought_iron vp:labelNonPreferred "iron, wrought"@en .
aat:wrought_iron vp:labelNonPreferred "wrought-iron"@en .

This is quite old rep. The new rep uses numeric URL: http://vocab.getty.edu/aat/300011012 (and a bunch more data). So we need to construct a numeric URL.

Cornetto is NL Wordnet and AATNED is NL AAT. I got another file saying:

<aatned_cornetto_mappings>
  a void:Linkset ;
    dcterms:title "AATNED-Cornetto mappings by Anna Tordai (baseline)";
    lib:source <http://semanticweb.cs.vu.nl/lod/rkd/aatned/> ;
    void:dataDump <bl_aatned_cn.rdf.gz> , <bl_norm_aatned_cn.rdf.gz> , <bl_sing_aatned_cn.rdf.gz> .

Eg we have this for AAT 300191645 "salinity":

bl_aatned_cn.ttl: aatned:zoutheid  skos:closeMatch  cornetto:synset-zoutheid-1-noun .
cornetto-wn20.ttl: cornetto:synset-zoutheid-1-noun cornetto:eqNearSynonym instances:synset-brininess-noun-1 .
cornetto-wn30.ttl: cornetto:synset-zoutheid-1-noun cornetto:eqNearSynonym wn30:synset-brininess-noun-1 .
aatned.ttl: aatned:zoutheid core:notation "300191645" .

The number of AATNED-Cornetto matches is as follows:

> cat bl*|sort|uniq> bl_aatned_all.ttl
> wc -l bl_aatned_all.ttl
6917 bl_aatned_all.ttl
> cat bl_aatned_all.ttl|cut -d " " -f 1 | sort| uniq | wc -l
6857

There are more matches than AAT-Wordnet. There are also overlaps: 60 AATNED concepts (0.9%) have two Cornetto matches.

We need to merge AATNED-Cornetto with AAT-Wordnet. The correlation is simply by id, eg

aatned.ttl: aatned:zwerfkeien core:notation "300011671"
aat.ttl: aat:boulder vp:id "300011671"

I guess the overlaps between them are quite big, eg for wrought_iron:

aatned.ttl: aatned:smeedijzer core:notation "300011012"
bl_aatned_all.ttl: aatned:smeedijzer  skos:closeMatch  cornetto:synset-smeedijzer-1-noun .
cornetto-wn20.ttl:cornetto:synset-smeedijzer-1-noun cornetto:eqNearSynonym instances:synset-wrought_iron-noun-1 .
cornetto-wn30.ttl:cornetto:synset-smeedijzer-1-noun cornetto:eqNearSynonym wn30:synset-wrought_iron-noun-1 .

The other problem is bigger:

• http://www.w3.org/2006/03/wn/wn20/instances/synset-wrought_iron-noun-1 is Wordnet 2.0 in 9-year old rep, with wn20schema:synsetId "113958999"
• while BabelNet: http://babelnet.org/rdf/s00081730n has

    bn:s00081730n skos:exactMatch dbpedia:Wrought_iron, lemon-WordNet:wn30-14802262-n

• that uses a modern LEMON wordnet rep: http://lemon-model.net/lexica/pwn/wn30-14802262-n
• Note: Wordnet 3.1 has IDs like "14826432" and "wrought_iron%1:27:00::"

It doesn't look like Wordnet3 and Wordnet2 share any IDs; we'll deal with that in next section.

Lets first do some queries at http://babelnet.org/sparql/ to see what we can see. Look for DBpedia-Wordnet matches:

SELECT * WHERE {
  ?x skos:exactMatch ?y, ?z 
  filter(strstarts(str(?y),"http://dbpedia.org/resource/"))
  filter(strstarts(str(?z),"http://lemon-model.net/lexica/pwn/"))
} LIMIT 30

Download: https://www.dropbox.com/s/92gq5r1qm3yytkp/WN3toDBP.csv?dl=1.

It has 47607 rows like this (there's a decent chance this will cover the 6k AAT matches):

"http://babelnet.org/rdf/s00075206n","http://dbpedia.org/resource/Sundowner_(drink)","http://lemon-model.net/lexica/pwn/wn30-07913081-n"
"http://babelnet.org/rdf/s00039711n","http://dbpedia.org/resource/Sonora_(genus)","http://lemon-model.net/lexica/pwn/wn30-01736256-n"
"http://babelnet.org/rdf/s00070026n","http://dbpedia.org/resource/Sealskin","http://lemon-model.net/lexica/pwn/wn30-04160261-n"

Since Wordnet3 and Wordnet2 don't share any IDs, we can try to use Wordnet2-Wordnet3 coref made by Jacco van Ossenbruggen and Marc van Assem (VU University Amsterdam) in May 2010 with this VOID (manifest):

<wn30-wn20-mappings-jacco>
	a void:Linkset ;
	dcterms:title "synset-level mappings from Wordnet 3.0 to 2.0, created by jacco's code" ;
	lib:source <http://purl.org/vocabularies/princeton/wn30/> ;
	void:dataDump
		<label-child-matches.ttl.gz> ,
		<label-childparent-matches.ttl.gz> ,
		<label-instance-matches.ttl.gz> ,
		<label-meronym-matches.ttl.gz>,
		<label-neargloss-matches.ttl.gz> ,
		<label-parent-matches.ttl.gz> ,
		<label-unique-matches.ttl.gz> ,
		<nearlabel-matches.ttl.gz> ,
		<glossmatches-m.ttl.gz> .

<wn30-wn20-mappings-sense>
	a void:Linkset ;
	dcterms:title "synset-level mappings from Wordnet 3.0 to 2.0, created by Mark using the Princeton WordSense mappings" ;
	lib:source <http://purl.org/vocabularies/princeton/wn30/> ;
	void:dataDump
		<synset-matches-based-on-multiple-sense-mappings-princeton.ttl.gz> ,
		<synset-matches-based-on-single-sense-mappings-princeton.ttl.gz> .

It's a complex affair consisting of many steps, but the major step (contributing 87% of all matches) is glossmatches-m.ttl that looks like

    wn30:synset-wrought_iron-noun-1 terms:replaces instances:synset-wrought_iron-noun-1 .

And looking at wordnet-synset.ttl, we find the required wn30 ID:

    wn30:synset-wrought_iron-noun-1 wn20schema:synsetId 114802262 .

After much querying and manual cleaning (over a day of effort), I made some sheets in this google folder:

• AAT-DBpedia-Babelnet.xlsx: 3324 potential matches, fairly clean, but need checking by more people
• AAT-DBpedia-Babelnet-80-judged.xlsx: example of correct & incorrect matches
• AAT-Wikidata-25-judged.xlsx: example of correct & incorrect matches on Mix-n-Match

Notified participants of WikiProject Authority control: I need your help!

Notified participants of WikiProject Visual arts: Yours too!

• Do some checks (add your initials in column "check")
• Add Q numbers to the sheet
• Merge WD items that already have Art & Architecture Thesaurus ID (P1014) (there are 8477) to the sheet to compare the matches (or remove them from the sheet if you're quite confident)

I could post the sheet as QuickStatements, but I think there are still 10% incorrect matches, especially for Styles and Periods (see Wikidata talk:WikiProject Visual arts/Item structure/Art movements. --Vladimir Alexiev (talk) 16:02, 7 March 2017 (UTC)[reply]

445 AAT-LCSH coreferences made by Getty editors.

400 of them are on the Getty LOD site (see query below), 45 are newly extracted

select * {
  ?x skos:exactMatch|skos:closeMatch ?y.
  ?x skos:inScheme aat:
  filter not exists {?y skos:inScheme aat:}}

Notified participants of WikiProject Authority control

Notified participants of WikiProject Companies WikiProject Cultural heritage has more than 50 participants and couldn't be pinged. Please post on the WikiProject's talk page instead.

Notified participants of WikiProject Visual arts

GeoNames feature code (P2452) is applied only 33 times (see disscussion and archive, while there are 669 codes on Geonames. I'll ask Magnus to add the Geonames list to Mix-n-Match.

Update Jul 2020: Things are much better at https://www.wikidata.org/wiki/Property:P2452:

• Fully matched 545 82.4%
• Preliminarily matched 7
• Unmatched 109 16.4%
• Total 661
• The actual total at http://www.geonames.org/export/codes.html is 680. --Vladimir Alexiev (talk) 12:31, 15 July 2020 (UTC)[reply]

http://www.geonames.org/ontology/mappings_v3.01.rdf has the following mappings:

     32 dbo    http://dbpedia.org/ontology/
      5 frgeo  http://rdf.insee.fr/geo/
     79 lgdo   http://linkedgeodata.org/ontology/
     31 schema http://schema.org/

Can we use them somehow to push this coreferencing further?

I'm a little wary about importing these wholescale, because geonames in not a CC0 database. It's one thing to be providing external links to GeoNames, it's another to be importing data.

I did look at these values recently for English places with geonames links that are marked as both village (Q532) and civil parish (Q1115575) (see eg Abberton (Q3137539) for an example), to identify which Geonames link corresponded to which role; but I purposely decided not to add a GeoNames feature code (P2452) statement.

It might be useful to be able to map the codes to Q-numbers here, to facilitate sanity checking of exisiting or proposed co-references. However, even then there are difficulties -- for example, I found that PPLA3 or PPLA4 at Geonames didn't necessarily match to distinctions we would want to make in a instance of (P31) here. Jheald (talk) 14:02, 13 March 2017 (UTC)[reply]

Data from DifferentiatedPersons of GND can be used to fill missing properties of according items, e.g.,

• date of birth/death (directly from gnd:dateOfBirth and gndo:dateOfDeath, for entries following YYYY or YYYY-MM-DD - everything else to be skipped)
• affiliation (P1416) can be obtained by a join of gndo:affiliation to wd organizations (may be sparse currently, but can be repeated later on)
• country (country (P17) or country of citizenship (P27)??) requires translation from gndo:geographicAreaCode, which refers to a customized code table derived from ISO 3166 (not part of GND) (table (pdf), rules)
• aliases - require filtering of gndo:variantNameForThePerson, which carry no language tag, re. script and presumed language (would Lingua::Identify work here?)

For appropriate source statements see project chat -- Jneubert (talk) 06:35, 21 May 2017 (UTC) (with thanks to User:MisterSynergy and User:ChristianKl)[reply]

I am a big fan of standards but the ISO 3166 is used for modern countries, it does not as a consequence give the "nationality" of people who did precede a country. Thanks, GerardM (talk) 07:03, 21 May 2017 (UTC)[reply]

NB yes there are some, but at Wikidata we know about many more former countries. GerardM (talk) 07:13, 21 May 2017 (UTC)[reply]

A "sibling" of Mix-n-Match now imports birth/death dates from authority files: https://www.wikidata.org/wiki/User:Magnus_Manske/Mix%27n%27match_date_import. Also see discussion about this in relation to Getty ULAN: https://groups.google.com/forum/#!topic/gettyvocablod/TkdelW9RP1g --Vladimir Alexiev (talk) 09:38, 2 October 2017 (UTC)[reply]

In order be able to map a thesaurus more completely, and - more general - to make Wikidata fit as a linking hub for knowledge organiziation systems, I've proposed a new property which allows to qualify individual links by properties of type "external identifier" as in-exact (close/broad/narrow/related) match.

Please feel free to comment at https://www.wikidata.org/wiki/Wikidata:Property_proposal/mapping_relation_type.

Cheers, Jneubert (talk) 12:27, 28 August 2017 (UTC)[reply]

Notified participants of WikiProject Authority control

There is a new grant proposal soweego for authority control. See discussion at https://meta.wikimedia.org/wiki/Grants:Project/Hjfocs/soweego#Endorsements.

I've considered it seriously but I think it doesn't address the main problem, see a list of 11 considerations (which can also be read as a sort of programme for next important steps for WD authority control). Please express your opinion there. --Vladimir Alexiev (talk) 10:19, 2 October 2017 (UTC)[reply]

Where is this discussion with 11 considerations? Thanks, GerardM (talk) 12:22, 2 October 2017 (UTC)[reply]

Getty Vocabs have 8k obsolete subjects (identifiers). You can find them at http://vocab.getty.edu/sparql with a query like

select * {
  ?old a gvp:ObsoleteSubject; skos:inScheme aat: .
  optional {?old dct:isReplacedBy ?new}
}

Here are the numbers: I can put up the files somewhere: obsolete-AAT-2106.tsv, obsolete-TGN-1016.tsv, obsolete-ULAN-5574.tsv.

Notified participants of WikiProject Authority control

Any takers to replace any old values of AAT ID Search, TGN ID Search, ULAN ID Search with the new values from the respective files?

@Magnus Manske: rdf:type gvp:ObsoleteSubject should be removed from Mix-n-Match consideration.

Done, but only changed 5 entries for the ~2100 obsolete ones, the others were either already N/A or matched with an item. --Magnus Manske (talk) 15:22, 21 February 2018 (UTC)[reply]

Data should not be removed, but - if necessary - marked as deprecated, with a qualifier giving the reason for deprecation. Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits 11:02, 26 February 2018 (UTC)[reply]

Monday june 11 the Swedish National Library is moving in direction Linked data and release a new system called LIBRIS XL ==>

1. We have a new identifier LIBRIS-URI Libris-URI (P5587) in Wikidata

Open issues are if we need to "upgraded" to support the new identifier in other systems. I have found

1. User:Magnus_Manske/authority_control.js
   1. tool to read from VIAF.org and populate WD
2. User:Magnus_Manske/Mix'n'match_date_import
3. wdmapper

See also:

• BIBFRAME in Libris XL
• Datamodel chosen by The National Library of Sweden (KB)
• Presentation from 2017 Bibframe in an European perspective

- Salgo60 (talk) 14:07, 10 June 2018 (UTC)[reply]

Status update 16 September 2018

A project wmse-library-data-2018 done by Alicia_Fagerving_(WMSE) is converting and adding about 60 000 identifiers to Wikidata....

I sent in today a question to VIAF asking about plans they have to start supporting Libris-URI (P5587) - Salgo60 (talk) 06:54, 16 September 2018 (UTC)[reply]

Status update 28 September 2019

• Still no one has specified how LIBRIS <-> VIAF should work together see Phabricator T223259
• We lack a change process for errors in VIAF and LIBRIS see example of error Talk:Q21522286#Summary

- Salgo60 (talk) 18:32, 28 September 2019 (UTC)[reply]

For knowledge organization systems which do not deal with clearly defined entities (such as human (Q5)), you often find in-exact relations between the external entity and matching Wikidata items. E.g., the STW concept Yugoslavia (until 1990) is not an exact match to Yugoslavia (Q36704), which is described as "1918–1992 country in Southeastern and Central Europe". For STW's Executive selection, assessment centre (Q265558) is closely related, but categorically different (process vs. instrument). While in the latter case it might be useful to create an exactly matching item, in the former it clearly would be not.

Within the domain of traditional knowledge organization system, different mapping relations (such as "close match") have been used to cover such situations. The "external id" properties of Wikidata lacked such expressiveness, till the mapping relation type (P4390) property was introduced. The property, to be used exclusively as a qualifier on external-id properties, allows to more precisely define relations as exact match (Q39893449), close match (Q39893184), broad match (Q39894595), narrow match (Q39893967) or related match (Q39894604). These relation types reflect the according SKOS mapping properties.

Since its introduction in October last year, the property has seen some uptake, particularly in the biomedial field. The ongoing mapping of STW Thesaurus for Economics (Q26903352) to Wikidata is based on qualified relations (state of the mapping, shown as SKOS). The effort aims at creating a finally complete mapping of the STW descriptors.

Our - still experimental - workflow for the STW mapping is as follows:

1. Use Mix-n-match catalogs for each sub-thesaurus (#507, #1259/#1260) to assign STW descriptor IDs one-by-one to Wikidata items.

   This sounds simple, yet often reveals qualitiy issues. Some of these, such as obvious duplicates in Wikidata, can be resolved immediately. Sometimes an ugly mess shows up, which can only be reported to be solved by the community later on. Of course, quality flaws in STW might be revealed also. During this step, it may also be advisable to take notes about other items, which are not the closest ones and not linked via external-id entry in Wikidata, but may be worth linking from the side of the external vocabulary.

   • For non-exact relationships, immediately open the newly linked item and manually qualify STW Thesaurus for Economics ID (P3911).
   • For STW descriptors which are lacking a counterpart in Wikidata and would make sense there, add an item semi-automatically, with exact match (Q39893449) to the STW ID and all avaialable information from the thesaurus (more on that later).
2. Assign exact match (Q39893449) to the remaining unqualified STW Thesaurus for Economics ID (P3911) via Quickstatements, with the input produced by a script executing a SPARQL query. This step can be executed multiple times during the mapping process for one sub-thesaurus, in order to keep the list of unqualified entries short.

The sequence within the Mix-n-match input file turned out to be crucial for a smooth one-by-one workflow. We sorted the generated M-n-m input by the minimal notation of attached subject categories for the descriptors, and within that alphabetically.

Maintenance and qualitiy control on a mapping have to take into account that multiple external-id values for a Wikidata item, or one external-id linked to multiple Wikidata items are possible and may perfectly make sense with in-exact mapping relations (e.g., STWs Appenzell is a "broad match" to Appenzell Ausserrhoden (Q12079) and Appenzell Innerrhoden (Q12094)). This is not reflected in the "single value" and "distinct values" constraints. Therefore, we defined a number of QS reports for the STW mapping to catch anomalies specifically in qualified mappings.

We are interested to exchange experiences with others who are mapping KOS to Wikidata, possibly with different workflows or other tools used. The reports and scripts linked above are meant to be customizable, we'd be happy to receive suggestions for improvment or github pull requests. -- Jneubert (talk) 15:09, 29 August 2018 (UTC)[reply]

Akuckartz (talk) 07:29, 9 August 2020 (UTC) JakobVoss (talk) Vladimir Alexiev (talk) 09:27, 5 July 2016 (UTC) TomT0m author  TomT0m / talk page 20:15, 21 September 2016 (UTC) Jneubert (talk) 09:22, 29 January 2018 (UTC) Ettorerizza (talk) 11:00, 26 September 2018 (UTC) ElanHR (talk) 10:01, 17 March 2019 (UTC) Dcflyer (talk) 19:46, 29 March 2019 (UTC) ArthurPSmith (talk) 16:32, 1 September 2022 (UTC) Melmakko (talk) 17:34, 4 November 2022 (UTC) Maxime[reply]

Notified participants of WikiProject KOS

During the above mentioned mapping process from STW Thesaurus for Economics (Q26903352), we sometimes want to create new items in Wikidata. With the "New item" button in Mix-n-match, only rudimentary information can be transferred to the new item. Therefore we generated a list of all not-yet-mapped STW descriptors, formatted for Quickstatements input. It includes labels and aliases (skos:preLabel/skos:altLabel) in all available languages, as well an instance-of economic concept (Q29028649) statement, sourced from the STW descriptor, optionally a link to the according GND concept, derived from the STW/GND mapping, rarely a description (skos:scopeNote), but always of course a STW Thesaurus for Economics ID (P3911) link.

The workflow is - during working through a mix-n-match list - for any missing concept simply to copy & paste the complete set of QS statements into the QS input window, removing aliases which are not appropriate for Wikidata (such as "oil platform" for "offshore industry"), and running the statements. The list is sorted exactly like the mix-n-match list and recreated every hour, so the same items are on top of both lists, and every case solved by either linking or creating an item disappears automatically from the list. In our experiences, this works quite smoothly.

If others want to adapt such a workflow for other vocabularies - thanks to SKOS and LOD standards that shouldn't be too difficult -, here is the script for generating the list, and the query called with it. -- Feedback, as always, welcome. Jneubert (talk) 12:11, 5 September 2018 (UTC)[reply]