[go: up one dir, main page]

Page MenuHomePhabricator

RFC: Future of magic links
Open, MediumPublic

Description

  • Affected components: Parser (MediaWiki core).
  • Engineer for initial implementation: Parsing Team (WMF).
  • Code steward: Parsing Team (WMF).

Motivation

Background

Magic links are a feature of MediaWiki core that create automatic links for 3 hardcoded external identifiers:

See also: https://en.wikipedia.org/wiki/Help:Magic_links.

For the purposes of this RFC, we are considering free external links (e.g. typing just "https://www.example.org") to not be a "magic link".

Problem

These three magic links are hardcoded, inflexible, un-localizable (T15335), and generally unexpected. If this feature were proposed today, it would be rejected in favor of using templates or interwiki links. There have been long standing requests to make them disable-able (T47942), move them to an extension (T28207), or remove them outright (T136342).

In many cases, local templates are preferable and more advanced than magic links. For example, on the English Wikipedia, Template:ISBN checks for invalid ISBNs and adds them to a tracking category for editors to fix up.

Requirements

Plain text that does not involve some kind of syntatical grammar (such as {{ or <), must not have rich text side-effects.


Exploration

The RFC proposes three steps:

  1. Disable the magic link functionality by default for the MediaWiki 1.28 release, and mark it as deprecated. (approved in E287)
  2. Deprecate magic links on Wikimedia wikis (e.g. Wikipedia), providing alternatives for this functionality and tools to aid the migration. We agreed to start building these tools in E287.
  3. Disable magic links functionality a year or so after the MediaWiki 1.28 release (in time for the next MediaWiki LTS release)

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

I agree with SV above. This is a major (and somewhat foolish) change. It should not be imposed by a technocracy.

At first I thought this was a good idea, but not anymore. This type of magic formatting is a good thing. The problem isn't that it happen, but that it isn't smart enough. We should actively limit the number of weird templates the editors must remember. If we can add a little smartness to wikitext, then we should do it.

Whether architecture committee agree on something has virtually no bearing in this case, this is at its core about editing not about boxes. If the editing communities want a feature, then the devs should deliver that feature. It is the editors that is the customer in this case, not the architecture committee or any other dev.

At first I thought this was a good idea, but not anymore. This type of magic formatting is a good thing. The problem isn't that it happen, but that it isn't smart enough. We should actively limit the number of weird templates the editors must remember. If we can add a little smartness to wikitext, then we should do it.

I think this argument conflates the ease with which editors can insert formatted markup and whether we should support magic formatting. As noted repeatedly, these three magic links are anomalies. Editors expect to be able to use templates for formatting and it often causes confusion when editors notice that plain syntax has become a link. Templates can also be localized and have a built-in and supported means of tracking uses. Templates are what we use for nearly every other kind of similar inline formatting. Adding a little smartness to wikitext is historically and demonstrably where we get into trouble.

Adding a little smartness to wikitext is historically and demonstrably where we get into trouble.

Which is a unsubstantiated claim.

If I should make an unsubstantiated claim I would say that the current template solutions is what creates havoc, as it is very as they invites an editing style that creates extremely messy and unreadable markup. Don't replace simple markup with more messy and difficult to read and understand markup.

Not sure that the template markup "{{PMID|somenumber}}" is in any way messier and unreadabler than "PMID somenumber". The very existence of this task is kind of proof that the magic links can make problems. As well as a number of "RFC somenumber" making false positive links on enwiki.

Since the vast majority of editors are non-coders, as well as potential new authors (especially older people), increasing the amount of special characters in the source text (e.g. for templates) is imho a step in the wrong direction. I wish more of mediawiki would work like magic words...

I think between all bad ideas which were discussed the MediaWiki development, removing magic links and especially ISBN is one of the worst. I do not consent with MZMcBride when he talks about confusion under editors because of the ISBN is suddenly linked. Editors and especially new editors are happy that this happens.

I do also contest PerfektesChaos' claim he would like to see the issue resolved by templates in the German WP. I am part of the German WP a good time longer than he and I can assure that that community is absolutely template-hostile, and they explicitely decided against Vorlage:ISBN and Vorlage:PMID in 2007. The main reason is that the German WP community dislikes, and part of it hates, template within "Fließtext" as they call the part of source text outside infoboxes and categories.

If you'll remove the magic links the following will happen (off course only if the community agrees to reinstate the formerly deleted template):

  • some editors will use the ISBN template and some will not
  • some editors will add templates which other editors had omitted
  • they will start editwars about the necessarity to link a specific ISBN (using template)
  • regular bot runs needed to fix where editors had been too lazy/scatty to use template

Or the outcome will be that what happened with doi: Most of doi's are standing in articles unformatted (unlinked) because of editors don't care. Jepp, I think that will happen to ISBN links. Most editors won't care and Wikipedia again goes one step user unfriendly.

Thus I call to revise the decision to remove magic links, I also call to reintroduce doi magic link and to introduce yet more useful magic links.

I do also contest PerfektesChaos' claim he would like to see the issue resolved by templates in the German WP. I am part of the German WP a good time longer than he and I can assure that that community is absolutely template-hostile, and they explicitely decided against Vorlage:ISBN and Vorlage:PMID in 2007. The main reason is that the German WP community dislikes, and part of it hates, template within "Fließtext" as they call the part of source text outside infoboxes and categories.

What about references? ISBNs are typically used as part of references, correct?

Does the German Wikipedia use <ref></ref> tags with templates? On the English Wikipedia, it's common to use references with templates such as <ref>{{cite book|isbn=1234}}</ref>, which can then accept the ISBN as an argument. This means no change to the markup of many pages, since only the underlying "Template:Cite book" needs to be potentially updated.

How do you explain to users that almost every citation identifier, including DOI, needs extra markup, except ISBN, PMID, and RFC? These three are the exception to otherwise consistent behavior. Why not remove the magical behavior and be explicit instead? If you look at https://en.wikipedia.org/wiki/Template:Citation/identifier#Usage, you can see many other kinds of citation identifiers that use wrapper templates such as {{ASIN}} or {{OCLC}}? Why is this bad? Why does it make sense to have the software try to parse the page and guess what is and is not a valid citation identifier?

You could also just write [[Special:BookSources/1234]] in the page instead of using a template such as {{ISBN|1234}}. Do you and other German Wikipedians dislike all wiki markup, including link markup? If so, perhaps VisualEditor is the answer?

Or the outcome will be that what happened with doi: Most of doi's are standing in articles unformatted (unlinked) because of editors don't care. Jepp, I think that will happen to ISBN links. Most editors won't care and Wikipedia again goes one step user unfriendly.

Your argument is that nobody will care enough to make links? This seems to point to the questionable value of having the links at all, if nobody is using them or interested in supporting them.

Hi MZMcBride,
yes, ISBN are part of references. We generally use ref-tags, but citing templates are much less used than in en.wp. For example, in natural sciences afaik mostly a PMID- and DOI-converter is used (http://www.hbz-da.de/wikipedia/PMID2reference.php) instead of templates. This also reduces imho unnecessary code and increases readability of the source text. I wish magic links were extended to reduce the amount of special characters in the source text. And due to the necessity of mouse, trackpad or trackpoint for all operations besides typing words in the visual editor, it is much slower than writing wikitext. Which could be simplified with more magic links. So VE is only the answer for some. Most wikipedians i know personally (mainly active and very active editors) write wikitext. And i don't think his argument was that nobody would care to make links but most would prefer not to have to and some won't care to.

By the way, how would a bot do the job?

By the way, how would a bot do the job?

The idea is to have a bot do a one-time conversion. Then we can stop supporting these magic links and this magical linking behavior in MediaWiki core forever. RFC in particular has been very problematic since the wiki universe has its own concept of requests for comment that is distinct from the Internet Engineering Task Force's concept of RFCs.

Another nice advantage to templates is that we can then translate/localize these identifiers, instead of relying on the English "ISBN" or "PMID". For example, perhaps the abbreviation would be different in German or French or Spanish. By using templates, we also can track usage more easily.

I completely agree that {{template syntax}} and [[link syntax]] is ugly, but it's what we're already using everywhere, so I'm struggling to see a reason to continue supporting magic links or expanding their use.

A bot is currently running on the English Wikipedia to do this one-time conversion: https://en.wikipedia.org/wiki/Special:Contributions/Magic_links_bot. For wikis that do not want to do this conversion, the ISBNs and PMIDs will likely become unlinked at some point in the future. The options then will be to use link or template markup to make ISBN and PMID clickable, the same way we do with everything else currently, or to have the markup be unlinked. A third option might be a JavaScript hack, if the local wiki really were interested in setting that up and maintaining it.

what we're already using everywhere

Tradition by itself is imho not a very strong argument. And a one-time conversion, does it mean we have to manually correct missing ISBN- and PMID-templates in future edits? I guess, the RfC isn't used in de.wp, as we have polls and requests for decisions instead. And, scnr, do i have a chance of convincing you of the good that is in our child?

Hope I am not too late to chip in a reply -

! In T145604#3290707, @MZMcBride wrote:
How do you explain to users that almost every citation identifier, including DOI, needs extra markup, except ISBN, PMID, and RFC?

Why is an explanation needed? For average users they probably only want to see them linked, and that's it.

Your argument is that nobody will care enough to make links? This seems to point to the questionable value of having the links at all, if nobody is using them or interested in supporting them.

We don't have unlimited manpower. I am from the Japanese Wikipedia, where the user base is broad but the number of people able to (let alone interested to) support is really thin. Simply being "different and inconsistent" (or other technical reasons) is not a valid argument to cut a working feature and increase the already-too-heavy workload. If it is not like MediaWiki will break down tomorrow, can you please revise this decision to drop magic links?
(Adding one footnote: using a bot increases the number of revisions and uses up some manpower so that doesn't change my argument.)

In T145604#3290707, @MZMcBride wrote:

How do you explain to users that almost every citation identifier, including DOI, needs extra markup, except ISBN, PMID, and RFC?

Why should there be a need to explain that? That is, how it works, period.

Do you and other German Wikipedians dislike all wiki markup, including link markup?

It's not me, but some people want a clean, easy readable source text, and they dislike the VE either. However as Ghilt wrote, the usage of citation templates in DE:WP is not common, less than 10 percent of all articles use them, and there even exist users which remove them and convert them in plain link citations. What is much more needed is making en:Template:Citation a global template so it works in each and every language version. It are the same people blocking each development of citation templates (de:Vorlage:Citation is full protected for years!)

Your argument is that nobody will care enough to make links?

Not really. My argument is that they won't care enough to make links which formerly had been generated automatically, i.e. they will create the link only if it has a value for their own interest as in an article, for instance. They won't if it would be only courtesy, f. ex. in a discussion. My argument is also that people won't care to add such links if other editors did not add them.

In T145604#3290882, @MZMcBride wrote:

Another nice advantage to templates is that we can then translate/localize these identifiers, instead of relying on the English "ISBN" or "PMID".

Well, ISBN is ISBN everywhere on the world. Besides, localization has been the worst idea WP developpers implemented so far. Why?

  1. Users who want to use localized syntax still need to learn English syntax when they want to work on commons – [[Datei:Irgendwas.jpg|mini|hochkant|Irgendeinbild]] won't work on Commons.
  2. There are thousands of useless edits each and every day because of useless users convert File: to Datei: and thumb to mini, and those users spam watchlists and make it more difficult for authors to detect vandalism.

If it was me I would remove all syntax localizations. However, I like the usage of centralized templates with localized output, e.g. en:Template:Infobox UK settlement and de:Vorlage:Infobox Ort im Vereinigten Königreich use the same syntax of course the appearance is in the respective language. (That, basically, is the believe, that for example the Italian Wikipedia is most comprehensive for Italien settlement so DE:WP uses their (the Italian) syntax, and it uses the French WP syntax for French settlements infoboxes. Sorry I made a disgression but I felt it was needed to advert the impression I was some kind of fundamentalist ;-)

The explanation is needed because otherwise the feature is unused or people make bad markup for things that don't have the feature.

The explanation is needed because otherwise the feature is unused or people make bad markup for things that don't have the feature.

That's not a valid argument. If the feature is removed it will be unused as well. Or the other way around, so far it isn't possible to not-use the feature as ISBN <numbers> is marking up the number as ISBN link. Also "people making bad mark-up" is no argument. There are two possibilities: the ISBN a user typed is correct ( -> good markup) or the ISBN is wrong ( -> still good mark-up but the number does not exist). "ISBN 1234" is creating a bad ISBN-Link, as "1234" is not a valid ISBN. However, {{ISBN|1234}} still creates the same bad link. There is no difference between them. And the last part of your argument isn't valid as well. Actually thats kind of [[:en:WP:POINT]]; and most major language versions won't accept it. There is no reason to remove a feature because of another feature does not exist. The question shoudl be wether that other feature link should be a magic link.

For make it clear: IMHO we need much more magic links, not the other way around.

I was just contesting the claim that people will figure out by themselves how these things work.

"ISBN 1234" is creating a bad ISBN-Link, as "1234" is not a valid ISBN. However, {{ISBN|1234}} still creates the same bad link. There is no difference between them.

On the English Wikipedia, https://en.wikipedia.org/wiki/Template:ISBN uses https://en.wikipedia.org/wiki/Module:Check_isxn for input validation. If a user adds {{ISBN|1234}} to a page, the page will be automatically added to https://en.wikipedia.org/wiki/Category:Pages_with_ISBN_errors and the user is informed with red error text.

An "intentionally" invalid ISBN is handled by a separate template, as noted at https://en.wikipedia.org/wiki/Template_talk:ISBN#Allow_hiding_.22Invalid_ISBN.22_error.

Besides, localization has been the worst idea WP developpers implemented so far.

This is wrong, period. I suggest you read https://www.mediawiki.org/wiki/Principles, where #2 is "internationalized, with equal support for all languages;"

Just to let you know that I've wrapped wsd's patch to https://www.mediawiki.org/wiki/Extension:MagicalLinkers
It has been developed against MW 1.31. I think that with this extension, the core functionality can be removed.

Strange, it seems like mw:Requests for comment/Future of magic links is marked as accepted, still there are a lot of serious opposing comments that are not addressed. It the points to phabricator, but this thread neither does address the raised concerns.

Quite frankly, I wonder if this RfC is accepted at all.

<rant>What a few developers do in this case is a quite good example of reimplementing a markup language A with another markup language B, without having a real discussion of why B would be a better language than A. Because of this "I have a much better idea then you" we have several partially implemented languages. It has turned into a complete mess, and it will not be better by adding more templates. The editors already has to memorize to many weird templates, and that is not a good path forward!</rant>

Saimongoltinio subscribed.

А я попробую , час и будет готово

What is the status of this ticket? As far as I can tell, it has been more than three years since the release of MW 1.28 (https://www.mediawiki.org/wiki/MediaWiki_1.28), but magic links are still functioning, at least on en.WP.

Is it up to individual WP instances to turn off magic links locally? Did we at en.WP miss a memo at some point?

Krinkle renamed this task from [RfC] Future of magic links to RFC: Future of magic links.Apr 3 2020, 11:43 PM
Krinkle updated the task description. (Show Details)
Krinkle updated the task description. (Show Details)
Krinkle moved this task from Under discussion to P4: Tune on the TechCom-RFC board.

FWIW, we do not believe this task is blocked on Parsing-Team--ARCHIVED and we don't currently have this on our roadmap. Magic links are implemented and work in both the legacy parser and Parsoid as well as Visual Editor, so this is not blocking any feature development. It seems likely that magic links wouldn't be included in any notional "wikitext 2.0" proposal, but that's a long way off (if we ever get there).

If there is someone who wants to drive magic link removal, we are happy to support, but at this point that seems like it would be limited to a 2-ish line patch to add either a tracking category or a linter category to track "legacy" magic link usage on pages. We wouldn't write or deploy such a patch until we knew someone had an active effort to use this, though.

I don't understand the comment about tracking. The MW software has performed this tracking for more than three years, per Part 2 of the Exploration section of the original ticket description above.

See https://en.wikipedia.org/wiki/Category:Pages_using_ISBN_magic_links and https://en.wikipedia.org/wiki/Special:TrackingCategories

The categories, at least on en.WP, have been the subject of an active effort to convert article-space instances of ISBN magic links to ISBN templates for over three years, because MWF developers told us that magic links were going away.

At this point, this ticket is about Part 3 of the "Exploration" section of this ticket's original description, disabling magic links functionality. My understanding is that implementation of Part 3 is wholly in the hands of WMF developers. If I am wrong about that, and we should be implementing Part 3 (disabling magic links) locally on a per-wiki basis, please let us know.

I think @cscott was trying to say: we don't want to pursue removal at this time if it needs us to ensure that all wikis fix up their links first. But, if wikis have already done the work of fixing up their pages, then it becomes easier for us to remove this. This is mostly a matter of work priorities. We haven't researched where different wikis are at this point. We are currently focused on getting Parsoid to replace the current parser and if taking this RFC to completion requires us to spend energies to ensure all wikis to fix their magic links first, then, it is lower priority for us at this point.

That said, over the next year, we are likely going to discover new linting categories for making Parsoid the default wikitext engine for wikimedia wikis. At that point, we could potentially fold magic link removal effort with it if there are still wikis out there that are using magic links.

Feedback welcome.

Thanks for the response.

Can en.WP turn off magic links locally?

Yes, magic links are a per-wiki feature flag: https://www.mediawiki.org/wiki/Manual:$wgEnableMagicLinks

That said, I'm not certain that Parsoid and VE can turn these off quite that easily yet: https://codesearch.wmflabs.org/search/?q=EnableMagicLinks&i=nope&files=&repos= doesn't show anything checking this configuration variable in the VIsualEditor or Parsoid codebases. So if a wiki (enwiki?) is actually ready to turn off magic links in core, perhaps I was wrong and this is a task the Editing-team and Parsing-Team--ARCHIVED need to schedule. That's T145590: Update Parsoid to be compatible with magic links being disabled and T145589: Update VisualEditor to be compatible with magic links being disabled, which were created in 2016 but not linked up (until I just did now) as subtasks of this RFC.

Created T252054: Implement magic link functionality as an extension for possible future follow-up work that would let us pull the magic link functionality out of the core codebase. Not a blocker here; in fact pulling magic links out of the core codebase could be done even if we got blocked on actually turning off magic links on one of the WMF wikis.

@cscott Are there unanswered questions here and/or other outreach you'd like to happen before this proceeds? Note that closing of this RFC does not mean that it has to be disabled/removed from prod the next day, so if this plan is accurate and up-to-date and there are no unknowns, I'd recommend you propose this for Last Call :)

Krinkle triaged this task as Medium priority.Jul 29 2020, 8:35 PM

Capturing a note from other travels on a reasonable task: Sanitizer::safeEncodeAttribute() is hiding an arm/leg of the auto-linking code.

Pretty weird that we've disabled magic links on English wikipedia (where they are generally reasonable, ie RFC links to an english-language documentation site) but not yet disabled them on other wikis, which I thought were the original complaintants here. Is there a plan for turning these off WMF-wide? @Legoktm ?

No plan, but still interested in moving this forward, I'll try to look at what the landscape looks like (presumably there are small wikis that aren't using these entirely) and propose a plan.

IIRC a lot of the complaints against disabling came from de.wp. Unfortunately they've disabled the tracking categories so we don't even have quick stats (I'll pull them from a dump).

No plan, but still interested in moving this forward, I'll try to look at what the landscape looks like (presumably there are small wikis that aren't using these entirely) and propose a plan.

Still working on surveying wikis, but as an update I've been mostly refreshing my knowledge of the RfC and objections, as well as looking at software things, especially T179769#8608433 and T145590#8608455.

IIRC a lot of the complaints against disabling came from de.wp. Unfortunately they've disabled the tracking categories so we don't even have quick stats (I'll pull them from a dump).

In the main namespace, via the enterprise HTML dump:

  • ISBN: 2,471,474 (in refs: 755,037 or ~30%)
  • RFC: 3,010 (in refs: 239 or ~8%)
  • PMID: 71,487 (in refs: 58,891 or ~82%)

Note that these are total number of links, not pages with the links.

Regarding German Wikipedia you need a smart stategy, then it will work.

  • Looking at LINT errors dewiki will proceed; compare with enwiki.
  • However, among 20.000 people there are a dozen fellows who started in 2005 and they do not want any change and everything has to be kept as it was in 2005. Unfortunately, they are quite loud and experienced and collected a lot of merits for their work as authors over two decades.

A promising plan would be:

  • Wait until Vector2022 has been established successfully.
    • A war on two frontiers simultaneously is not a good idea.
    • Currently even enwiki is reluctant.
    • Gadgets need to be adapted to new page arrangement.
  • After sea has calmed, announce a change for RFC within about 3 or 6 months; end of 2023 or July 1st 2024 or whatever, for all projects.
    • There is already a template.
    • RFC 1 can be switched easily by bot in article space towards RFC&nbsp;1<ref>{{RFC-Internet if not already within <ref>, then change access inside <ref> elements, then cleaning up the remainders. Other namespaces are no longer linked.
    • Tracking category may be activated then and tells occurrences for several years even if no link is generated any longer.
  • Then announce discontinuation of PMID within 3 or 6 months.
    • Those will be overcome by [[pmid:1|PMID&nbsp;1]] whereever they occur.
  • For ISBN you will need a global parser function {{#isbn:0-123-456-7|1}} with second parameter for invalid ISBN but printed in books and registered in national library catalogues.
    • This should format and hyphenate correctly and will apply a class and nowrap and error message and maintenance category and everybody is happy.
    • Since it is widely used, bots will need ages and version histories get lots of entries.
    • Small wikis have no bots.
    • A global migration plan needs to be developed, perhaps via server script (a parser function can be applied everywhere while a template needs local support and might collide with a local template).
    • When migrating to next Parsoid storage level this might be done automatically.
    • VE might dump parser function formatting.
    • When retrieving wikitext content deliver new syntax, or store new syntax every time when publishing. Will take some ages.
  • After each stage, check riot level in all wikis. Adapt roadmap if necessary.

Thanks, this is a good starting point.

What is the value of splitting up the migration of RFC and PMID into two steps? Wouldn't it be less of a hassle to do both at the same time?

  • Wait until Vector2022 has been established successfully.

Definitely, and it'll take a while to get the necessary technical work done first anyways.

  • After sea has calmed, announce a change for RFC within about 3 or 6 months; end of 2023 or July 1st 2024 or whatever, for all projects.
    • There is already a template.
    • RFC 1 can be switched easily by bot in article space towards RFC&nbsp;1<ref>{{RFC-Internet if not already within <ref>, then change access inside <ref> elements, then cleaning up the remainders. Other namespaces are no longer linked.

I am a little surprised a template is preferred over [[rfc:1|RFC 1]], but OK.

  • Tracking category may be activated then and tells occurrences for several years even if no link is generated any longer.

Having the tracking category stick around longer is an interesting idea, I think it should be possible.

  • Then announce discontinuation of PMID within 3 or 6 months.
    • Those will be overcome by [[pmid:1|PMID&nbsp;1]] whereever they occur.
  • For ISBN you will need a global parser function {{#isbn:0-123-456-7|1}} with second parameter for invalid ISBN but printed in books and registered in national library catalogues.

So...why do we need a parser function? Back in 2016 you had proposed using a template (c.f. T148274#2788218), which I agree with and prefer.

  • A global migration plan needs to be developed, perhaps via server script (a parser function can be applied everywhere while a template needs local support and might collide with a local template).

I have some ideas for a global bot, but yeah, that should be part of this. That said, I do think spending the time to get the local support part right is worth it (e.g. maybe standardize https://www.wikidata.org/wiki/Q5617482), it'll help with other cross-wiki imports, ContentTranslation, etc.

I have some ideas for a global bot

I have one that's already been run(ning) on two wikis and could be run globally.

So...why do we need a parser function? Back in 2016 you had proposed using a template (c.f. T148274#2788218), which I agree with and prefer.

Does not sound like a good idea to rely on templates until we have global templates

What is the value of splitting up the migration of RFC and PMID into two steps? Wouldn't it be less of a hassle to do both at the same time?

Look at the figues, especially deWP.

  • RFC is a test balloon.
  • Amount is not too large, check how support for small wiki will succeed.
  • Give them some months to take breath; learn from experiences in RFC step.

I am a little surprised a template is preferred over [[rfc:1|RFC 1]], but OK.

It has been an answer for deWP circumstances.

  • We do have a template already.
  • In deWP it is actually not permitted to have external links within major article texts, but magic links are below radar here.
  • An RFC actually needs a title in citation, but RFC and number is part of the title. It is a native publication as Internet RFC. A PMID is only a catalogue number for something published whenever in any journal, and PMID is an additional hint to catch an abstract or the entire text for free. PMID is appended to a citation from a journal issue.
  • For global purpose I would recommend wake procedure as with pmid:.
  • For ISBN you will need a global parser function {{#isbn:0-123-456-7|1}} with second parameter for invalid ISBN but printed in books and registered in national library catalogues.

So...why do we need a parser function? Back in 2016 you had proposed using a template (c.f. T148274#2788218), which I agree with and prefer.

The answer was given below meanwhile.

  • {{#isbn:0-123-456-7|1}} cannot collide with any local template.
  • Functionality is the same as by local template, and should include nowrap, hyphenating and intentionally invalid code as well as detection of invalid ISBN (not really possible for PMID and RFC).
  • This is more functionality than just [[rfc:1|RFC&nbsp;1]] since there are smaller numbers in ID.
  • A parser function is implemented and maintained once globally, and available everywhere.
  • I am trying to learn something, at least since 2016, and might get additional insights.

I have some ideas for a global bot, but yeah, that should be part of this. That said, I do think spending the time to get the local support part right is worth it (e.g. maybe standardize https://www.wikidata.org/wiki/Q5617482), it'll help with other cross-wiki imports, ContentTranslation, etc.

Such local template might have differing parameters and additional functionalities, making more use of the ISBN, providing direct link to national library of that language. And you would have to distribute and update the hyphenization database to every single wiki.

  • As you might have noted in that Wikidata item, there is no such template in deWP.
  • However, we do have a formatting function (example).

Why, since the comments on the RFC were overwhelmingly against this proposal, was it implemented?

Why, since the comments on the RFC were overwhelmingly against this proposal, was it implemented?

It was not, AFAIK. See the part of https://www.mediawiki.org/wiki/Requests_for_comment/Future_of_magic_links#Proposal where it says "Update: this part was controversial, and will be re-evaluated based on the response to disabling by default."

Magic links were made enable-able on a per-wiki basis instead of a built-in feature of the core code that could not be turned off or on.

The English Wikipedia overwhelmingly decided to turn them off: https://en.wikipedia.org/w/index.php?title=Wikipedia:Village_pump_(proposals)&oldid=772133164#Future_of_magic_links

12 Support votes for en.wp does not qualify as "overwhelmingly decided", even if there were only few opposing voices because this small number is far from being representative and therefore is prone to small group bias. I really doubt that a community representative vote was wanted. Imho, that was underwhelming.

Please, this is not the venue for parochial on-wiki community disputes; take it there.

The first sentence is correct, but please don't ask community members to go away. Imho, this is not the best problem resolution.

@Legoktm: German Wikipedia discontinued to use RFC magic in article space for almost a year now.

I suggest to publish via TechNews now that RFC magic will no longer cause links by 1 Jan 2025.

  • RFC has the smallest number of usages, PMID is next.
  • Both are easily to replace, and do no harm if not linked.
  • Some three months are sufficient time for minor wiks to run bots.
  • It should be offered to run bots of major wikis in small wikis if requested.

See what will happen,

  • on announcement,
  • in the following months.

Next step might be to discontinue PMID magic by 1 Jan 2026, announced in summer 2025.

An entirely differnet issue is ISBN, which is more important and link to special page is used and needed in all wikis. Only some larger wikipedias are using RFC and PMID at larger scale.

Why, since the comments on the RFC were overwhelmingly against this proposal, was it implemented?

It was not, AFAIK. See the part of https://www.mediawiki.org/wiki/Requests_for_comment/Future_of_magic_links#Proposal where it says "Update: this part was controversial, and will be re-evaluated based on the response to disabling by default."

Magic links were made enable-able on a per-wiki basis instead of a built-in feature of the core code that could not be turned off or on.

The English Wikipedia overwhelmingly decided to turn them off: https://en.wikipedia.org/w/index.php?title=Wikipedia:Village_pump_(proposals)&oldid=772133164#Future_of_magic_links

The first quote "Update: this part was controversial, and will be re-evaluated based on the response to disabling by default." means that WMF decided to do it and maybe revert if enough noise was made.

That second link was not the RFC, that was an attempt to get consensus on EN about what to do when it was turned off. However the fact that WMF had already made the decision, meant that there was less support, or even discussion about retaining the magic. One editor even commented:

@Rich: This discussion is about how to replace magic links, not whether to replace magic links.

Even worse, if I understand it correctly, the whole thing was started because the WMF engineers could not make it work in RLT languages.

The major issue on this kind of magic is that they are a nightmare on parsing.

  • And hard to understand and teach and describe to authors as well.
  • It is blowing up syntax description and make things even more complicated.

Every time you encounter letters like IPR you need to investigate the preceding and following characters, whether letters before or punctuation and spacing characters, and whether next is SMF, then BIC, then ND, then look for a space, even as entity, or U+00A0. Then inspect the sequence of digits, perhaps hyphens or dots in ISBN.

A good language is triggered by some special characters, like <, [, {{, __, -{, {| etc. Then you can look for the closing counterpart. If matching, you can examine what happens in between, perhaps before.

A : is hard enough. Are letters preceding, until first non-letter, then check whether they match things like html or mailto, and perhaps // are following where expected, and find the characters terminating a magic URL.

And yes, for all non-latin scripts things like ISBN, PMID, RFC are not appropriate, and the Wikipedia audience is not necessarily able to understand such glyphs they have never seen.