With quick look I failed a place in integration/config repo where I could add Scribunto there, thus the ticket.

gatedextensions list in zuul/parameter_functions.py. https://www.mediawiki.org/wiki/Continuous_integration/Tutorials/Adding_your_extension_to_the_shared_gate_job is uh, non-existent :(

That setting I've actually found. But, as you say, it is, well, shared. I thought I could just add it for Wikibase (but that would mean having a quibble job for Wikibase and not using wmf-quibble-vendor-mysql-hhvm-docker job).
Let me add it to it and let's see if @hashar objects.

Change 449949 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[integration/config@master] Added scribunto to shared gate quibble job

https://gerrit.wikimedia.org/r/449949

Scribunto is intentionally not in the list of gated extensions. Its tests take a while and would be run for any patch made on any of the extension, even when we well know the tests are not needed.

T125050 has some history about a previous attempt.

We need CI to be smarter by making run solely:

• run the test of the extension for which a patch got sent
• for the other extensions, only run a subset of tests (integration tests)

Right now, any patch on any of the gated extension also run all the Wikibase tests (as well as all tests from all extensions). That partly explains why the job is so slow.

(note I recently phased out the 'mediawiki-extensions-*' jobs in favor of 'wmf-quibble-*' jobs which use Quibble)

Let's create a second gate job that runs in parallel? Scribunto is an important enough extension that we should be regularly running its tests.

Just to clarify: the Scribunto tests do run for patches sent to Scribunto. It is just that when sending a patch to Wikibase, its integration tests with Scribunto are skip since the extension is not present.

We would need all our test suites (PHPUnit, QUnit, Selenium) to be smarter. Timo wrote a nice summary for Selenium tests at T199939#4442512 which would be applicable to others suites.

So essentially I would like to hold on this task until CI / test suites are ready.

Change 449949 abandoned by Jforrester:
Added scribunto to shared gate quibble job

Reason:
Duplicate of Ia128e3abe7 (which also isn't merged).

https://gerrit.wikimedia.org/r/449949