[go: up one dir, main page]

Page MenuHomePhabricator

Switch to spec reporter
Closed, ResolvedPublic

Description

Status

TODO

  • mediawiki/core 773549 793446 2.1.0
  • mediawiki/extensions/AbuseFilter
  • mediawiki/extensions/AchievementBadges 1069238
  • mediawiki/extensions/AdvancedSearch
  • mediawiki/extensions/CampaignEvents
  • mediawiki/extensions/Cargo
  • mediawiki/extensions/CentralNotice 793784
  • mediawiki/extensions/CheckUser
  • mediawiki/extensions/CirrusSearch 1069241
  • mediawiki/extensions/CodeMirror
  • mediawiki/extensions/ContactPage
  • mediawiki/extensions/ContentTranslation
  • mediawiki/extensions/Echo 797186
  • mediawiki/extensions/ElectronPdfService 798624
  • mediawiki/extensions/examples
  • mediawiki/extensions/FacetedCategory 1069242 T373857
  • mediawiki/extensions/FileImporter
  • mediawiki/extensions/GlobalWatchlist
  • mediawiki/extensions/GrowthExperiments 798629
  • mediawiki/extensions/IPInfo
  • mediawiki/extensions/Math 798630 1069237
  • mediawiki/extensions/Newsletter 798634
  • mediawiki/extensions/PageTriage
  • mediawiki/extensions/Popups
  • mediawiki/extensions/ProofreadPage 1060799
  • mediawiki/extensions/RelatedArticles 798646
  • mediawiki/extensions/ReportIncident
  • mediawiki/extensions/RevisionSlider
  • mediawiki/extensions/Sanctions 1070208 T373863
  • mediawiki/extensions/TemplateWizard
  • mediawiki/extensions/TwoColConflict
  • mediawiki/extensions/VisualEditor 793463
  • mediawiki/extensions/Wikibase
  • mediawiki/extensions/WikibaseLexeme
  • mediawiki/extensions/WikiLambda 798636
  • mediawiki/extensions/Wikistories
  • mediawiki/skins/MinervaNeue
  • wikibase/termbox
  • wikidata/query/gui 1070211
  • edwardspec/mediawiki-moderation
  • femiwiki/FemiwikiSkin (hosted on gihub instead of gerrit, uses yarn instead of npm)
  • wmde/banner-screenshots
  • wmde/wdio-wikibase
  • wmde/wikibase-release-pipeline
  • commtech/wishlist-intake
  • data-engineering/mpic

Description

We have switched from spec to dot reporter in 2019 (528270). dot reporter provides no information on which tests suites and tests are executed. spec reporter is more verbose but it also provides much more information that is useful in general, but especially when debugging. For example, which tests are executed and which tests are skipped.

Output of dot reporter. Not super helpful. Doesn't say much. Actually, it's lying. It says five tests suites ran and all of them passed. That's not true. Read on.

1~/Documents/gerrit/mediawiki/core$ npm run selenium-test
2
3> selenium-test
4> wdio ./tests/selenium/wdio.conf.js
5
6Execution of 5 workers started at 2022-03-22T16:37:02.776Z
7
8[0-0] RUNNING in chrome - /tests/selenium/specs/page.js
9[0-0] PASSED in chrome - /tests/selenium/specs/page.js
10[0-1] RUNNING in chrome - /tests/selenium/specs/recentchanges.js
11[0-1] PASSED in chrome - /tests/selenium/specs/recentchanges.js
12[0-2] RUNNING in chrome - /tests/selenium/specs/user.js
13[0-2] PASSED in chrome - /tests/selenium/specs/user.js
14[0-3] RUNNING in chrome - /tests/selenium/specs/watchlist.js
15[0-3] PASSED in chrome - /tests/selenium/specs/watchlist.js
16[0-4] RUNNING in chrome - /tests/selenium/wdio-mediawiki/specs/BlankPage.js
17[0-4] PASSED in chrome - /tests/selenium/wdio-mediawiki/specs/BlankPage.js
18
19 "dot" Reporter:
20..............
21
22Spec Files: 5 passed, 5 total (100% completed) in 00:00:54

Output of spec reporter. Much more verbose but much more useful. It says that out of five test suites, only two are running, and three are skipped!

1~/Documents/gerrit/mediawiki/core$ npm run selenium-test
2
3> selenium-test
4> wdio ./tests/selenium/wdio.conf.js
5
6Execution of 5 workers started at 2022-03-22T16:45:16.385Z
7
8[0-0] RUNNING in chrome - /tests/selenium/specs/page.js
9[0-0] PASSED in chrome - /tests/selenium/specs/page.js
10[0-1] RUNNING in chrome - /tests/selenium/specs/recentchanges.js
11[0-1] PASSED in chrome - /tests/selenium/specs/recentchanges.js
12[0-2] RUNNING in chrome - /tests/selenium/specs/user.js
13[0-2] PASSED in chrome - /tests/selenium/specs/user.js
14[0-3] RUNNING in chrome - /tests/selenium/specs/watchlist.js
15[0-3] PASSED in chrome - /tests/selenium/specs/watchlist.js
16[0-4] RUNNING in chrome - /tests/selenium/wdio-mediawiki/specs/BlankPage.js
17[0-4] PASSED in chrome - /tests/selenium/wdio-mediawiki/specs/BlankPage.js
18
19 "spec" Reporter:
20------------------------------------------------------------------
21[Chrome 99.0.4844.83 darwin #0-0] Running: Chrome (v99.0.4844.83) on darwin
22[Chrome 99.0.4844.83 darwin #0-0] Session ID: 5b441791-dca8-4bbb-b82d-7e3713ba58c9
23[Chrome 99.0.4844.83 darwin #0-0]
24[Chrome 99.0.4844.83 darwin #0-0] » /tests/selenium/specs/page.js
25[Chrome 99.0.4844.83 darwin #0-0] Page
26[Chrome 99.0.4844.83 darwin #0-0] ? should be previewable
27[Chrome 99.0.4844.83 darwin #0-0] - should be previewable
28[Chrome 99.0.4844.83 darwin #0-0] ? should be creatable
29[Chrome 99.0.4844.83 darwin #0-0] - should be creatable
30[Chrome 99.0.4844.83 darwin #0-0] ? should be re-creatable
31[Chrome 99.0.4844.83 darwin #0-0] - should be re-creatable
32[Chrome 99.0.4844.83 darwin #0-0] ? should be editable @daily
33[Chrome 99.0.4844.83 darwin #0-0] - should be editable @daily
34[Chrome 99.0.4844.83 darwin #0-0] ? should have history @daily
35[Chrome 99.0.4844.83 darwin #0-0] - should have history @daily
36[Chrome 99.0.4844.83 darwin #0-0] ? should be deletable
37[Chrome 99.0.4844.83 darwin #0-0] - should be deletable
38[Chrome 99.0.4844.83 darwin #0-0] ? should be restorable
39[Chrome 99.0.4844.83 darwin #0-0] - should be restorable
40[Chrome 99.0.4844.83 darwin #0-0] ? should be protectable
41[Chrome 99.0.4844.83 darwin #0-0] - should be protectable
42[Chrome 99.0.4844.83 darwin #0-0] - should be undoable
43[Chrome 99.0.4844.83 darwin #0-0]
44[Chrome 99.0.4844.83 darwin #0-0] 9 skipped (1.4s)
45------------------------------------------------------------------
46[Chrome 99.0.4844.83 darwin #0-1] Running: Chrome (v99.0.4844.83) on darwin
47[Chrome 99.0.4844.83 darwin #0-1] Session ID: bf38cf84-f29f-4de6-bf6e-83736626d229
48[Chrome 99.0.4844.83 darwin #0-1]
49[Chrome 99.0.4844.83 darwin #0-1] » /tests/selenium/specs/recentchanges.js
50[Chrome 99.0.4844.83 darwin #0-1] Special:RecentChanges
51[Chrome 99.0.4844.83 darwin #0-1] ? shows page creation
52[Chrome 99.0.4844.83 darwin #0-1] - shows page creation
53[Chrome 99.0.4844.83 darwin #0-1]
54[Chrome 99.0.4844.83 darwin #0-1] 1 skipped (1.4s)
55------------------------------------------------------------------
56[Chrome 99.0.4844.83 darwin #0-2] Running: Chrome (v99.0.4844.83) on darwin
57[Chrome 99.0.4844.83 darwin #0-2] Session ID: a692643b-0b55-4d84-91db-849380031f32
58[Chrome 99.0.4844.83 darwin #0-2]
59[Chrome 99.0.4844.83 darwin #0-2] » /tests/selenium/specs/user.js
60[Chrome 99.0.4844.83 darwin #0-2] User
61[Chrome 99.0.4844.83 darwin #0-2] ✓ should be able to create account
62[Chrome 99.0.4844.83 darwin #0-2] ✓ should be able to log in @daily
63[Chrome 99.0.4844.83 darwin #0-2]
64[Chrome 99.0.4844.83 darwin #0-2] 2 passing (11.3s)
65------------------------------------------------------------------
66[Chrome 99.0.4844.83 darwin #0-3] Running: Chrome (v99.0.4844.83) on darwin
67[Chrome 99.0.4844.83 darwin #0-3] Session ID: fd021e49-412b-43d9-96b5-b53d6a9822b8
68[Chrome 99.0.4844.83 darwin #0-3]
69[Chrome 99.0.4844.83 darwin #0-3] » /tests/selenium/specs/watchlist.js
70[Chrome 99.0.4844.83 darwin #0-3] Special:Watchlist
71[Chrome 99.0.4844.83 darwin #0-3] ? should show page with new edit
72[Chrome 99.0.4844.83 darwin #0-3] - should show page with new edit
73[Chrome 99.0.4844.83 darwin #0-3]
74[Chrome 99.0.4844.83 darwin #0-3] 1 skipped (2.9s)
75------------------------------------------------------------------
76[Chrome 99.0.4844.83 darwin #0-4] Running: Chrome (v99.0.4844.83) on darwin
77[Chrome 99.0.4844.83 darwin #0-4] Session ID: b36c8b98-a4bf-4fda-a205-6fe4d1f68432
78[Chrome 99.0.4844.83 darwin #0-4]
79[Chrome 99.0.4844.83 darwin #0-4] » /tests/selenium/wdio-mediawiki/specs/BlankPage.js
80[Chrome 99.0.4844.83 darwin #0-4] BlankPage
81[Chrome 99.0.4844.83 darwin #0-4] ✓ should have its title @daily
82[Chrome 99.0.4844.83 darwin #0-4]
83[Chrome 99.0.4844.83 darwin #0-4] 1 passing (2.2s)
84
85Spec Files: 5 passed, 5 total (100% completed) in 00:00:32

Details

SubjectRepoBranchLines +/-
mediawiki/extensions/Sanctionsmaster+64 -19
mediawiki/extensions/FacetedCategorymaster+64 -19
wikidata/query/guimaster+1 -31
mediawiki/extensions/AchievementBadgesmaster+64 -19
mediawiki/extensions/CirrusSearchmaster+3 -33
mediawiki/extensions/Mathmaster+0 -30
mediawiki/extensions/ProofreadPagemaster+116 -37
mediawiki/extensions/Newslettermaster+117 -37
mediawiki/extensions/ElectronPdfServicemaster+117 -37
mediawiki/extensions/CentralNoticemaster+125 -45
mediawiki/extensions/RelatedArticlesmaster+53 -19
mediawiki/extensions/WikiLambdamaster+149 -322
mediawiki/extensions/GrowthExperimentsmaster+262 -54
mediawiki/extensions/Mathmaster+117 -42
mediawiki/extensions/Echomaster+129 -47
mediawiki/extensions/VisualEditormaster+117 -38
mediawiki/coremaster+6 -2
mediawiki/coremaster+111 -32
mediawiki/extensions/FileImportermaster+34 -119
mediawiki/coremaster+86 -12
Show related patches Customize query in gerrit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
zeljkofilipin changed the task status from Open to Stalled.May 27 2022, 3:53 PM
zeljkofilipin removed zeljkofilipin as the assignee of this task.

Blocked on T256626: Refactor WebdriverIO tests from sync to async mode. Only repositories already updated to wdio-mediawiki v2.0.0 (required by T256626) can be updated to v2.1.0.

zeljkofilipin raised the priority of this task from Medium to Needs Triage.May 27 2022, 3:53 PM
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin changed the task status from Stalled to Open.Aug 24 2023, 1:17 PM
zeljkofilipin moved this task from Waiting 🏹 to Q3 👔 on the User-zeljkofilipin board.
zeljkofilipin changed the task status from Open to In Progress.Aug 8 2024, 10:59 AM
zeljkofilipin claimed this task.
zeljkofilipin triaged this task as High priority.
zeljkofilipin moved this task from Backlog 🪒 to Deep work 🌊 on the User-zeljkofilipin board.

Change #1060799 had a related patch set uploaded (by Zfilipin; author: Zfilipin):

[mediawiki/extensions/ProofreadPage@master] WIP selenium: Use spec instead of dot reporter

https://gerrit.wikimedia.org/r/1060799

zeljkofilipin changed the task status from In Progress to Open.Aug 8 2024, 12:01 PM
zeljkofilipin removed zeljkofilipin as the assignee of this task.
zeljkofilipin raised the priority of this task from High to Needs Triage.
zeljkofilipin moved this task from Deep work 🌊 to Backlog 🪒 on the User-zeljkofilipin board.

Change #1060799 merged by jenkins-bot:

[mediawiki/extensions/ProofreadPage@master] selenium: Use spec-reporter instead of dot-reporter

https://gerrit.wikimedia.org/r/1060799

zeljkofilipin changed the task status from Open to In Progress.Aug 30 2024, 1:43 PM
zeljkofilipin triaged this task as Low priority.
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin moved this task from Backlog 🪒 to Deep work 🌊 on the User-zeljkofilipin board.
zeljkofilipin added a project: Browser-Tests.

Change #1069237 had a related patch set uploaded (by Zfilipin; author: Zfilipin):

[mediawiki/extensions/Math@master] selenium: Delete unneeded @wdio/dot-reporter npm package

https://gerrit.wikimedia.org/r/1069237

Change #1069238 had a related patch set uploaded (by Zfilipin; author: Zfilipin):

[mediawiki/extensions/AchievementBadges@master] selenium: Use @wdio/spec-reporter

https://gerrit.wikimedia.org/r/1069238

Change #1069241 had a related patch set uploaded (by Zfilipin; author: Zfilipin):

[mediawiki/extensions/CirrusSearch@master] selenium: Use @wdio/spec-reporter

https://gerrit.wikimedia.org/r/1069241

Change #1069242 had a related patch set uploaded (by Zfilipin; author: Zfilipin):

[mediawiki/extensions/FacetedCategory@master] selenium: Use @wdio/spec-reporter

https://gerrit.wikimedia.org/r/1069242

Change #1069237 merged by jenkins-bot:

[mediawiki/extensions/Math@master] selenium: Delete unneeded @wdio/dot-reporter npm package

https://gerrit.wikimedia.org/r/1069237

Change #1069241 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] selenium: Use @wdio/spec-reporter

https://gerrit.wikimedia.org/r/1069241

Change #1070208 had a related patch set uploaded (by Zfilipin; author: Zfilipin):

[mediawiki/extensions/Sanctions@master] selenium: Use @wdio/spec-reporter

https://gerrit.wikimedia.org/r/1070208

Change #1070211 had a related patch set uploaded (by Zfilipin; author: Zfilipin):

[wikidata/query/gui@master] selenium: Use @wdio/spec-reporter

https://gerrit.wikimedia.org/r/1070211

Change #1069238 merged by jenkins-bot:

[mediawiki/extensions/AchievementBadges@master] selenium: Use @wdio/spec-reporter

https://gerrit.wikimedia.org/r/1069238

Change #1070211 merged by jenkins-bot:

[wikidata/query/gui@master] selenium: Use @wdio/spec-reporter

https://gerrit.wikimedia.org/r/1070211

Change #1069242 merged by jenkins-bot:

[mediawiki/extensions/FacetedCategory@master] selenium: Use @wdio/spec-reporter

https://gerrit.wikimedia.org/r/1069242

zeljkofilipin changed the task status from In Progress to Stalled.Sep 13 2024, 9:27 AM
zeljkofilipin removed zeljkofilipin as the assignee of this task.
zeljkofilipin raised the priority of this task from Low to Needs Triage.
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin moved this task from Deep work 🌊 to Waiting 🏹 on the User-zeljkofilipin board.

Change #1070208 merged by jenkins-bot:

[mediawiki/extensions/Sanctions@master] selenium: Use @wdio/spec-reporter

https://gerrit.wikimedia.org/r/1070208

zeljkofilipin claimed this task.

T373863: CI broken for Sanctions is resolved, the last remaining commit (1070208) is merged.

I'm not updating FemiwikiSkin since it's hosted on gihub (instead of gerrit) and uses yarn (instead of npm).