[go: up one dir, main page]
Page MenuHomePhabricator
Create Task
Maniphest T302038

[Epic] Define testing and QA strategy for Codex
Closed, InvalidPublic
Actions

Description

We need to define a comprehensive testing and QA strategy for Codex (specifically, for individual components as they get added to the library).

This comprehensive strategy needs to accomplish the following goals:

  • Define a way to manually test components in isolation as part of the development process (i.e. before release)
  • Define a way to manually test components in a MediaWiki environment (either before or after release)
    • MW's internationalization tools can be utilized to ensure that Codex components work properly across different languages, scripts, directionalities, etc
  • Define standards to write/structure automated component tests (mainly run in CI and required before a new component can be merged)
  • Define a way to automate visual regression testing and accessibility testing

Details

Other Assignee
NBaca-WMF

Related Objects
Search...

StatusSubtypeAssignedTask
 Duplicate• STHT302348 <Platform Initiative> Release Codex in Beta (Limited Production Release)
 InvalidNoneT302025 Upgrade Codex to Beta version
 InvalidNoneT302038 [Epic] Define testing and QA strategy for Codex
 ResolvedNoneT299143 Test/stage environment to view Codex on MediaWiki
 ResolvedegardnerT299956 Create a MediaWiki extension to test Codex
 Resolved• CatropeT299957 Automatically stage and deploy Codex changes inside MediaWiki
 Resolvedldelench_wmfT306180 [Epic] Define manual testing & QA strategy in Codex
ResolvedSpike• DAbadT287044 Determine how to perform QA testing of Codex components inside a MediaWiki environment
 ResolvedegardnerT309884 Figure out how to best test Codex in different language environments
 Resolvedldelench_wmfT312113 QTE team review & publishing: manual testing guidelines
 ResolvedegardnerT311243 Clean up the Vite sandbox
 Resolved• EUdoh-WMFT314792 Publish the sandbox as part of Netlify deployment previews
 Resolved• CatropeT316170 Copy latest version of Codex sandbox to VueTest
 ResolvedegardnerT318842 Enable embedding of Codex component demos inside of VueTest MediaWiki extension
 OpenNoneT307744 Draft Decision Record around Design Review
 Resolved• CatropeT308291 Define test cases for Codex functional testing
 Declined• CatropeT308446 Set up automated functional testing skeleton for Codex
 DeclinedNoneT310551 Set up basic infrastructure for Cypress component tests
 ResolvedSpikeAnneTT308447 Spike: Automated testing for visual regression design
 Resolved• CatropeT315621 Install VueTest extension in beta labs
 OpenNoneT315622 Make main branch of Codex available in beta labs
 ResolvedAnneTT325234 Make VueTest work on mobile view

Event Timeline

• STH created this task.Feb 17 2022, 10:14 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 17 2022, 10:14 PM
• STH added a parent task: T302025: Upgrade Codex to Beta version.Feb 17 2022, 10:15 PM
• STH added a subtask: T299143: Test/stage environment to view Codex on MediaWiki.
• STH changed the task status from Open to In Progress.Feb 17 2022, 10:19 PM
• STH triaged this task as High priority.
• STH added a subtask: T302039: Gather requirements for Codex QA workflows.
• STH mentioned this in T302025: Upgrade Codex to Beta version.Mar 10 2022, 4:38 PM
• STH moved this task from Design Systems Team FY2021-22 Kanban Board to FY21-22 Roadmap Epics on the Design-Systems-team-20200324-20220422 board.Mar 23 2022, 8:18 PM
• STH edited projects, added Design-Systems-team-20200324-20220422; removed Design-Systems-team-20200324-20220422 (Design Systems Team FY2021-22 Kanban Board).
ldelench_wmf edited projects, added Design-System-Team; removed Design-Systems-team-20200324-20220422.Apr 21 2022, 5:46 PM
ldelench_wmf moved this task from Inbox to Quarterly Plan on the Design-System-Team board.
• STH closed subtask T299143: Test/stage environment to view Codex on MediaWiki as Resolved.Apr 25 2022, 10:34 PM
• STH renamed this task from [Epic] Set up a QA workflow for Codex to QTE, testing and QA workflow for Codex.Apr 25 2022, 11:43 PM
• STH updated the task description. (Show Details)
• STH updated the task description. (Show Details)Apr 25 2022, 11:48 PM
• STH renamed this task from QTE, testing and QA workflow for Codex to [Needs Discussion] QTE, testing and QA workflow for Codex.Apr 26 2022, 12:09 AM
• STH added a subtask: T306180: [Epic] Define manual testing & QA strategy in Codex .
• STH renamed this task from [Needs Discussion] QTE, testing and QA workflow for Codex to Testing and QA strategy for Codex.Apr 30 2022, 3:53 PM
• STH changed the task status from In Progress to Stalled.
• STH assigned this task to • Catrope.
• STH updated Other Assignee, added: NBaca-WMF.
• STH updated the task description. (Show Details)
• STH moved this task from Quarterly Plan to Up Next on the Design-System-Team board.
• STH added a subtask: T291945: [Spike] Automated accessibility testing for Codex.Apr 30 2022, 4:23 PM
• STH moved this task from Up Next to Quarterly Plan on the Design-System-Team board.
• STH added a subtask: T282284: Develop guidelines for quality assessment of Vue.js projects.Apr 30 2022, 5:46 PM
• STH removed a parent task: T302025: Upgrade Codex to Beta version.May 6 2022, 6:11 PM
• STH closed subtask T302039: Gather requirements for Codex QA workflows as Resolved.
• STH changed the status of subtask T302039: Gather requirements for Codex QA workflows from Resolved to Invalid.
• STH moved this task from Quarterly Plan to Needs Refinement on the Design-System-Team board.May 6 2022, 8:18 PM
Volker_E subscribed.May 9 2022, 5:28 PM

@Catrope shared an important point about how to learn and deviate from OOUI release QA process. In OOUI we've run into post-release issues having visual breakages quite often (needed to have follow-up patch releases).
We should collaborate on a QA process that not only involves a dedicated QTE in design system team, we should also ensure to bridge QTE's to be informed about Codex releases and test in their respective teams.

• STH moved this task from Needs Refinement to Up Next on the Design-System-Team board.May 12 2022, 8:09 PM
• STH changed the task status from Stalled to Open.May 12 2022, 8:16 PM
• STH removed a subtask: T282284: Develop guidelines for quality assessment of Vue.js projects.May 12 2022, 8:51 PM
• STH removed a subtask: T302039: Gather requirements for Codex QA workflows.
• STH removed a subtask: T308289: Design Systems Test Engineer Onboarding .May 12 2022, 8:53 PM
• STH mentioned this in T308289: Design Systems Test Engineer Onboarding .
• STH removed a subtask: T291945: [Spike] Automated accessibility testing for Codex.May 12 2022, 8:56 PM
• STH added a subtask: T307744: Draft Decision Record around Design Review.May 12 2022, 8:58 PM
• STH updated the task description. (Show Details)May 12 2022, 9:10 PM
• STH moved this task from Up Next to Quarterly Plan on the Design-System-Team board.May 17 2022, 5:20 PM
• STH added a parent task: T302025: Upgrade Codex to Beta version.May 20 2022, 4:34 PM
• STH changed the status of subtask T306180: [Epic] Define manual testing & QA strategy in Codex from Open to In Progress.May 26 2022, 12:31 AM
• STH changed the task status from Open to In Progress.May 26 2022, 1:02 AM
egardner added a subtask: T287044: Determine how to perform QA testing of Codex components inside a MediaWiki environment.Jul 21 2022, 3:56 PM
egardner added a subtask: T309884: Figure out how to best test Codex in different language environments.Jul 21 2022, 5:48 PM
egardner removed a subtask: T287044: Determine how to perform QA testing of Codex components inside a MediaWiki environment.
egardner removed a subtask: T309884: Figure out how to best test Codex in different language environments.Jul 21 2022, 5:50 PM
egardner claimed this task.Aug 1 2022, 8:53 PM
egardner added a subscriber: • Catrope.
egardner renamed this task from Testing and QA strategy for Codex to [Epic] Define testing and QA strategy for Codex.Aug 1 2022, 8:55 PM
egardner added a parent task: T314082: Design System Infrastructure Capabilities Scoping.
egardner removed a parent task: T314082: Design System Infrastructure Capabilities Scoping.Aug 1 2022, 8:59 PM
egardner updated the task description. (Show Details)Aug 1 2022, 9:04 PM
egardner updated the task description. (Show Details)
egardner removed a subtask: T312113: QTE team review & publishing: manual testing guidelines.Aug 1 2022, 9:07 PM
egardner updated the task description. (Show Details)Aug 1 2022, 9:27 PM
ldelench_wmf moved this task from Quarterly Plan to Design-System-Product-Roadmap on the Design-System-Team board.Aug 12 2022, 1:37 PM
ldelench_wmf edited projects, added Design-System-Team (Design-System-Product-Roadmap); removed Design-System-Team.
ldelench_wmf moved this task from Now to Later on the Design-System-Team (Design-System-Product-Roadmap) board.Aug 12 2022, 2:08 PM
• DAbad moved this task from Later to Next on the Design-System-Team (Design-System-Product-Roadmap) board.Sep 26 2022, 4:21 PM
Restricted Application edited projects, added Design-System-Team; removed Design-System-Team (Design-System-Product-Roadmap). · View Herald TranscriptSep 26 2022, 4:21 PM
egardner removed egardner as the assignee of this task.Oct 14 2022, 7:02 PM
egardner subscribed.
egardner added a subtask: T315621: Install VueTest extension in beta labs.Nov 7 2022, 10:21 PM
egardner added a subtask: T315622: Make main branch of Codex available in beta labs.
AnneT closed subtask T325234: Make VueTest work on mobile view as Resolved.Dec 14 2022, 7:36 PM
egardner moved this task from Inbox to Design-System-Product-Roadmap on the Design-System-Team board.Dec 20 2022, 9:48 PM
egardner edited projects, added Design-System-Team (Design-System-Product-Roadmap); removed Design-System-Team.
AnneT changed the status of subtask T308447: Spike: Automated testing for visual regression design from Open to In Progress.Jan 4 2023, 2:27 PM
ldelench_wmf closed this task as Invalid.Jan 12 2023, 10:47 PM
ldelench_wmf subscribed.

Closing this epic in favor of the more actionable subtasks:

  • T306180 Define manual testing & QA strategy in Codex - we expect to resolve this as a first iteration within a few days
  • VueTest Improvements - epic to be created
ldelench_wmf closed subtask T308291: Define test cases for Codex functional testing as Resolved.Jan 12 2023, 10:49 PM
ldelench_wmf closed subtask T306180: [Epic] Define manual testing & QA strategy in Codex as Resolved.Jan 12 2023, 10:59 PM
AnneT closed subtask T308447: Spike: Automated testing for visual regression design as Resolved.Jan 23 2023, 6:01 PM
• Catrope closed subtask T308446: Set up automated functional testing skeleton for Codex as Declined.Feb 17 2023, 7:44 PM
• Catrope closed subtask T315621: Install VueTest extension in beta labs as Resolved.Feb 21 2023, 10:34 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits