[go: up one dir, main page]
Page MenuHomePhabricator
Create Task
Maniphest T364363

[Epic] Productionize federated wdqs graph-split endpoints
Open, HighPublic
Actions

Description

This epic tracks the SRE operational work to get at least a couple wdqs hosts fully in production with the new graph split way of doing things (main vs scholarly graphs).

Related Objects
Search...

StatusSubtypeAssignedTask
 OpenNoneT335067 Epic: Wikidata Query Service stabilization
 OpenNoneT337013 [Epic] Splitting the graph in WDQS
 OpenNoneT364363 [Epic] Productionize federated wdqs graph-split endpoints
 ResolvedRKemperT364364 Provision DNS and certificates for wdqs graph split domains
ResolvedStevemuneneT364366 Implement puppet logic for wdqs graph split
 ResolvedStevemuneneT364367 Create dedicated UIs for wdqs graph split endpoints
 ResolvedStevemuneneT364368 Create separate pybal pools for wdqs graph split (main vs scholarly)
 ResolvedStevemuneneT372919 Bring wqds2024 back into service
 ResolvedRKemperT373145 Create new service catalog entries for wdqs-main and wdqs-scholarly
 ResolvedRKemperT364077 Adapt the wdqs data-transfer cookbook to operate with federated subgraphs
 InvalidGehelT368972 Use Envoy instead of LVS to route internal federation traffic for WDQS
 ResolvedRKemperT370754 Import WDQS subgraphs to production nodes
 ResolvedItamarWMDET371346 Add a banner to query.wikidata.org to inform users about the scholarly articles split
 ResolvedStevemuneneT371833 Tear down wdqs graph split experimental endpoints
 DuplicateNoneT372816 Add federated queries in the test queries used to validate deployment
 ResolvedRKemperT372818 Verify and correct WDQS Grafana dashboard with graph split
 ResolveddcausseT373391 Create wdqs-main and wdqs-scholarly specific test queries
 ResolvedbkingT373791 Transfer a sane journal (subgraph:main) to wdqs2021 from wdqs2022
 ResolveddcausseT373812 Internal federation sometimes fail with HttpConnectionOverHTTP

Event Timeline

RKemper created this task.May 7 2024, 6:25 AM
Gehel moved this task from Current work to Ops / SRE on the Discovery-Search board.May 10 2024, 8:16 AM
Gehel edited projects, added Discovery-Search; removed Discovery-Search (Current work).
Gehel added a project: Data-Platform-SRE.
Gehel triaged this task as High priority.May 10 2024, 8:22 AM
Gehel moved this task from Incoming to Quarterly Goals on the Data-Platform-SRE board.
Gehel moved this task from Incoming to Operations/SRE on the Wikidata-Query-Service board.May 13 2024, 12:47 PM
RKemper added a subtask: T364077: Adapt the wdqs data-transfer cookbook to operate with federated subgraphs.Jun 20 2024, 4:03 PM
Stevemunene closed subtask T364364: Provision DNS and certificates for wdqs graph split domains as Resolved.Jun 26 2024, 12:15 PM
Gehel reopened subtask T364364: Provision DNS and certificates for wdqs graph split domains as Open.Jul 8 2024, 6:41 PM
Gehel closed subtask T368972: Use Envoy instead of LVS to route internal federation traffic for WDQS as Invalid.Jul 9 2024, 10:04 AM
RKemper closed subtask T364367: Create dedicated UIs for wdqs graph split endpoints as Resolved.Jul 16 2024, 5:57 PM
dcausse added a subtask: T371346: Add a banner to query.wikidata.org to inform users about the scholarly articles split.Jul 30 2024, 7:17 AM
dcausse mentioned this in T371871: Very high maxlag on Wikidata due to WDQS lag.Aug 6 2024, 9:30 AM
Lucas_Werkmeister_WMDE mentioned this in T369079: Update `UniqueValueChecker` to query a list of endpoints.Aug 15 2024, 11:21 AM
Gehel closed subtask T364366: Implement puppet logic for wdqs graph split as Resolved.Aug 21 2024, 9:34 AM
Gehel mentioned this in T364366: Implement puppet logic for wdqs graph split.
dr0ptp4kt mentioned this in T371833: Tear down wdqs graph split experimental endpoints.Aug 23 2024, 10:52 AM
dcausse added a subtask: T373791: Transfer a sane journal (subgraph:main) to wdqs2021 from wdqs2022.Sep 2 2024, 9:49 AM
dcausse added a subtask: T373812: Internal federation sometimes fail with HttpConnectionOverHTTP.Sep 2 2024, 2:25 PM
Stevemunene closed subtask T371833: Tear down wdqs graph split experimental endpoints as Resolved.Sep 4 2024, 9:18 AM
Stevemunene closed subtask T364368: Create separate pybal pools for wdqs graph split (main vs scholarly) as Resolved.
RKemper changed the status of subtask T372818: Verify and correct WDQS Grafana dashboard with graph split from Open to In Progress.Sep 5 2024, 10:01 PM
bking closed subtask T373791: Transfer a sane journal (subgraph:main) to wdqs2021 from wdqs2022 as Resolved.Sep 11 2024, 5:32 PM
RKemper closed subtask T364364: Provision DNS and certificates for wdqs graph split domains as Resolved.Sep 17 2024, 5:44 PM
Lydia_Pintscher closed subtask T371346: Add a banner to query.wikidata.org to inform users about the scholarly articles split as Resolved.Sep 18 2024, 8:24 AM
Gehel closed subtask T370754: Import WDQS subgraphs to production nodes as Resolved.Sep 20 2024, 9:04 AM
Gehel moved this task from Quarterly Goals to 2024.09.06 - 2024.09.27 on the Data-Platform-SRE board.Sep 24 2024, 2:30 PM
Gehel edited projects, added Data-Platform-SRE (2024.09.06 - 2024.09.27); removed Data-Platform-SRE.
Gehel edited projects, added Data-Platform-SRE (2024.09.28 - 2024.10.18); removed Data-Platform-SRE (2024.09.06 - 2024.09.27).Sep 27 2024, 1:00 PM
Gehel closed subtask T373391: Create wdqs-main and wdqs-scholarly specific test queries as Resolved.Sep 27 2024, 1:11 PM
RKemper closed subtask T372818: Verify and correct WDQS Grafana dashboard with graph split as Resolved.Oct 1 2024, 2:47 PM
Stevemunene closed subtask T364077: Adapt the wdqs data-transfer cookbook to operate with federated subgraphs as Resolved.Oct 4 2024, 7:31 AM
BTullis edited projects, added Data-Platform-SRE (2024.10.19 - 2024.11.08); removed Data-Platform-SRE (2024.09.28 - 2024.10.18).Oct 18 2024, 3:09 PM
Gehel closed subtask T373812: Internal federation sometimes fail with HttpConnectionOverHTTP as Resolved.Fri, Nov 1, 9:53 AM
Gehel edited projects, added Data-Platform-SRE (2024.11.09 - 2024.11.29); removed Data-Platform-SRE (2024.10.19 - 2024.11.08).Fri, Nov 8, 9:48 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits