Milestone for Data Platform SRE work
Details
Details
Description
Wed, Nov 20
Wed, Nov 20
gerritbot added a comment to T327259: Enable the Container Storage Interface (CSI) and the Ceph CSI plugin on dse-k8s cluster.
Change #1093333 merged by Btullis:
[operations/puppet@production] Upgrade the remainder of the cephosd cluster to nftables
gerritbot added a comment to T327259: Enable the Container Storage Interface (CSI) and the Ceph CSI plugin on dse-k8s cluster.
Change #1093333 had a related patch set uploaded (by Btullis; author: Btullis):
[operations/puppet@production] Upgrade the remainder of the cephosd cluster to nftables
gerritbot added a comment to T327259: Enable the Container Storage Interface (CSI) and the Ceph CSI plugin on dse-k8s cluster.
Change #1089716 merged by Btullis:
[operations/puppet@production] Canary cephosd1001 to use nftables instead of iptables
Mon, Nov 11
Mon, Nov 11
gerritbot added a comment to T327259: Enable the Container Storage Interface (CSI) and the Ceph CSI plugin on dse-k8s cluster.
Change #1089716 had a related patch set uploaded (by Btullis; author: Btullis):
[operations/puppet@production] Canary cephosd1001 to use nftables instead of iptables
Oct 21 2024
Oct 21 2024
brouberol added a comment to T327259: Enable the Container Storage Interface (CSI) and the Ceph CSI plugin on dse-k8s cluster.
As an aside, we have enabled both the ceph-csi-rdb and ceph-csi-cephfs (https://phabricator.wikimedia.org/T376401) storage interfaces in dse-k8s-eqiad.
gerritbot added a comment to T327259: Enable the Container Storage Interface (CSI) and the Ceph CSI plugin on dse-k8s cluster.
Change #1079542 abandoned by Brouberol:
[operations/puppet@production] envoy: Fix firewall_srange not being taken into account
Oct 11 2024
Oct 11 2024
gerritbot added a comment to T327259: Enable the Container Storage Interface (CSI) and the Ceph CSI plugin on dse-k8s cluster.
Change #1079545 merged by Btullis:
[operations/puppet@production] Revert cephosd servers from nftables to ferm
gerritbot added a comment to T327259: Enable the Container Storage Interface (CSI) and the Ceph CSI plugin on dse-k8s cluster.
Change #1079545 had a related patch set uploaded (by Btullis; author: Btullis):
[operations/puppet@production] Revert cephosd servers from nftables to ferm
gerritbot added a comment to T327259: Enable the Container Storage Interface (CSI) and the Ceph CSI plugin on dse-k8s cluster.
Change #1079542 had a related patch set uploaded (by Brouberol; author: Brouberol):
[operations/puppet@production] ceph.server: invert firewall::src_sets and firewall::srange hiera values
gerritbot added a comment to T327259: Enable the Container Storage Interface (CSI) and the Ceph CSI plugin on dse-k8s cluster.
Change #1050331 merged by Btullis:
[operations/puppet@production] cephosd: Switch to use nftables instead of iptables
Oct 10 2024
Oct 10 2024
gerritbot added a comment to T327259: Enable the Container Storage Interface (CSI) and the Ceph CSI plugin on dse-k8s cluster.
Change #1050330 merged by Btullis:
[operations/puppet@production] Switch cephosd1001 to use the nftables based firewall
Oct 9 2024
Oct 9 2024
Maintenance_bot removed a project from T364367: Create dedicated UIs for wdqs graph split endpoints: Patch-For-Review.
gerritbot added a comment to T364367: Create dedicated UIs for wdqs graph split endpoints.
Change #1078664 merged by jenkins-bot:
[operations/deployment-charts@master] wikidata-query-gui: remove experimental endpoints
Oct 8 2024
Oct 8 2024
gerritbot added a comment to T364367: Create dedicated UIs for wdqs graph split endpoints.
Change #1078664 had a related patch set uploaded (by Jelto; author: Jelto):
[operations/deployment-charts@master] wikidata-query-gui: remove experimental endpoints
Sep 25 2024
Sep 25 2024
Sep 13 2024
Sep 13 2024
Maintenance_bot removed a project from T371210: Create the airflow-analytics-test domain and certs: Patch-For-Review.
gerritbot added a comment to T371210: Create the airflow-analytics-test domain and certs.
Change #1057830 abandoned by Stevemunene:
[operations/puppet@production] trafficserver: add airflow-analytics-test discovery record
Reason:
Abandoned in favour of airflow-test-k8s done here https://gerrit.wikimedia.org/r/c/operations/puppet/+/1063848
Sep 6 2024
Sep 6 2024
Ahoelzl set the point value for T367848: Publishing conda environments with WMF Data Workflow Utils is broken to 1.
Aug 26 2024
Aug 26 2024
RKemper updated the task description for T364367: Create dedicated UIs for wdqs graph split endpoints.
gerritbot added a comment to T364367: Create dedicated UIs for wdqs graph split endpoints.
Change #1066812 merged by Ryan Kemper:
[operations/puppet@production] wdqs graph split: routing for wdqs backends
gerritbot added a project to T364367: Create dedicated UIs for wdqs graph split endpoints: Patch-For-Review.
gerritbot added a comment to T364367: Create dedicated UIs for wdqs graph split endpoints.
Change #1066812 had a related patch set uploaded (by Ryan Kemper; author: Ryan Kemper):
[operations/puppet@production] wdqs graph split: routing for wdqs backends
Aug 16 2024
Aug 16 2024
brouberol closed T372620: Report on the initial growthbook installation PoC, a subtask of T365839: Deploy an instance of GrowthBook to Kubernetes, as Resolved.
Aug 13 2024
Aug 13 2024
Stevemunene closed T356230: Conda-Analytics packages incompatible with latest versions of Pandas and Numpy as Resolved.
Marking this as resolved in favour of the created tasks on upgrading numpy, pyarrow, pandas and pyspark.
Aug 3 2024
Aug 3 2024
Maintenance_bot removed a project from T364367: Create dedicated UIs for wdqs graph split endpoints: Patch-For-Review.
gerritbot added a comment to T364367: Create dedicated UIs for wdqs graph split endpoints.
Change #1053765 merged by Ryan Kemper:
[operations/puppet@production] wdqs graph split: routing for wdqs backends
Jul 31 2024
Jul 31 2024
bking updated the task description for T371061: Update CirrusSearch dashboards to use new metrics/refresh dashboards.
bking updated Other Assignee for T368760: Configure airflow webserver under Kubernetes to use OIDC authentication, added: bking.
gerritbot added a comment to T364368: Create separate pybal pools for wdqs graph split (main vs scholarly).
Change #1046120 abandoned by Stevemunene:
[operations/puppet@production] [WIP] wdqs: create wdqs split pybal pools
Reason:
Duplicate of https://gerrit.wikimedia.org/r/c/operations/puppet/+/1054520
Jul 30 2024
Jul 30 2024
Maintenance_bot removed a project from T363001: Create a helm chart for airflow that is appropriate to our needs: Patch-For-Review.
gerritbot added a comment to T363001: Create a helm chart for airflow that is appropriate to our needs.
Change #1041759 merged by Bking:
[operations/deployment-charts@master] dse-k8s-services: Add net-new chart for Airflow
bking added a comment to T368033: Design a suitable DAG deployment method.
Thanks @hashar and @Ottomata for following up. Based on this exchange as well as the comments in the Gitlab HDFS synchronizer design doc , I think DPE SRE and Releng are on the same page as far as the potential security issues posed by this CD implementation.
gerritbot added a comment to T364367: Create dedicated UIs for wdqs graph split endpoints.
Change #1057889 merged by Ryan Kemper:
[wikidata/query/gui-deploy@production] wdqs-main: set title to "WDQS main graph"
gerritbot added a comment to T364367: Create dedicated UIs for wdqs graph split endpoints.
Change #1057889 had a related patch set uploaded (by Ryan Kemper; author: DCausse):
[wikidata/query/gui-deploy@production] wdqs-main: set title to "WDQS main graph"
Maintenance_bot removed a project from T368757: Create a git-sync container image to be used with airflow: Patch-For-Review.
Stevemunene moved T365449: Upgrade Airflow to 2.9.3 from In Progress to To Be Deployed on the Data-Platform-SRE (2024.07.08 - 2024.07.28) board.
The airflow v2.9.3 is ready to be deployed for testing on the an-test-client1002
CodeReviewBot added a comment to T368757: Create a git-sync container image to be used with airflow.
Stashbot added a comment to T368518: decommission clouddb1021.
Mentioned in SAL (#wikimedia-operations) [2024-07-30T13:58:18Z] <marostegui> Remove clouddb1021 from zarcillo database T368518
Marostegui added a comment to T368518: decommission clouddb1021.
I am going to remove this host from zarcillo database - even if it is used for reimage tests it will be eventually decommissioned.
Stevemunene added a comment to T365449: Upgrade Airflow to 2.9.3.
With the tags work around we have been able to release the airflow 2.9.3 upgrade, I don't think we shall need to rework the pipeline as is since we are moving to dse-k8s. Thanks for the help @Jelto and @LSobanski
gerritbot added a comment to T371209: Create airflow-test-k8s OIDC configuration.
Change #1057805 merged by Stevemunene:
[operations/dns@master] dns: provision airflow-analytics-test domain
gerritbot added a comment to T371210: Create the airflow-analytics-test domain and certs.
Change #1057805 merged by Stevemunene:
[operations/dns@master] dns: provision airflow-analytics-test domain
Jul 29 2024
Jul 29 2024