Mentioned in SAL (#wikimedia-operations) [2022-03-14T12:03:48Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3314 (T298294)', diff saved to https://phabricator.wikimedia.org/P22411 and previous config saved to /var/cache/conftool/dbconfig/20220314-120347-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-14T12:49:03Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3314 (T298294)', diff saved to https://phabricator.wikimedia.org/P22415 and previous config saved to /var/cache/conftool/dbconfig/20220314-124902-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-14T12:49:12Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1147 (T298294)', diff saved to https://phabricator.wikimedia.org/P22416 and previous config saved to /var/cache/conftool/dbconfig/20220314-124911-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-14T13:12:21Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1147 (T298294)', diff saved to https://phabricator.wikimedia.org/P22418 and previous config saved to /var/cache/conftool/dbconfig/20220314-131220-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-14T13:57:36Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1147 (T298294)', diff saved to https://phabricator.wikimedia.org/P22424 and previous config saved to /var/cache/conftool/dbconfig/20220314-135736-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-14T13:57:44Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1148 (T298294)', diff saved to https://phabricator.wikimedia.org/P22425 and previous config saved to /var/cache/conftool/dbconfig/20220314-135744-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-14T14:25:02Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1148 (T298294)', diff saved to https://phabricator.wikimedia.org/P22426 and previous config saved to /var/cache/conftool/dbconfig/20220314-142502-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-14T15:10:18Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1148 (T298294)', diff saved to https://phabricator.wikimedia.org/P22431 and previous config saved to /var/cache/conftool/dbconfig/20220314-151017-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-14T15:10:26Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1149 (T298294)', diff saved to https://phabricator.wikimedia.org/P22432 and previous config saved to /var/cache/conftool/dbconfig/20220314-151025-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-14T15:39:46Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1149 (T298294)', diff saved to https://phabricator.wikimedia.org/P22435 and previous config saved to /var/cache/conftool/dbconfig/20220314-153945-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-14T16:25:01Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1149 (T298294)', diff saved to https://phabricator.wikimedia.org/P22441 and previous config saved to /var/cache/conftool/dbconfig/20220314-162501-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-14T16:25:09Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1160 (T298294)', diff saved to https://phabricator.wikimedia.org/P22442 and previous config saved to /var/cache/conftool/dbconfig/20220314-162509-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-14T16:49:28Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1160 (T298294)', diff saved to https://phabricator.wikimedia.org/P22444 and previous config saved to /var/cache/conftool/dbconfig/20220314-164927-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-14T17:34:43Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1160 (T298294)', diff saved to https://phabricator.wikimedia.org/P22448 and previous config saved to /var/cache/conftool/dbconfig/20220314-173442-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-14T17:53:54Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1146:3314 (T298294)', diff saved to https://phabricator.wikimedia.org/P22449 and previous config saved to /var/cache/conftool/dbconfig/20220314-175352-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-14T18:17:10Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1146:3314 (T298294)', diff saved to https://phabricator.wikimedia.org/P22450 and previous config saved to /var/cache/conftool/dbconfig/20220314-181709-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-14T19:02:25Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1146:3314 (T298294)', diff saved to https://phabricator.wikimedia.org/P22454 and previous config saved to /var/cache/conftool/dbconfig/20220314-190224-marostegui.json

Going for s7

Mentioned in SAL (#wikimedia-operations) [2022-03-16T07:52:49Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1127 (T298294)', diff saved to https://phabricator.wikimedia.org/P22655 and previous config saved to /var/cache/conftool/dbconfig/20220316-075248-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-16T07:54:48Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1127 (T298294)', diff saved to https://phabricator.wikimedia.org/P22656 and previous config saved to /var/cache/conftool/dbconfig/20220316-075448-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-16T08:40:04Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1127 (T298294)', diff saved to https://phabricator.wikimedia.org/P22663 and previous config saved to /var/cache/conftool/dbconfig/20220316-084003-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-16T08:40:12Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1101:3317 (T298294)', diff saved to https://phabricator.wikimedia.org/P22664 and previous config saved to /var/cache/conftool/dbconfig/20220316-084011-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-16T08:42:20Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1101:3317 (T298294)', diff saved to https://phabricator.wikimedia.org/P22667 and previous config saved to /var/cache/conftool/dbconfig/20220316-084219-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-16T09:27:35Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1101:3317 (T298294)', diff saved to https://phabricator.wikimedia.org/P22672 and previous config saved to /var/cache/conftool/dbconfig/20220316-092735-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-16T09:27:44Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1098:3317 (T298294)', diff saved to https://phabricator.wikimedia.org/P22673 and previous config saved to /var/cache/conftool/dbconfig/20220316-092742-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-16T09:29:48Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1098:3317 (T298294)', diff saved to https://phabricator.wikimedia.org/P22674 and previous config saved to /var/cache/conftool/dbconfig/20220316-092947-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-16T10:15:03Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1098:3317 (T298294)', diff saved to https://phabricator.wikimedia.org/P22681 and previous config saved to /var/cache/conftool/dbconfig/20220316-101502-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-16T10:17:30Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1181 (T298294)', diff saved to https://phabricator.wikimedia.org/P22682 and previous config saved to /var/cache/conftool/dbconfig/20220316-101729-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-16T10:18:48Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1181 (T298294)', diff saved to https://phabricator.wikimedia.org/P22683 and previous config saved to /var/cache/conftool/dbconfig/20220316-101848-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-16T11:04:03Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1181 (T298294)', diff saved to https://phabricator.wikimedia.org/P22688 and previous config saved to /var/cache/conftool/dbconfig/20220316-110403-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-16T11:04:12Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1174 (T298294)', diff saved to https://phabricator.wikimedia.org/P22689 and previous config saved to /var/cache/conftool/dbconfig/20220316-110411-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-16T11:15:32Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1174 (T298294)', diff saved to https://phabricator.wikimedia.org/P22690 and previous config saved to /var/cache/conftool/dbconfig/20220316-111532-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-16T12:00:48Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1174 (T298294)', diff saved to https://phabricator.wikimedia.org/P22698 and previous config saved to /var/cache/conftool/dbconfig/20220316-120047-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-16T12:01:01Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1158 (T298294)', diff saved to https://phabricator.wikimedia.org/P22699 and previous config saved to /var/cache/conftool/dbconfig/20220316-120100-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-16T12:02:19Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1158 (T298294)', diff saved to https://phabricator.wikimedia.org/P22700 and previous config saved to /var/cache/conftool/dbconfig/20220316-120219-marostegui.json

I have altered s6 master as the table were not big.

Going for s5 master now with a sleep between wikis

Mentioned in SAL (#wikimedia-operations) [2022-03-16T12:47:35Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1158 (T298294)', diff saved to https://phabricator.wikimedia.org/P22709 and previous config saved to /var/cache/conftool/dbconfig/20220316-124734-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-16T12:47:43Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1170:3317 (T298294)', diff saved to https://phabricator.wikimedia.org/P22710 and previous config saved to /var/cache/conftool/dbconfig/20220316-124742-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-16T12:49:44Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1170:3317 (T298294)', diff saved to https://phabricator.wikimedia.org/P22711 and previous config saved to /var/cache/conftool/dbconfig/20220316-124943-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-03-16T13:35:00Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1170:3317 (T298294)', diff saved to https://phabricator.wikimedia.org/P22721 and previous config saved to /var/cache/conftool/dbconfig/20220316-133458-marostegui.json

This is all done, pending only the masters.

I have finally done s1 master as well. The table was quite big (1.5GB) but the last update was received at 03:50AM UTC, so I decided to give it a try, and it was done in around 4 minutes.

In T298294#7784608, @Marostegui wrote:

I have finally done s1 master as well. The table was quite big (1.5GB) but the last update was received at 03:50AM UTC, so I decided to give it a try, and it was done in around 4 minutes.

This table gets an update if someone deletes or undeletes an image (or file in general). So writes should be pretty small on most wikis. Wanna try on s3?

I can make auto_schema accommodate for these cases (e.g. a ten seconds sleep between each wiki if it's the master of active dc)

Nah, s3 has a master swap scheduled that I plan to do as soon as T303605 is done - no worth the risk now.

Mentioned in SAL (#wikimedia-operations) [2022-03-29T09:11:15Z] <marostegui> dbmaint s3@eqiad T298294

Mentioned in SAL (#wikimedia-operations) [2022-04-12T08:51:57Z] <marostegui> dbmaint s4@eqiad T298294