Wikimedia deployers audit
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	sbassett
	Nov 7 2019, 11:08 PM

Description

The Security-Team recently reviewed membership of the Wikimedia deployment group. We validated a few things such as affiliation (WMF, WMDE) a valid NDA on file with WMF-Legal and the last date an individual performed a deploy according to the SAL and archives. These results are partially generated by a python script (soon to live within wikimedia/security/tooling) and exist within a protected Google sheet. The following is a subset of the aforementioned results and includes users who have not performed a deploy within the last two years. The Security-Team believes that if these individuals do not regularly perform deploys and are unlikely to do so any time in the future, we should remove their deployment rights, given the sensitivity and power of such rights. Some of these users very recently received these rights and should not be considered within this discussion.

Updated 2020-02-25:

Shell username	Name	WMF	WMDE	WMF Legal NDA?	Last Deployed	Date Updated
aude	Katie Filbert (former WMDE)	0	0	1	No deploys last 2 years	2020-02-25 21:06
awjrichards	Arthur Richards	1	0	1	No deploys last 2 years	2020-02-25 21:06
daniel	Daniel Kinzler	1	0	1	No deploys last 2 years	2020-02-25 21:06
demon	Chad Horohoe (former WMF)	0	0	1	No deploys last 2 years	2020-02-25 21:06
dereckson	Sébastien Santoro	0	0	1	No deploys last 2 years	2020-02-25 21:06
dr0ptp4kt	Adam Baso	1	0	1	No deploys last 2 years	2020-02-25 21:06
dsharpe	David Sharpe	1	0	1	No deploys last 2 years	2020-02-25 21:06
esanders	Ed Sanders	1	0	1	No deploys last 2 years	2020-02-25 21:06
jdl	Jason David Linehan	1	0	1	No deploys last 2 years	2020-02-25 21:06
jfishback	James Fishback	1	0	1	No deploys last 2 years	2020-02-25 21:06
kaldari	Ryan Kaldari	1	0	1	No deploys last 2 years	2020-02-25 21:06
mattflaschen	Matt Flaschen (former WMF)	0	0	1	No deploys last 2 years	2020-02-25 21:06
mstyles	Maryam Styles	1	0	1	No deploys last 2 years	2020-02-25 21:06
musikanimal	Leon Ziemba	1	0	1	No deploys last 2 years	2020-02-25 21:06
niedzielski	Steven Niedzielski	0	0	1	No deploys last 2 years	2020-02-25 21:06
phuedx	Sam Smith	1	0	1	No deploys last 2 years	2020-02-25 21:06
samwilson	Sam Wilson	1	0	1	No deploys last 2 years	2020-02-25 21:06
wmde-leszek	Leszek Manicki	0	1	1	No deploys last 2 years	2020-02-25 21:06
zpapierski	Zbyszko Papierski	1	0	1	No deploys last 2 years	2020-02-25 21:06

@MoritzMuehlenhoff - do you or anyone else in SRE have any opinions on this? The Security-Team would like to perform this audit and make any necessary access adjustments on a quarterly basis.

Details

	Subject	Repo	Branch	Lines +/-
	Deployment group audit	operations/puppet	production	+16 -16
	Security Tools: Deployer Audit	wikimedia/security/tooling	master	+278 -0

Customize query in gerrit

Related Objects

Mentioned In: T258119: Requesting access to deployment for jlinehan
T254201: Compile, organize and schedule various Wikimedia security-related user audits
T246388: Deployer Audit script improvements
Mentioned Here: T258119: Requesting access to deployment for jlinehan
P10586 T237696
T246388: Deployer Audit script improvements
T161181: Requesting access to deploy hosts for musikanimal
T238960: Conversion to volunteer NDA for MaxSem

Event Timeline

sbassett created this task.Nov 7 2019, 11:08 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 7 2019, 11:08 PM

sbassett claimed this task.Nov 7 2019, 11:09 PM

sbassett triaged this task as Medium priority.

sbassett added a project: Security-Team.

sbassett moved this task from Incoming to Watching on the Security-Team board.

sbassett edited subscribers, added: Reedy, • JBennett, • chasemp, • Dsharpe; removed: sbassett.

Sure, no objections. Access to the deployers group is overseen by @greg, so just sync up with him on removing people. When you have a final list of people to remove from the group, just open a Phab task, tag it "Operations" and "SRE-Access-Requests" and whoever is on SRE Clinic Duty the week will deal with removing the access.

sbassett added a project: Release-Engineering-Team.Nov 12 2019, 5:27 PM

• chasemp added a project: Security.Feb 10 2020, 10:56 PM

sbassett added a project: user-sbassett.Feb 18 2020, 3:54 PM

• chasemp removed a project: acl*security.Feb 20 2020, 8:06 PM

sbassett updated the task description. (Show Details)Feb 25 2020, 9:08 PM

sbassett updated the task description. (Show Details)Feb 25 2020, 9:29 PM

sbassett changed the visibility from "Custom Policy" to "Public (No Login Required)".

Change 574869 had a related patch set uploaded (by SBassett; owner: SBassett):
[operations/puppet@production] Deployment group audit

https://gerrit.wikimedia.org/r/574869

gerritbot added a project: Patch-For-Review.Feb 25 2020, 9:29 PM

@MoritzMuehlenhoff - patch is up, see above. I wanted to make this task public and give those being removed of their deployment rights a chance to respond if 1) they still have good reason to keep deployment 2) my audit script made a mistake combing the SAL.

sbassett updated the task description. (Show Details)Feb 25 2020, 9:39 PM

sbassett moved this task from Backlog to In Progress on the user-sbassett board.

So should we add those users as subscribers here on the ticket so that they get notified?

• chasemp awarded a token.Feb 25 2020, 9:50 PM

• chasemp moved this task from Watching to In Progress on the Security-Team board.Feb 25 2020, 10:05 PM

In T237696#5917908, @Dzahn wrote:

So should we add those users as subscribers here on the ticket so that they get notified?

Done.

sbassett updated the task description. (Show Details)Feb 25 2020, 10:55 PM

sbassett added subscribers: MaxSem, Mattflaschen-Personal.

Krenair subscribed.Feb 25 2020, 11:01 PM

Peachey88 subscribed.Feb 26 2020, 3:34 AM

For myself, I don't do deployments any more (now that I'm managing). I suspect the same is true for Adam.

kaldari updated the task description. (Show Details)Feb 26 2020, 8:27 PM

kaldari updated the task description. (Show Details)

I am in a similar boat as Kaldari. It has been many years since I accidentally took Wikipedia offline :p Please feel free to remove me as a deployer.

I've recently stepped sideways into development from management and will likely be doing more deployments in the future. I can't give an expected date of when I'll next be deploying though.

Your stats are wrong if they think it was more than 2 years since I've deployed:) And I'm not in the deployers group anymore either. Anyway, I was under the impressionthat my account was reenabled to keep me in the WMF-NDA LDAP group, as requested in T238960.

In T237696#5923215, @MaxSem wrote:

Anyway, I was under the impressionthat my account was reenabled to keep me in the WMF-NDA LDAP group, as requested in T238960.

That's correct, Max is currently only in the cn=nda LDAP group and doesn't have any shell access.

In T237696#5923379, @MoritzMuehlenhoff wrote:

That's correct, Max is currently only in the cn=nda LDAP group and doesn't have any shell access.

But when he still had deployment access he deployed and that is not over 2 years ago. So something went wrong here.

https://gerrit.wikimedia.org/r/q/project:operations%252Fmediawiki-config+owner:maxsem.wiki%2540gmail.com

In T237696#5923751, @Dzahn wrote:

But when he still had deployment access he deployed and that is not over 2 years ago. So something went wrong here.

https://gerrit.wikimedia.org/r/q/project:operations%252Fmediawiki-config+owner:maxsem.wiki%2540gmail.com

Yes, thanks all for the feedback on this. I've removed MaxSem from the table within the task description. I'm guessing this was due in part to a merge of the data when this audit was initially run last November and, as noted, the automated searching of the SAL being a bit buggy. Hence why this task was made public and why I tried to add everyone whose access was to change as a result to this task and the patch.

In T237696#5922925, @phuedx wrote:

I've recently stepped sideways into development from management and will likely be doing more deployments in the future. I can't give an expected date of when I'll next be deploying though.

This is fine, I'll add you back on the patch.

I do not need or have a strong desire for deployment rights, however I would like to keep terbium access if possible. Could I be moved to the "restricted" group? My deployer rights were added way back when I was working on IP range contributions and needed to run queries on a new table (T161181). Incidentally, I'm back in the same position with our ongoing work on Expiring-Watchlist-Items. Pinging @aezell in case you need manager approval. Thanks!

In T237696#5921523, @kaldari wrote:

For myself, I don't do deployments any more (now that I'm managing). I suspect the same is true for Adam.

In T237696#5921579, @Awjrichards wrote:

I am in a similar boat as Kaldari. It has been many years since I accidentally took Wikipedia offline :p Please feel free to remove me as a deployer.

Noted, thanks!

In T237696#5923923, @MusikAnimal wrote:

I do not need or have a strong desire for deployment rights, however I would like to keep terbium access if possible. Could I be moved to the "restricted" group?

Sounds reasonable to me, done in PS3.

In T237696#5923923, @MusikAnimal wrote:

however I would like to keep terbium access if possible. Could I be moved to the "restricted" group?

FYI, terbium does not exist anymore. use "mwmaint1001.eqiad.wmnet" and "mwmaint2001.codfw.wmnet"

Yes, the restricted group will give you access to that just like deployment does.

@sbassett Hey, thanks for this. Does the script also provide when they acquired their rights? That'd be useful for me to review these from my end. Thanks!

DannyS712 subscribed.Feb 27 2020, 7:10 PM

In T237696#5924354, @greg wrote:

@sbassett Hey, thanks for this.

You're welcome.

Does the script also provide when they acquired their rights? That'd be useful for me to review these from my end.

Not at the moment, but I've just filed T246388.

I've not deployed for a long time, and not often ever, however as the other deployers on my team (Community-Tech) are losing their rights or are generally feeling like they've got too much to do, we think it'd be good for me to re-train for deployments and keep the rights. Is that okay?

In T237696#5925494, @Samwilson wrote:

I've not deployed for a long time, and not often ever, however as the other deployers on my team (Community-Tech) are losing their rights or are generally feeling like they've got too much to do, we think it'd be good for me to re-train for deployments and keep the rights. Is that okay?

Added you back in PS4.

kaldari unsubscribed.Feb 29 2020, 12:24 AM

I'll review the current and merge if things seem up and up, assigning to me so I don't forget

Change 574869 merged by Rush:
[operations/puppet@production] Deployment group audit

https://gerrit.wikimedia.org/r/574869

P10586 T237696

1	Info: Using configured environment 'production'
2	Info: Retrieving pluginfacts
3	Info: Retrieving plugin
4	Info: Retrieving locales
5	Info: Loading facts
6	Info: Caching catalog for deploy1001.eqiad.wmnet
7	Info: Applying configuration version '(ab7b86bead) Rush - Deployment group audit'
8	Info: Computing checksum on file /etc/ssh/userkeys/aude
9	Info: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/aude]: Filebucketed /etc/ssh/userkeys/aude to puppet with sum 8fdeabe0e72af264970cbe66b4cf8486
10	Notice: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/aude]/ensure: removed
11	Info: Computing checksum on file /etc/ssh/userkeys/awjrichards
12	Info: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/awjrichards]: Filebucketed /etc/ssh/userkeys/awjrichards to puppet with sum 22a2bde854ab1e97f1ab893532000ed1
13	Notice: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/awjrichards]/ensure: removed
14	Info: Computing checksum on file /etc/ssh/userkeys/daniel
15	Info: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/daniel]: Filebucketed /etc/ssh/userkeys/daniel to puppet with sum c2839f6c641f40d0ebc184b7b2c8673d
16	Notice: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/daniel]/ensure: removed
17	Info: Computing checksum on file /etc/ssh/userkeys/dereckson
18	Info: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/dereckson]: Filebucketed /etc/ssh/userkeys/dereckson to puppet with sum 4c1ae16c9722911b583fb588676eb303
19	Notice: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/dereckson]/ensure: removed
20	Info: Computing checksum on file /etc/ssh/userkeys/dr0ptp4kt
21	Info: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/dr0ptp4kt]: Filebucketed /etc/ssh/userkeys/dr0ptp4kt to puppet with sum 8bbc35a7df0f04cce76bfe5b219c297e
22	Notice: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/dr0ptp4kt]/ensure: removed
23	Info: Computing checksum on file /etc/ssh/userkeys/dsharpe
24	Info: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/dsharpe]: Filebucketed /etc/ssh/userkeys/dsharpe to puppet with sum 4d924d1447bbc0da6cc694855781b945
25	Notice: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/dsharpe]/ensure: removed
26	Info: Computing checksum on file /etc/ssh/userkeys/esanders
27	Info: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/esanders]: Filebucketed /etc/ssh/userkeys/esanders to puppet with sum a34f3096ad1f33e56aad3899059a003c
28	Notice: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/esanders]/ensure: removed
29	Info: Computing checksum on file /etc/ssh/userkeys/jdl
30	Info: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/jdl]: Filebucketed /etc/ssh/userkeys/jdl to puppet with sum d2d4f5bc42ba50e95901220598c26d4e
31	Notice: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/jdl]/ensure: removed
32	Info: Computing checksum on file /etc/ssh/userkeys/jfishback
33	Info: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/jfishback]: Filebucketed /etc/ssh/userkeys/jfishback to puppet with sum a42bf9bf8c515a771353b91244577d66
34	Notice: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/jfishback]/ensure: removed
35	Info: Computing checksum on file /etc/ssh/userkeys/kaldari
36	Info: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/kaldari]: Filebucketed /etc/ssh/userkeys/kaldari to puppet with sum ecc334a7ee563e863aa215f108fbf5f3
37	Notice: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/kaldari]/ensure: removed
38	Info: Computing checksum on file /etc/ssh/userkeys/mattflaschen
39	Info: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/mattflaschen]: Filebucketed /etc/ssh/userkeys/mattflaschen to puppet with sum aa5a55e32b05184a9f8a8d74469225f8
40	Notice: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/mattflaschen]/ensure: removed
41	Info: Computing checksum on file /etc/ssh/userkeys/musikanimal
42	Info: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/musikanimal]: Filebucketed /etc/ssh/userkeys/musikanimal to puppet with sum 3665b28ca17c2e4f81b3e3cb419c2fba
43	Notice: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/musikanimal]/ensure: removed
44	Info: Computing checksum on file /etc/ssh/userkeys/wmde-leszek
45	Info: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/wmde-leszek]: Filebucketed /etc/ssh/userkeys/wmde-leszek to puppet with sum b8ee1749697d6718faf0948ba72839d2
46	Notice: /Stage[main]/Ssh::Server/File[/etc/ssh/userkeys/wmde-leszek]/ensure: removed
47	Notice: /Stage[main]/Admin/Admin::Groupmembers[deployment]/Exec[deployment_ensure_members]/returns: executed successfully
48	Notice: /Stage[main]/Admin/Exec[enforce-users-groups-cleanup]/returns: /usr/local/sbin/enforce-users-groups removing user/id: dr0ptp4kt/2962
49	Notice: /Stage[main]/Admin/Exec[enforce-users-groups-cleanup]/returns: /usr/local/sbin/enforce-users-groups removing user/id: aude/1185
50	Notice: /Stage[main]/Admin/Exec[enforce-users-groups-cleanup]/returns: /usr/local/sbin/enforce-users-groups removing user/id: esanders/2875
51	Notice: /Stage[main]/Admin/Exec[enforce-users-groups-cleanup]/returns: /usr/local/sbin/enforce-users-groups removing user/id: kaldari/1271
52	Notice: /Stage[main]/Admin/Exec[enforce-users-groups-cleanup]/returns: /usr/local/sbin/enforce-users-groups removing user/id: mattflaschen/2662
53	Notice: /Stage[main]/Admin/Exec[enforce-users-groups-cleanup]/returns: /usr/local/sbin/enforce-users-groups removing user/id: dereckson/2362
54	Notice: /Stage[main]/Admin/Exec[enforce-users-groups-cleanup]/returns: /usr/local/sbin/enforce-users-groups removing user/id: musikanimal/11106
55	Notice: /Stage[main]/Admin/Exec[enforce-users-groups-cleanup]/returns: /usr/local/sbin/enforce-users-groups removing user/id: jdl/20295
56	Notice: /Stage[main]/Admin/Exec[enforce-users-groups-cleanup]/returns: /usr/local/sbin/enforce-users-groups removing user/id: wmde-leszek/12300
57	Notice: /Stage[main]/Admin/Exec[enforce-users-groups-cleanup]/returns: /usr/local/sbin/enforce-users-groups removing user/id: dsharpe/20828
58	Notice: /Stage[main]/Admin/Exec[enforce-users-groups-cleanup]/returns: /usr/local/sbin/enforce-users-groups removing user/id: jfishback/21257
59	Notice: /Stage[main]/Admin/Exec[enforce-users-groups-cleanup]/returns: executed successfully
60	Notice: Applied catalog in 92.07 seconds

back to scott for confirmation and whatever else :)

• chasemp removed a project: Patch-For-Review.Mar 2 2020, 7:12 PM

@chasemp - LGTM, thanks for the deploy! I'll plan to track any access restoration requests that might trickle in.

sbassett moved this task from In Progress to Done on the user-sbassett board.Mar 2 2020, 7:19 PM

sbassett mentioned this in T254201: Compile, organize and schedule various Wikimedia security-related user audits.Jun 1 2020, 9:27 PM

• jlinehan mentioned this in T258119: Requesting access to deployment for jlinehan.Jul 15 2020, 10:59 PM

In T237696#5934288, @sbassett wrote:

@chasemp - LGTM, thanks for the deploy! I'll plan to track any access restoration requests that might trickle in.

@sbassett There is one at T258119

In T237696#6310782, @Dzahn wrote:

In T237696#5934288, @sbassett wrote:

@chasemp - LGTM, thanks for the deploy! I'll plan to track any access restoration requests that might trickle in.

@sbassett There is one at T258119

Thanks, @Dzahn - looks like that has now been resolved.

Wikimedia deployers auditClosed, ResolvedPublicActions

Description

Details

Related Objects

Event Timeline

Wikimedia deployers audit
Closed, ResolvedPublic
Actions