Thanks @CDanis. For reference, there was some initial discussion about this functionality here on Slack.

I agree that this cross-correlation of data sources could be very useful to a number of different stakeholders, not least the SRE team.

The presto catalogs are defined in hieradata, both for the coordinator and worker roles.

So for the test cluster it's here for the coordinator and here for the workers. The prod cluster settings are similarly located.

I can't see any governance nor technical reason why we wouldn't want to add this Prometheus catalog to Presto, but I think that we should still seek approval before carrying out any work.

Perhaps @odimitrijevic would be the person to approve any integration. I've added a few other subscribers to invite their opinions and hopefully make sure that we're not barking up the wrong tree.

Also, I believe that @nshahquinn-wmf knows most about wmfdata-python so may be the person best placed to say whether or not Prometheus support would be easy or difficult to add to this library.

This looks super interesting, moving to radar for when we need to help out.

Adding this to our radar as well, to keep an eye when we start querying.

In T347430#9201371, @BTullis wrote:

Also, I believe that @nshahquinn-wmf knows most about wmfdata-python so may be the person best placed to say whether or not Prometheus support would be easy or difficult to add to this library.

From the Presto client point of view, is it correct that Prometheus would be accessed exactly the same as the Data Lake as now, just with a different catalog argument? If so, Wmfdata wouldn't need any changes to support that as presto.run already has a catalog argument (it currently defaults to "analytics-hive").

@Ahoelzl why was this moved to "Radar (External Teams)" column? Per @BTullis's post, I think this was awaiting DE approval before DE would work on it...?

@CDanis I believe people currently want this kind of work to be planned more strategically, and to prioritize it appropriately I agree this would be very useful!

I'm working on compiling user stories based on the recent data usage at wmf interviews, in which I see you did call this need out! So, it will get a user story which @VirginiaPoundstone and other PMs will be looking at along with many others.

However, I agree Radar isn't quite the right place for this. I'll move it back to backlog and we can re-groom it.