Thank you @hashar !

One thing about the user journeys: I think it's important that we can identify which team is responsible for each journey so we don't create the tests and they will not be used.

Ping @Gilbertococchi :)

The restart issue: before my vacation I added a new user to the instance that handles the queue and I think I missed added that user to the Docker group. It's added now lets see if the restart works.

I got this up and running again. The problem was that the keydb queue was in the state "unhealthy". What's strange is that it should be configured to restart when it becomes unhealthy but it didn't work. I'm gonna look into it later today before I close the issue.

Lets do this when the device lab is up and running.

When we have the new setup up and running, lets run tests that ends the test when there's been 5 seconds of no network requests and see if that helps.

Nothing has happened for this Chromedriver issue.

The new setup use the latest ARM build for gnirehtet.

We have pretty solid way for our tests so we can skip this one.

Running FF on Moto G5 is ok for now.

Everything is installed now. Lets wait on getting the actual Raspberries and collect public keys for team members that should have access.

I have a default image ready that we can use for all PIs and then change the hostname for each device. I think that should work. When I have all the phones, we can make a new copy with all phones having access through USB to each device.

Thank you @Edtadros let's discuss it the next time we meet!

Thanks @fgiunchedi ! If I understand correctly the Jenkins workers also runs in Cloud VPS.

Feel free to send me an invite when it works for you, if I don't have any meetings at that time, I usually can join.

@Jdlrobson I haven't tested this at all sorry, so I don't know.

The bug is fixed. The problem/bug was how I merged objects,.

@zeljkofilipin checkout https://wikitech.wikimedia.org/wiki/Performance/Synthetic_testing#Responsibilities and the rest of that page. Lets sync in office hours.

I got this running but there's one bug that needs to be fixed. When I add multiple servers with multiple mobile devices, some configuration is lost. Haven't been able to nail exactly what happens.

Today I got help from @zeljkofilipin to verify that he could access my Raspberry PI using ssh and with VNC. It worked.

All phones are updated running the latest versions of Chrome/Firefox. What's left is to verify all settings and run tests on them.

All phones are rooted, next step is to install what's necesarry.

Running with Moto G5 I don't get any failures at all, this happens only with Samsung A51.

I still get this error:

[2024-08-30 09:45:53] ERROR: WebDriverError: Failed to decode response from marionette

Since we do not use bitbar anymore we can close this one.

This is fixed now.

Hi @hashar @fgiunchedi who knows if this is doable? Can the Jenkins main server reach Prometheus or even better can the workers reach the push gateway? I'm looking for a way to get actionable metrics into Grafana so we better can see/understand flakiness of our tests. We also have T363969 where we are looking for doing the same for the Pixel project running in Cloud VPS.

Hi @taavi are you the right person who knows how Prometheus is setup for Cloud VPS?

I got this to work using the built VNC and then building scrcpy using https://github.com/Genymobile/scrcpy/blob/master/doc/linux.md#latest-version

I think we can skip this one, I got a new more interesting one coming up with Mann Whitney U and bare metal servers.

Oh, it's too long time for the summary. Lets do that the next time something like this comes up.

There's other blog posts coming up so lets skip this one.

Hi WMCS, I need your help/feedback! We run the Pixel service at Cloud VPS. The Pixel project compare screenshots between branches/commits and help developers find regressions.

I think this a good first version that we can iterate over time.

Made a new version here: https://grafana.wikimedia.org/d/c6a56131-dc83-4c9a-8d50-2e687f83ab70/reading-web-performance-peter-h-copy

https://wikitech.wikimedia.org/wiki/Performance/Synthetic_testing/Run_a_test

I started to move on with this today, trying out building a Prometheus exporter for webdriver.io.

@valerio.bozzolan no I don't know. It seems like there was one account that was specific to the performance team, I couldn't see other usage.

Hi @lmata and @fgiunchedi I wanted to check where we are with the sunsetting of Graphite, is the plan early 2025 or late 2025? I'm thinking about the performance data that we still have in Graphite. I wanted to check how much blocker that part is and how it can be fixed so you can do the move?

https://phabricator.wikimedia.org/phame/post/view/315/moving_performance_testing_tools_out_of_aws/

I kind of do not like delivering different code for synthetic tests vs real users, I'm thinking about the risk of missing out of things. When we used speed index in our performance alerts, we created the alerts so that they also looked at user timings, Gilles added a user timing that is created when we show a banner, so we could tune the alerts so that they do not alert when we showed a banner. We could graph that metric to in your graph to make it easier to understand? Talked with Nat about having a session on how we can include more things in our performance graphs long time ago but I got stuck on other things. Let me make sure that happens from my side one a month or so.

I would say its the dark mode banner that made the last visual change faster.

This is better now. When we start to run more tests if we use mobile phones, we should upgrade the disk to a larger disk.

This works by adding --webpagereplay to the test.

I just did one tets after my tuning and it works. We need to add it to the documentation but let me do that in another task.

This has been fixed now and I could verify that data was removed, Lets keep it open for some days though so I also can see that the graphs get flat.

There where some misconfiguration on the server, it happened when I setup new user for the database a couple of weeks ago, I changed it now and waits on the next cleanup run in a couple of hours.

Hi @bwang sorry for leaving you without any documentation. Let me know if you have some time and I can do a walkthrough with you. I'm gonna document how to run tests but it will take a couple of weeks. A couple of things: