In T363587#10357493, @hashar wrote:

I have reverted that patch (https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/1097995) because the change got merged while I was doing the MediaWiki train deployment. I did not want to deploy the config change given I don't know what it is about.

I can repro by simply trying to run spark3-sql in the shell.

In T378342#10267687, @CodeReviewBot wrote:

gmodena opened https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/892

Draft: add a staging airflow dag for webrequest_frontend refinement

For naming of the buckets, should we use the same pattern as mw-page-content-change-enrich?

In T368746#10277534, @xcollazo wrote:

To test, we can override the DagProperty hive_wikitext_raw_table with xcollazo.wikitext_raw_rc2_wiki_partitioned_plus_bf_on_rev_id.

I am not familiar with this part of the stack. Nowadays, can we access spark/ airflow runtime metrics from prometheus? Airflow UI?

flink-app (chart) and eventualities-python (streaming framework) are integrated with the observability platform. We just need to remember to enable alerts for the content_history app before going live.

@BTullis this was closed on the DE board, and IMHO we can resolve it. From what I can tell, emails are being correctly forwarded,

In T374811#10292057, @kostajh wrote:

@gmodena is there an estimated completion date for this task? Thanks!

Discussed with @tchin about adding the Hive consumer into ESC update. We should discuss with the team about creating the list at runtime or adding the 171 block into the PHP file manually.

In T358883#10269971, @xcollazo wrote:

The fact that dependent systems don't have an SLO is not a blocker, but we should model their reliability expectation explicitly.

Could you expand on how we would do that?

I want to start drafting an SLO document this week and would like to validate the direction first.

In T377130#10241164, @gmodena wrote:

Good catch! I missed that one and our tests suite as well... I think we should address it in this CR because it might break things.

In T374359#10254572, @Ottomata wrote:

Are we still hoping to merge include https://gitlab.wikimedia.org/repos/data-engineering/eventutilities-python/-/merge_requests/85#152012d68e4daf2dbab132526851e859f98a98dd in this next major version?

You might want to start finding and engaging with consumers ASAP so they aren't surprised by this. E.g. FRtech uses their own puppet repository and infra, so I have no clue where to look to help change this.

In T377931#10254599, @Ottomata wrote:

For data validation, we can set a cut off date, and expose it via webrequest only from that date onward.

IIUC, webrequest_frontend_text then is a temporary 'test' topic, and when we are production ready the intention is to produce to the existent production e.g. webrequest_text topic?

If so, sounds good fine to me. Let's make sure that is noted in comments in your patches, and linked back to phab.

Down in beam, value will end up in a code path that triggers out_stream.write_int64(value.seconds) (in pyflink/fn_execution/coder_impl_slow.py), and fails with a type error because value.seconds is float, and can't be packed with >q format. If I cast int(value.seconds), bin packing works fine.

After some more digging, I think the issue comes from the return value of flink_instant_of_datetime() (our method) triggered when generating an error event on the DLQ. You can reproduce with:

from pyflink.common import Instant
from datetime import datetime

Now I'm backtracking the root cause of this type mismatch.

Previously (benthos iteration) we decided to use separate topics for Test and Production versions of the log shipper. This made sense when we could wrap topic declaration in an Event Platform stream definition.

In T377773#10250269, @xcollazo wrote:

dumpsdata* and snapshot* are the required hosts for dumpsgen.

@gmodena can you try:

ssh snapshot1014.eqiad.wmnet
sudo -l

And list here the output of sudo -l?

In T376812#10225722, @dcausse wrote:

In T376812#10225610, @gmodena wrote:

@dcausse I'm not sure how you'd like to organize the work, but perhaps we can create a dedicated subtask for the wikimedia-event-utilities version bump?

Agreed, might be clearer to have substasks for this, filed T377130 for this. Do you have some time to dig a bit more into this classpath issue? If not @pfischer or myself could possibly take a closer look this week?

Quick update on this. I build the wrappers locally, using a SNAPSHOT of wikimedia-event-utiliites that bundles Flink 1.20

In T377773#10249354, @hnowlan wrote:

Could you please specify which groups access is needed to? There are a few dumps groups but it appears that @gmodena should inherit access to all of them by virtue of being part of the old platform engineering group. The full list of groups is here.

I agree though that we will not block haproxy migration on this, and can prioritize this separately in a future date.
Please proceed with removing the existing schema and stream configs and we can revisit later. Thank youuu!