[go: up one dir, main page]

Page MenuHomePhabricator

Upgrade Thumbor to Buster
Closed, ResolvedPublic

Description

Upgade Thumbor from Debian stretch to Debian buster, according to Operating system upgrade policy and guidance from serviceops.

Things to consider:

Repackage/rebuild for buster We will be using Blubber and pip requirements to deploy Thumbor

To be resolved

  • Figure out and decide what version of librsvg we require for Thumbor on Debian Buster. There are complex caveats both with too new and too old versions. Ref T265549 and T40010 for (long) backstory here.

Optional/Related tasks:

  • T212941 Investigate systemd hardening to replace Firejail for Thumbor
  • T212946 Stream Thumbor logs to logstash
  • T187765 Replace the Nginx fronting Thumbor with a reverse proxy capable of queuing requests

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Is there still work that needs to be done to prepare for this, or is it just waiting for someone to get around to handling the necessary configuration changes, testing, and deployment?

It's waiting for someone to do it. @jijiki when she gets back from leave, possibly?

According to https://www.mediawiki.org/w/index.php?title=Topic:Wbuk08w1anifjyak#flow-post-wbunuhciop2u1yl8 and https://wikitech.wikimedia.org/wiki/Operating_system_upgrade_policy#Policy_proposal Thumbor is currently on Debian Stretch (currently oldstable)
I recommend to rename the description from

After successfully upgrading Thumbor to stretch, we can proceed with the upgrade to Debian buster.

to

Upgrade Thumbor from Debian stretch to Debian buster, accoding to Operating system upgrade policy#Policy proposal.

Stretch was supposed to be phased out by June 2021 per https://wikitech.wikimedia.org/wiki/Operating_system_upgrade_policy, and will be EOL in less than 6 months (June 30, 2022) per https://wiki.debian.org/LTS. Is any work being done on this?

Is any work being done on this?

At the moment, no. Thumbor currently has no maintainer, see T294484.

WDoranWMF renamed this task from Upgrade Thumbor to Buster to Upgrade Thumbor to Bullseye.Mar 15 2022, 6:12 PM
WDoranWMF updated the task description. (Show Details)

Note that unlike most other packages, there is an especially sensitive dependency on the behaviour of librsvg which is the component of Thumbor responsible for converting SVGs to PNGs.

I don't have a succint summary, but there's a lot of backstory worth reading up on in these three tasks:

Update: After looking into it, it looks like backporting ≥2.42.3 in Debian stretch brings a long list of dependencies. This task is blocked until we upgrade Thumbor to Buster

Backporting software because it is not available for the current OS version is one thing. Intentionally running old software when new, stable version are available in the repos is another, and it goes against good practice.

We'll get a new version of librsvg sooner rather than later, whenever T216815: Upgrade Thumbor to Buster gets done (probably before mid-2021).

[…]

[…] it is no longer just a matter of installing the latest librsvg. The latest version will choke if hyphenated langtags (e.g., sr-latn, zh-hant, ku-arab) are passed to librsvg via the $LANG environment variable.

I do not know PHP or Python, but here are the changes needed to wiki configuration, SVGHandler.php, and Thumbor's svg.py.

They are breaking changes. They need the Rust version of librsvg/rsvg-convert. If used with the old version of librsvg, I expect the new command line arg would cause an exception.

MediaWiki could be made compatible by having rsvg and rsvglang entries and testing for a "$lang" substring in the conversion string.

Thumbor is hardwired, so making it compatible with both versions would be more complicated. However, it would be good to allow Thumbor to use both rsvg-convert or resvg.

Huh, is anyone tasked with this? This is one of the few cases that's keeping Stretch alive in cloud-vps and prod.

Huh, is anyone tasked with this? This is one of the few cases that's keeping Stretch alive in cloud-vps and prod.

See Thumbor Migration. Work is continuing in T233196, last I heard there was a mostly-working version in staging.

Huh, is anyone tasked with this? This is one of the few cases that's keeping Stretch alive in cloud-vps and prod.

@Andrew I'm the new product manager working on Thumbor Migration. This seems critical, but I don't have enough context (Thumbor has been full of surprises). From this December news, it looks like Stretch was deprecated from cloud-vps. Please point me to docs/tasks/tools that describe how Stretch and Thumbor work together. Thank you, in advance!

@hnowlan for awareness.

@VirginiaPoundstone, as far as I'm aware, it's not so much that Thumbor is dependent on features of Debian 9 (Stretch) and is incompatible with Debian 10 (Buster) or 11 (Bullseye), it's that currently running on Stretch requires us to use obsolete and buggy versions of other libraries such as librsvg. lt's mostly a matter of setting up and configuring Thumbor on a Bullseye instance, although there are some modifications that would be needed to support the latest versions of librsvg, as documented at T265549.

@Ahecht: If supporting the latest version of librsvg is difficult, some Users prefer (i) resvg (faster and less regressions than updating librsvg) or (ii) Inkscape (slower but better know by the community), as documented at T40010.

Inkscape offers AppImages, which run portable on any Debian-Version without any dependencies, so you could run multiple Inkscape-version in parallel, so up- and downgrading Debian and/or Inkscape would be much easier, that it is currently.

resvg only exists in bullseye https://packages.debian.org/search?suite=all&searchon=names&keywords=resvg and even long-term support has ended for stretch in summer of 2022, so it's just EOL. Afaict keeping this on stretch is not really an option.

hnowlan renamed this task from Upgrade Thumbor to Bullseye to Upgrade Thumbor to Buster.Jan 20 2023, 5:08 PM
hnowlan updated the task description. (Show Details)
hnowlan updated the task description. (Show Details)

Scap updates on Thumbor hosts are currently failing since Scap requires Python 3.7. I would like to disable any Scap deployments until this ticket is completed (or alternatively Python3 is updated on the hosts)

So far the only Scap deployment I've found that seems related to Thumbor is this: https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/refs/heads/production/modules/threedtopng/manifests/deploy.pp#14

Can you confirm that's the right config or whether there are other deployments I've missed? Or point me to someone who would know?

Hi @hnowlan and @MoritzMuehlenhoff I've tentatively created a patch to try to address the problem with Scap updates and added you as reviewers: https://gerrit.wikimedia.org/r/c/operations/puppet/+/894542

Apologies if you were the wrong persons to assign to this, I'm still trying to find someone who knows about the thumbor deployments.

@jnuche, once we figure out T328033 and thus are able to complete T233196, thumbor will have nothing to do with scap. It's probably not worth solving this one only to ditch that work a few weeks down the line.

@akosiaris thanks for the feedback.

Just to clarify, we can work around the issue currently, but it makes the frequent Scap self-update process more error-prone. If the patch I created (or a small variation of it) fixes the issue, I think it's worth merging it.

@akosiaris thanks for the feedback.

Just to clarify, we can work around the issue currently, but it makes the frequent Scap self-update process more error-prone. If the patch I created (or a small variation of it) fixes the issue, I think it's worth merging it.

Patch was merged, but putting it in writing too, disabling scap for thumbor and 3dtopng is fine.

hnowlan claimed this task.