Thumbor was very straightforward, it only required a tiny patch to relax a python version check in setup.py (bringing it in line with thumbor master). My build is available on buster.thumbor.eqiad.wmflabs, in my home directory. It installs and runs fine.

I couldn't find a repo for the python-thumbor-community-core package Debian sources. I assume that the build we have is based on what I put together here originally: https://github.com/gi11es/thumbor-debian/tree/master/python-thumbor-community-core

Likewise, I fixed it up (required backporting an upstream bugfix) and built it on buster.thumbor.eqiad.wmflabs, in my home directory.

Same story got python-manhole, took it from https://github.com/gi11es/thumbor-debian/tree/master/python-manhole It build as-is. To be found with the other ones on buster.thumbor.eqiad.wmflabs, in my home directory.

Change 505727 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/software/thumbor-plugins@master] Buster compatibility

https://gerrit.wikimedia.org/r/505727

As reference:

root@install1002:/srv/wikimedia# reprepro lsbycomponent thumbor
thumbor |        6.3.2+git20170607-1 |  jessie-wikimedia | backports | amd64, source
thumbor | 6.3.2+git20170607-1+deb9u1 | stretch-wikimedia |      main | amd64, source

root@install1002:/srv/wikimedia# reprepro lsbycomponent python-thumbor-community-core
python-thumbor-community-core |    0.4.0-0wmf1 |  jessie-wikimedia | backports | amd64, i386, source
python-thumbor-community-core | 0.4.0-1+deb9u1 | stretch-wikimedia |      main | amd64, i386, source

root@install1002:/srv/wikimedia# reprepro lsbycomponent python-manhole
python-manhole | 1.3.0-1 |  jessie-wikimedia | backports | amd64, i386
python-manhole | 1.3.0-1 | stretch-wikimedia |      main | amd64, i386

@elukey are the sources for these debian packages in version control somewhere? I ended up packaging the current sid version of Thumbor, which is newer (6.5.1).

In T221562#5130813, @Gilles wrote:

@elukey are the sources for these debian packages in version control somewhere? I ended up packaging the current sid version of Thumbor, which is newer (6.5.1).

For simple rebuilds (as for the Thumbor packages) we simply pull the existing Debian source via "apt-get source thumbor=$VERSION" and rebuild it. boron is restricted, but AFAIK we had a role(builder) host in labs in the past.

That makes sense. I'll try that for the thumbor package, I think it's best to decouple the thumbor version upgrade from the Buster upgrade.

Me or @jijiki can pick the deb sources (with patches if needed) and rebuild them for buster-wikimedia on boron, and then upload them. Basically re-doing what has been done in labs in a "prod" environment, that's it :)

I am happy to do it, but: I am afraid it will have to wait for next week, as this is a short week for me:)

I need to find a working solution on labs first. Currently either I use thumbor-6.3.2+git20170607 and it has a bunch of test failures (package won't build). I haven't looked into a potential source for those, but there seems to be a lot. thumbor-6.5.1, on the other hand, builds fine. But I have a very mysterious hanging test in python-thumbor-wikimedia.

I need to find the root cause for one or the other, basically.

I'm started to get an idea of what's happening with 6.5.1 getting stuck on some of our thumbor plugin tests.

One of our tests, test_multi_max_paths, seems to leave a lock behind in BaseHandler.url_locks (via acquire_url_lock/release_url_lock).

I see that this is actually some upstream code I wrote a long time ago: https://github.com/thumbor/thumbor/commit/b8418fbb9e68644880895b66ac46eb7265622f0d

The multi handler, that this test leaving the lock behind covers, is a feature that we didn't end up using. It's something I wrote very early on in the migration to Thumbor, it was an optimisation for prerendering. We could delete that code entirely, TBH. Which means we can safely skip the multi_handler test (as a result, the other test doesn't get stuck anymore).

I'll still check if I can find the root cause of the issue and fix it if it's something obvious.

@Gilles ping me when you think we are ready to build packages for buster, we would do it soon either way

Change 505837 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/puppet@production] Remove unused "multi" thumbor handler

https://gerrit.wikimedia.org/r/505837

Change 505727 merged by Gilles:
[operations/software/thumbor-plugins@master] Buster compatibility

https://gerrit.wikimedia.org/r/505727

Change 506115 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/software/thumbor-plugins@master] Version bump

https://gerrit.wikimedia.org/r/506115

Change 506115 merged by Gilles:
[operations/software/thumbor-plugins@master] Version bump

https://gerrit.wikimedia.org/r/506115

Change 506116 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 2.5

https://gerrit.wikimedia.org/r/506116

@jijiki this is all good to go, successfully built on buster.thumbor.eqiad.wmflabs python-thumbor-community-core and thumbor need tiny patches (you'll find them in there). And python-thumbor-wikimedia needs the upgrade/patch above. Manhole build as-is.

In the end I had to upgrade Thumbor to the latest currently on Debian unstable (6.5.1) because the version we're currently running seemed to be having a lot of issues on Buster out of the box. It didn't seem worth the time to figure those out when upgrading made the issues go away.

Change 506116 merged by Effie Mouzeli:
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 2.5

https://gerrit.wikimedia.org/r/506116

Change 505837 merged by Effie Mouzeli:
[operations/puppet@production] Remove unused "multi" thumbor handler

https://gerrit.wikimedia.org/r/505837