Special:Newsletters doesn't show the newsletters
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Reedy
	Jul 16 2022, 11:06 AM

Description

Screenshot 2022-07-16 at 12.06.34.png (734×1 px, 122 KB)

Details

Subject	Repo	Branch	Lines +/-
IndexPager: Also protect against $offset being 0	mediawiki/core	REL1_40	+2 -2
IndexPager: Also protect against $offset being 0	mediawiki/core	REL1_39	+2 -2
IndexPager: Also protect against $offset being 0	mediawiki/core	REL1_38	+2 -2
IndexPager: Also protect against $offset being 0	mediawiki/core	REL1_35	+2 -2
IndexPager: Also protect against $offset being 0	mediawiki/core	master	+2 -2
Set offset to "" when not required	mediawiki/extensions/Newsletter	master	+4 -3

Customize query in gerrit

Related Objects

Mentioned Here: rENLT25221627f08c: newsletter-create-confirmation: Fix registered typo
rENLT3c2dd29e1344: Namespace extension

Event Timeline

Reedy created this task.Jul 16 2022, 11:06 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 16 2022, 11:06 AM

MariaDB [wikidb]> select * from mw_nl_newsletters;
+-------+---------+-------------------------------------------------+-----------------+-----------+---------------------+
| nl_id | nl_name | nl_desc                                         | nl_main_page_id | nl_active | nl_subscriber_count |
+-------+---------+-------------------------------------------------+-----------------+-----------+---------------------+
|     1 | Foobar  | Newsletter Foobar. More text apparently needed. |            1646 |         1 |                  -1 |
+-------+---------+-------------------------------------------------+-----------------+-----------+---------------------+
1 row in set (0.000 sec)

Hmm. Out of curiosity @Reedy: Was this on the wmf cluster, or on a standalone instance ?

On my local dev wiki.

Could be a PHP 8.1 support issue... I don't think it's fallout from rENLT3c2dd29e1344: Namespace extension as we'd have had bug reports from wmf/1.39.0-wmf.20 being rolled out.

Oh, wait. .20 didn't go out on the train...

https://en.wikipedia.beta.wmflabs.org/wiki/Special:Newsletters shows the same behaviour... Let me see if there's anything in the DB there

Yeah, there's 2. I think I created the first... lol

MariaDB [enwiki]> select * from nl_newsletters;
+-------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-----------+---------------------+
| nl_id | nl_name               | nl_desc                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        | nl_main_page_id | nl_active | nl_subscriber_count |
+-------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-----------+---------------------+
|     1 | Testing McTestFace    | Special:Random 4 lyf. Apparently my description isn't long enough. Is it now?                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |          184311 |         0 |                 -11 |
|     2 | Best newsletter ever! | Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum. |           32585 |         0 |                  -1 |
+-------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-----------+---------------------+
2 rows in set (0.001 sec)

Looks like the query being run on Special:Newsletters is

(SELECT  nl_name,nl_desc,nl_id,nl_subscriber_count,nls_subscriber_id,CONCAT("S",nl_subscriber_count+150000000,"|",nl_name) AS `sort`  FROM `nl_newsletters` JOIN `nl_subscriptions` ON ((nl_id=nls_newsletter_id) AND nls_subscriber_id = 1)   WHERE nl_active = 1 AND (((nl_subscriber_count>='0')))  ORDER BY nl_subscriber_count LIMIT 51  ) UNION ALL (SELECT  nl_name,nl_desc,nl_id,nl_subscriber_count,nls_subscriber_id,CONCAT("U",nl_subscriber_count+150000000,"|",nl_name) AS `sort`  FROM `nl_newsletters` LEFT OUTER JOIN `nl_subscriptions` ON ((nl_id=nls_newsletter_id) AND nls_subscriber_id = 1)   WHERE nl_active = 1 AND nls_subscriber_id IS NULL AND (((nl_subscriber_count>='0')))  ORDER BY nl_subscriber_count LIMIT 51  ) ORDER BY sort ASC LIMIT 51

Bringing it back to rENLT25221627f08c: newsletter-create-confirmation: Fix registered typo doesn't seem to fix it... So doesn't feel like a Regression...

So I just booted up a vagrant machine pointing to fresh "master" branch. Added the newsletter role, with this on top:

429be9a (HEAD -> master, origin/master, origin/HEAD) Localisation updates from https://translatewiki.net.

and mw-core versions saying:

Screenshot 2022-07-17 at 12.36.17.png (344×1 px, 58 KB)

And I could create and list a Newsletter. Check this:

Screenshot 2022-07-17 at 12.36.41.png (854×2 px, 161 KB)

Should I try with some other branch, @Reedy to reproduce ?

Btw, that complex query was authored with Brian during a hackathon to get us the subscribers count (sorted) in one go.

In T313157#8082290, @Reedy wrote:

https://en.wikipedia.beta.wmflabs.org/wiki/Special:Newsletters shows the same behaviour... Let me see if there's anything in the DB there

I’m also running git HEAD of master…

I should try creating a new newsletter I guess

Screenshot 2022-07-17 at 18.54.06.png (450×826 px, 59 KB)

Screenshot 2022-07-17 at 18.54.14.png (470×1 px, 53 KB)

Visiting the Newsletter:Testing_Letter pages (and same for the older ones) works fine

What seems to have happened here is that a newsletter page was created but the newsletter database thinks it's deleted. I have no idea how that happened, and I can't investigate why at this point.

Can this still be reproduced with newly-created newsletters? I've submitted a bunch of patches to improve the resilience in this area.

Pppery merged a task: T310336: Call to a member function canManage() on null.Jun 24 2023, 8:22 PM

Pppery added a subscriber: dom_walden.

I could just reproduce this locally on my Mediawiki Docker + Echo + Newsletter setup.

Change 932676 had a related patch set uploaded (by 01tonythomas; author: 01tonythomas):

[mediawiki/extensions/Newsletter@master] Set offest to "" when not required

https://gerrit.wikimedia.org/r/932676

gerritbot added a project: Patch-For-Review.Jun 25 2023, 8:50 PM

Before:

(SELECT  nl_name,nl_desc,nl_id,nl_subscriber_count,nls_subscriber_id,CONCAT('S',nl_subscriber_count+150000000,'|',nl_name) AS `sort`  FROM `mw_nl_newsletters` JOIN `mw_nl_subscriptions` ON ((nl_id=nls_newsletter_id) AND nls_subscriber_id = 0)   WHERE nl_active = 1 AND (nl_subscriber_count >= '0')  ORDER BY nl_subscriber_count LIMIT 51  ) UNION ALL 
(SELECT  nl_name,nl_desc,nl_id,nl_subscriber_count,nls_subscriber_id,CONCAT('U',nl_subscriber_count+150000000,'|',nl_name) AS `sort`  FROM `mw_nl_newsletters` LEFT OUTER JOIN `mw_nl_subscriptions` ON ((nl_id=nls_newsletter_id) AND nls_subscriber_id = 0)   WHERE nl_active = 1 AND nls_subscriber_id IS NULL AND (nl_subscriber_count >= '0')  ORDER BY nl_subscriber_count LIMIT 51  ) ORDER BY sort ASC LIMIT 51

After:

(SELECT  nl_name,nl_desc,nl_id,nl_subscriber_count,nls_subscriber_id,CONCAT('S',nl_subscriber_count+150000000,'|',nl_name) AS `sort`  FROM `mw_nl_newsletters` JOIN `mw_nl_subscriptions` ON ((nl_id=nls_newsletter_id) AND nls_subscriber_id = 0)   WHERE nl_active = 1  ORDER BY nl_subscriber_count LIMIT 51  ) UNION ALL
(SELECT  nl_name,nl_desc,nl_id,nl_subscriber_count,nls_subscriber_id,CONCAT('U',nl_subscriber_count+150000000,'|',nl_name) AS `sort`  FROM `mw_nl_newsletters` LEFT OUTER JOIN `mw_nl_subscriptions` ON ((nl_id=nls_newsletter_id) AND nls_subscriber_id = 0)   WHERE nl_active = 1 AND nls_subscriber_id IS NULL  ORDER BY nl_subscriber_count LIMIT 51  ) ORDER BY sort ASC LIMIT 51

Change 934531 had a related patch set uploaded (by Majavah; author: Majavah):

[mediawiki/core@master] IndexPager: Also protect against $offset being 0

https://gerrit.wikimedia.org/r/934531

With my patch:

(SELECT nl_name,nl_desc,nl_id,nl_subscriber_count,nls_subscriber_id,CONCAT('S',nl_subscriber_count+150000000,'|',nl_name) AS `sort` FROM `nl_newsletters` JOIN `nl_subscriptions` ON ((nl_id=nls_newsletter_id) AND nls_subscriber_id = 0) WHERE nl_active = 1 ORDER BY nl_subscriber_count LIMIT 51 ) UNION ALL (SELECT nl_name,nl_desc,nl_id,nl_subscriber_count,nls_subscriber_id,CONCAT('U',nl_subscriber_count+150000000,'|',nl_name) AS `sort` FROM `nl_newsletters` LEFT OUTER JOIN `nl_subscriptions` ON ((nl_id=nls_newsletter_id) AND nls_subscriber_id = 0) WHERE nl_active = 1 AND nls_subscriber_id IS NULL ORDER BY nl_subscriber_count LIMIT 51 ) ORDER BY sort ASC LIMIT 51