[go: up one dir, main page]
Page MenuHomePhabricator
Create Task
Maniphest T321082

Requests to be-x-old.wikipedia.org result in HTTP 504 Gateway Timeout
Closed, ResolvedPublic
Actions

Description

While working on T304549: Deploy "add a link" to 5th round of wikis, I noticed that queries to be-x-old.wikipedia.org result in HTTP 504 Gateway timeouts.

On a deploy server, try:

curl -vvv "http://localhost:6029/v1/linkrecommendations/wikipedia/be_x_old/Foo"

The practical impact is that attempts to use https://api.wikimedia.org/service/linkrecommendation/apidocs/ with a GET request for be_x_old will fail.

The relevant application code is in MediaWikiApi.py, e.g. https://gerrit.wikimedia.org/r/plugins/gitiles/research/mwaddlink/+/refs/heads/main/src/MediaWikiApi.py#68

Acceptance criteria

Details

SubjectRepoBranchLines +/-
Revert "swagger: Add an example for verifying redirection"research/mwaddlinkmain+0 -8
swagger: Add an example for verifying redirectionresearch/mwaddlinkmain+8 -0
Revert "linkrecommendation: Bump version"operations/deployment-chartsmaster+1 -1
linkrecommendation: Bump versionoperations/deployment-chartsmaster+1 -1
linkrecommendation: Bump versionoperations/deployment-chartsmaster+1 -1
MediaWikiApi: Follow redirects manuallyresearch/mwaddlinkmain+80 -32
app: Follow redirects when we get one from the proxyresearch/mwaddlinkmain+5 -3
Customize query in gerrit

Related Objects

Event Timeline

kostajh created this task.Oct 18 2022, 2:06 PM
Restricted Application added a project: Growth-Team. · View Herald TranscriptOct 18 2022, 2:06 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
kostajh updated the task description. (Show Details)Oct 18 2022, 2:07 PM
kostajh updated the task description. (Show Details)Oct 18 2022, 2:10 PM
Joe subscribed.Oct 18 2022, 2:11 PM

The redirect happens correctly; from the network namespace of the main app:

kubernetes1008:~# nsenter -t 61704 -n curl -H 'Host: be-x-old.wikipedia.org' localhost:6500/ -v 2>&1 | grep location:
< location: https://be-tarask.wikipedia.org/

but then if the application doesn't funnel the request to be-tarask.wikipedia.org back via the proxy...

kubernetes1008:~# nsenter -t 61704 -n curl https://be-tarask.wikipedia.org/
curl: (28) Failed to connect to be-tarask.wikipedia.org port 443: Connection timed out

because we have no egress rule allowing to access the wikis from the public. So I think this is what you're seeing here.

kostajh added a subscriber: Urbanecm.Oct 18 2022, 2:11 PM
Joe added a project: serviceops-radar.Oct 18 2022, 2:12 PM
gerritbot added a comment.Oct 18 2022, 2:14 PM

Change 843954 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[research/mwaddlink@main] app: Follow redirects when we get one from the proxy

https://gerrit.wikimedia.org/r/843954

gerritbot added a project: Patch-For-Review.Oct 18 2022, 2:14 PM
Urbanecm added a subscriber: Urbanecm_WMF.Oct 18 2022, 2:14 PM
Restricted Application removed a subscriber: Urbanecm. · View Herald TranscriptOct 18 2022, 2:14 PM
Joe added a comment.EditedOct 18 2022, 2:27 PM

Some code that would work:

resp = requests.get(url, headers=headers, allow_redirects=False)
while resp.is_redirect:
   url = urlparse(resp.headers['location'])
   headers = {'Host': url.netloc}
   resp= requests.get(url._replace(netloc='localhost:6500', scheme='http').geturl(), headers=headers, allow_redirects=False)
gerritbot added a comment.Oct 18 2022, 2:34 PM

Change 843954 abandoned by Kosta Harlan:

[research/mwaddlink@main] app: Follow redirects when we get one from the proxy

Reason:

going to use a different approach

https://gerrit.wikimedia.org/r/843954

kostajh mentioned this in T320961: LinkRecommendation: Don't convert underscores to dashes.Oct 18 2022, 3:05 PM
Maintenance_bot removed a project: Patch-For-Review.Oct 18 2022, 3:30 PM
kostajh assigned this task to Urbanecm_WMF.Oct 18 2022, 3:37 PM
kostajh edited projects, added Growth-Team (Sprint 0 (Growth Team)); removed Growth-Team.
kostajh moved this task from Incoming to In Progress on the Growth-Team (Sprint 0 (Growth Team)) board.
gerritbot added a comment.Oct 18 2022, 3:45 PM

Change 843968 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[research/mwaddlink@main] MediaWikiApi: Follow redirects manually

https://gerrit.wikimedia.org/r/843968

gerritbot added a project: Patch-For-Review.Oct 18 2022, 3:45 PM
Urbanecm_WMF moved this task from In Progress to Code Review on the Growth-Team (Sprint 0 (Growth Team)) board.Oct 18 2022, 6:02 PM
gerritbot added a comment.Oct 19 2022, 7:51 AM

Change 844430 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[research/mwaddlink@main] swagger: Add an example for verifying redirection

https://gerrit.wikimedia.org/r/844430

gerritbot added a comment.Oct 19 2022, 7:55 AM

Change 843968 merged by jenkins-bot:

[research/mwaddlink@main] MediaWikiApi: Follow redirects manually

https://gerrit.wikimedia.org/r/843968

Urbanecm mentioned this in rRMWA8b645e0cec5f: MediaWikiApi: Follow redirects manually.Oct 19 2022, 7:55 AM
gerritbot added a comment.Oct 19 2022, 8:27 AM

Change 844439 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[operations/deployment-charts@master] linkrecommendation: Bump version

https://gerrit.wikimedia.org/r/844439

gerritbot added a comment.Oct 19 2022, 8:46 AM

Change 844430 merged by jenkins-bot:

[research/mwaddlink@main] swagger: Add an example for verifying redirection

https://gerrit.wikimedia.org/r/844430

kostajh mentioned this in rRMWA19a1445cedd9: swagger: Add an example for verifying redirection.Oct 19 2022, 8:47 AM
gerritbot added a comment.Oct 19 2022, 8:59 AM

Change 844444 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[operations/deployment-charts@master] linkrecommendation: Bump version

https://gerrit.wikimedia.org/r/844444

gerritbot added a comment.Oct 19 2022, 9:12 AM

Change 844439 merged by jenkins-bot:

[operations/deployment-charts@master] linkrecommendation: Bump version

https://gerrit.wikimedia.org/r/844439

kostajh moved this task from Code Review to QA on the Growth-Team (Sprint 0 (Growth Team)) board.Oct 19 2022, 9:23 AM
Urbanecm_WMF updated the task description. (Show Details)Oct 19 2022, 9:26 AM
Urbanecm_WMF updated the task description. (Show Details)
gerritbot added a comment.Oct 19 2022, 4:20 PM

Change 844444 merged by jenkins-bot:

[operations/deployment-charts@master] linkrecommendation: Bump version

https://gerrit.wikimedia.org/r/844444

gerritbot added a comment.Oct 19 2022, 4:22 PM

Change 844038 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[operations/deployment-charts@master] Revert "linkrecommendation: Bump version"

https://gerrit.wikimedia.org/r/844038

gerritbot added a comment.Oct 19 2022, 4:26 PM

Change 844038 merged by jenkins-bot:

[operations/deployment-charts@master] Revert "linkrecommendation: Bump version"

https://gerrit.wikimedia.org/r/844038

Urbanecm_WMF added a comment.Oct 19 2022, 5:01 PM

The swagger patch did not work:

[urbanecm@deploy1002 /srv/deployment-charts/helmfile.d/services/linkrecommendation (master u=)]$ service-checker-swagger staging.svc.eqiad.wmnet https://staging.svc.eqiad.wmnet:4005 -t 2 -s /apispec_1.json
/v1/linkrecommendations/{project}/{domain}/{page_title} (Untitled test) is CRITICAL: Test Untitled test returned the unexpected status 404 (expecting: 200)
[urbanecm@deploy1002 /srv/deployment-charts/helmfile.d/services/linkrecommendation (master u=)]$
gerritbot added a comment.Oct 19 2022, 5:01 PM

Change 844039 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[research/mwaddlink@main] Revert "swagger: Add an example for verifying redirection"

https://gerrit.wikimedia.org/r/844039

gerritbot added a comment.Oct 19 2022, 5:05 PM

Change 844039 merged by jenkins-bot:

[research/mwaddlink@main] Revert "swagger: Add an example for verifying redirection"

https://gerrit.wikimedia.org/r/844039

Urbanecm mentioned this in rRMWA2a0abcd55326: Revert "swagger: Add an example for verifying redirection".Oct 19 2022, 5:06 PM
Urbanecm_WMF triaged this task as Medium priority.Oct 26 2022, 8:49 PM
Etonkovidova moved this task from QA to Test in Production | Watching on the Growth-Team (Sprint 0 (Growth Team)) board.Oct 28 2022, 9:21 PM
Etonkovidova subscribed.

Testing notes: when deployed to production verify, for example, the following:

curl -X GET "https://api.wikimedia.org/service/linkrecommendation/v1/linkrecommendations/wikipedia/be-x-old.wikipedia.org/M1_Abrams?threshold=0.5&max_recommendations=3" -H "accept: application/json"

  • presently the above request returns:
400 Undocumented
Error: Bad Request
Response body

  "message": "Unable to process request for wikipedia/be-x-old.wikipedia.org",
Urbanecm_WMF added a comment.Oct 31 2022, 6:35 AM

Hi @Etonkovidova, perhaps I'm missing something, but I think this URL
address is correct – description mentions
/v1/linkrecommendations/wikipedia/be_x_old/Foo. I think that for your
example, the URL should be
https://api.wikimedia.org/service/linkrecommendation/v1/linkrecommendations/wikipedia/be_x_old/M1_Abrams?threshold=0.5&max_recommendations=3,
which works on my end.

Etonkovidova closed this task as Resolved.Nov 4 2022, 1:25 AM
In T321082#8355656, @Urbanecm_WMF wrote:

Hi @Etonkovidova, perhaps I'm missing something, but I think this URL
address is correct – description mentions
/v1/linkrecommendations/wikipedia/be_x_old/Foo. I think that for your
example, the URL should be
https://api.wikimedia.org/service/linkrecommendation/v1/linkrecommendations/wikipedia/be_x_old/M1_Abrams?threshold=0.5&max_recommendations=3,
which works on my end.

Thx! My mistake, be-x-old should be be_x_old.

All works as expected.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits