Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HTTP redirects are not performed when --check-http #110

Closed
alexlapa opened this issue Nov 23, 2020 · 2 comments · Fixed by #129
Closed

HTTP redirects are not performed when --check-http #110

alexlapa opened this issue Nov 23, 2020 · 2 comments · Fixed by #129
Labels
A-http-check A: Checking http links C-bug Category: This is a bug
Milestone
1.0

Comments

@alexlapa
Copy link

Running cargo deadlinks --check http on links that rely on redirect will result in:

Found invalid urls in fn.asd.html:
        Unexpected HTTP status fetching https://tinyurl.com/rnxcavf: Permanent Redirect

curl'ing provided link will result in:

HTTP/2 301 
date: Mon, 23 Nov 2020 14:21:40 GMT
content-type: text/html; charset=UTF-8
set-cookie: __cfduid=d61fc4a9e687dd28369cfb836203612541606141299; expires=Wed, 23-Dec-20 14:21:39 GMT; path=/; domain=.tinyurl.com; HttpOnly; SameSite=Lax
x-powered-by: PHP/7.3.22
location: https://www.w3.org/TR/mediacapture-streams/#dom-mediadevices-getusermedia
cache-control: max-age=0, public, s-max-age=900, stale-if-error: 86400
referrer-policy: unsafe-url
strict-transport-security: max-age=31536000; includeSubDomains; preload
cf-cache-status: DYNAMIC
cf-request-id: 06971464da00002d7369832000000001
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 5f6b89b48bc22d73-KBP

<!DOCTYPE html>
<html>
    <head>
        <meta charset="UTF-8" />
        <meta http-equiv="refresh" content="0;url='https://www.w3.org/TR/mediacapture-streams/#dom-mediadevices-getusermedia'" />

        <title>Redirecting to https://www.w3.org/TR/mediacapture-streams/#dom-mediadevices-getusermedia</title>
    </head>
    <body>
        Redirecting to <a href="https://www.w3.org/TR/mediacapture-streams/#dom-mediadevices-getusermedia">https://www.w3.org/TR/mediacapture-streams/#dom-mediadevices-getusermedia</a>.
    </body>
</html>

HTTP status code is 301, and response contains location: https://www.w3.org/TR/mediacapture-streams/#dom-mediadevices-getusermedia, that should be enough to perform redirect.

This may be caused by reqwest => ureq migration, since i do not encounter this error on v0.4.2.
@jyn514 jyn514 added A-http-check A: Checking http links C-bug Category: This is a bug labels Nov 23, 2020
@jyn514 jyn514 added this to the 1.0 milestone Nov 23, 2020
@jyn514
Copy link
Contributor

jyn514 commented Nov 23, 2020 

[2020-11-23T23:12:26Z DEBUG ureq::unit] response 308 to HEAD https://tinyurl.com/rnxcavf
Found invalid urls in fn.foo.html:
	Unexpected HTTP status fetching https://tinyurl.com/rnxcavf: Permanent Redirect

This looks like algesten/ureq#187.

This was referenced Nov 23, 2020
Open
Merged
@jyn514
Copy link
Contributor

jyn514 commented Nov 27, 2020

I'm considering using patch.crates-io if algesten/ureq#253 doesn't land in the next few days, it's unclear how long that might take. The use cases they're worried about never appear for deadlinks since it only makes GET requests.

This was referenced Dec 2, 2020
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-http-check A: Checking http links C-bug Category: This is a bug
Projects
None yet
Milestone
1.0
Development

Successfully merging a pull request may close this issue.

Update ureq, handling 307/308 redirects jyn514/cargo-deadlinks
2 participants
@jyn514 @alexlapa