Issue 14132: Redirect is not working correctly in urllib2

When only the query string is sent by the server as the redirect url, urllib2 redirects to incorrect address.

Error is occuring on the page http://kniznica.uniza.sk/opac. Server sends only the query string part of the uri in the Location header (ie. ?fs=04D07295D4434730A51C95A9F1727373&fn=main). Path is then incorrectly stripped from the original url, and urllib2 redirects to http://kniznica.uniza.sk/?fs=04D07295D4434730A51C95A9F1727373&fn=main.

The error was introduced in the fix of the issue #2464. I think, the attached patch is fixing the error (it is working for me).

I forgot to mention that the correct url in the example would be http://kniznica.uniza.sk/opac?fs=04D07295D4434730A51C95A9F1727373&fn=main.

→  curl -sI  http://kniznica.uniza.sk/opac

HTTP/1.1 302 Moved Temporarily
Date: Wed, 06 Mar 2013 03:23:06 GMT
Server: Indy/9.0.50
Content-Type: text/html
Location: ?fs=C79F09C9F1304E7AA4FF7C211BEA2B9B&fn=main


→ python3.3

Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 01:25:11) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib.parse
>>> urllib.parse.urlparse("http://kniznica.uniza.sk/opac")
ParseResult(scheme='http', netloc='kniznica.uniza.sk', path='/opac', params='', query='', fragment='')
>>> urllib.parse.urlparse("?fs=C79F09C9F1304E7AA4FF7C211BEA2B9B&fn=main")
ParseResult(scheme='', netloc='', path='', params='', query='fs=C79F09C9F1304E7AA4FF7C211BEA2B9B&fn=main', fragment='')

Redirection is defined at
http://hg.python.org/cpython/file/5e294202f93e/Lib/urllib/request.py#l643

The proposed patch looks good to me. A test case would be nice though.

Also I wonder why the “malformed URL” logic needs to be in urllib.request. Surely it either belongs in urljoin(), or in the underlying http.client. That needs more thought, but either way the current patch is a definite improvement.

urllib2_redirect_fix.2.patch adds a test.

I was tempted to remove the whole block of code setting the path to “/”, but there is one minor disadvantage: if a redirect points to a so-called “malformed” URL without any path component, like “http://example.net” or “http://example.net?query”, geturl() would return this URL verbatim.

I will try to commit this soon

New changeset 52a7f580580c by Martin Panter in branch '3.5':
Issue #14132: Fix redirect handling when target is just a query string
https://hg.python.org/cpython/rev/52a7f580580c

New changeset 789a3f87bde1 by Martin Panter in branch '2.7':
Issue #14132: Fix redirect handling when target is just a query string
https://hg.python.org/cpython/rev/789a3f87bde1

New changeset 841a9a3f3cf6 by Martin Panter in branch 'default':
Issue #14132, Issue #17214: Merge two redirect handling fixes from 3.5
https://hg.python.org/cpython/rev/841a9a3f3cf6

title: SEO Services Development & PHP development -> Redirect is not working correctly in urllib2

messages: + msg359255

components: + XML, - Library (Lib)
type: behavior -> performance

stage: test needed -> patch review
messages: + msg244080
versions: + Python 3.6

messages: + msg243875

stage: test needed