Issue 7159: Urllib2 authentication memory.

For each request requiring HTTP authentication, urllib2 submits the
request without authentication, receives the server's 401
error/challenge, then re-submits the request with authentication.

This is compliant behavior. The problem comes in that urllib2 repeats
this for every ensuing request to the same namespace.

At times this is just an inefficiency--every request gets sent twice,
often with POST data (which can be sizeable).

Sometimes, especially with large POST bodies, this causes a connection
failure.

(Mercurial suffers greatly from this (and I have suggested workaround to
that team.)

This isn't non-compliant behavior, but RFC2617 (sections 2, 3.3)
suggests that once an HTTP client authenticates, it pre-emptively send
authentication with ensuing requests.

More analysis and fix suggestions at
bitbucket.org/bradobro/liquidhg/wiki/Home.

@Senthil what is your opinion of this?

See also issue 19494.  I think this would be a new feature, and it may be that it should leverge the feature added in issue 19494.  The difference here is that we are proposing to allow it to happen automatically after the initial 401, whereas in 19494 we send it pre-emptively even on the first call.

Adding a patch for Python 3+

Some notes:

* Adding a new password manager to handle this case
* The new handler added in issue 19494 had couple of issues
  * test passes even if we use the old handler in added test
  * uses `request.host` instead of `request.full_url` as key for password

* The new handler did assume realm = None before and still does.

I'm using the same logic for adding keys for adding authenticated urls/realm as for login credentials in basic auth handler.

Added some review comments.

I think the urllib documentation does not really explain how to *use* these classes, and it should, but that is a separate issue.

Updated patch with review changes

Here is the doc patch for the design that Akshit and I agreed to after re-consideration of the API.  This eliminates the HTTPBasicPriorAuthHandler in favor of the HTTPPasswdMgrWithPriorAuth password manager and putting support for prior auth on the AbstractBasicAuthHandler based on whether or not the password_mgr supports the is_authenticated and updated_authenticated methods.  This redesign means that the Proxy handler automatically gets support for prior auth.

Actually attaching the patch this time.

Updated code as per docs

New changeset 1b9e81cb83bc by R David Murray in branch 'default':
#7159: generalize urllib prior auth support.
https://hg.python.org/cpython/rev/1b9e81cb83bc

Should this be closed?  A substantial patch was pushed year ago and I see no indication of further issue.

Yes, Go ahead. Thanks.

On Tue, May 24, 2016 at 1:56 AM, Terry J. Reedy <report@bugs.python.org>
wrote:

>
> Terry J. Reedy added the comment:
>
> Should this be closed?  A substantial patch was pushed year ago and I see
> no indication of further issue.
>
> ----------
> nosy: +terry.reedy
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue7159>
> _______________________________________
>

messages: + msg241260

messages: + msg241259

messages: + msg241182

type: resource usage -> enhancement
stage: needs patch

messages: + msg236620

type: behavior -> resource usage

nosy: + orsenthil