Issue 38662: Decouple ensurepip from pip's internals using runpy

Inspired by https://github.com/python/cpython/pull/16782#discussion_r335285656

This changes ensurepip's current assumption that pip would expose a "main" function, which can be invoked to run pip.

I've selected all actively supported Python versions for this ticket, since this patch should be backported to all Python versions, to make it robust to internal changes within pip.

Note that the patch uses the fact that:


```
def foo():
    try:
        raise Exception("returned")
    except Exception as e:
        print("except")
        return e.args[0]
    finally:
        print("finally")


print(foo())
```

will print:

```
except
finally
returned
```

Can you explain how this would be rolled out?  Does this mean as of a planned release of pip the current ensurepip would break?  If so, when?  Is 2.7 still supported with that version of pip and, if so, have you tested the PR with it?

On Tue, 5 Nov 2019 at 4:30 PM, Ned Deily <report@bugs.python.org> wrote:

>
> Ned Deily <nad@python.org> added the comment:
>
> Can you explain how this would be rolled out?

This, as in? If you're referring to this specific change, I don't think
there's anything special needed for this change (it's backwards compatible
with past pip releases, but that doesn't matter).

ensurepip only installs the bundled wheel it already has, into the
environment. Thus, existing ensurepip releases won't be broken.

However, when we bump up to pip 19.3 [1], we'd need to be updating the
invocation since it's using a pip internal and pip's internals can change
without notice even in a patch version.

This change ensures that ensurepip is robust to changes in the location of
pip's main function, thus making those updates as simple as a bundled wheel
change.

[1]: IIUC we try to keep CPython's bundled wheel in sync with latest)

Does this mean as of a planned release of pip the current ensurepip would
> break?

Yes. pip 19.3 made a change and as can be seen in the PR for that (sorry,
it's a GitHub PR and my internet is ridiculously slow right now).

  If so, when?

That's already been released. 🙃

Is 2.7 still supported with that version of pip and, if so, have you tested
> the PR with it?

Yes, pip will support 2.7 at least until CPython does (and likely longer).

>
> ----------
> nosy: +ned.deily
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue38662>
> _______________________________________
>

New changeset 88f82b2b9ea3514359cb6e3218121f75334063ac by Miro Hrončok in branch 'master':
bpo-38662: ensurepip invokes pip via runpy (GH-18901)
https://github.com/python/cpython/commit/88f82b2b9ea3514359cb6e3218121f75334063ac

While it's not a major issue in CPython, I can say that being tightly coupled to pip internals is a major in Fedora which updates pip frequently and tries to support multiple pip versions.

Miro Hrončok: "I wasn't even hoping for a backport. Thanks!" ( https://github.com/python/cpython/pull/18901#issuecomment-597324810 )

pip wheel files are still updated in stable 3.7 and 3.8 branches. Example in August 2018 in 3.7:

* commit c1c1a3396391dcc948d332607a0f673e4434da97
* bpo-37664

> Yes, pip will support 2.7 at least until CPython does (and likely longer).

2.7 is no longer supported in CPython. So I will not backport the fix to 2.7. And I removed the 2.7 from versions of this issue.

New changeset 8d5c958ddebbb96c2de5d58da4c7aa1790629cf9 by Miss Islington (bot) in branch '3.7':
bpo-38662: ensurepip invokes pip via runpy (GH-18901)
https://github.com/python/cpython/commit/8d5c958ddebbb96c2de5d58da4c7aa1790629cf9

New changeset 212acf544a41a983aea7166d43d8dfe9a8296655 by Miss Islington (bot) in branch '3.8':
bpo-38662: ensurepip invokes pip via runpy (GH-18901)
https://github.com/python/cpython/commit/212acf544a41a983aea7166d43d8dfe9a8296655

Thanks Pradyun Gedam and Miro Hrončok for the fix.

stage: patch review -> resolved