Issue 38722: runpy should use io.open_code() instead of open()

Fairly obviously, if you're using something called runpy you're probably trying to run some code. To do this it has to open the script as a file.

This is similar to two other issues I'm posting, but they're in different modules, so different bugs.

I'll plan on tackling this one.  I already did pdb.

I made the change but the test suite is giving me fits and I don't know why.  

Running: ./python -m test
...
== Tests result: FAILURE ==

392 tests OK.

1 test failed:
    test_tools

26 tests skipped:
    test_curses test_dbm_gnu test_dbm_ndbm test_devpoll test_idle
    test_kqueue test_msilib test_ossaudiodev test_readline
    test_smtpnet test_socketserver test_startfile test_tcl
    test_timeout test_tix test_tk test_ttk_guionly test_ttk_textonly
    test_turtle test_urllib2net test_urllibnet test_winconsoleio
    test_winreg test_winsound test_xmlrpc_net test_zipfile64

Total duration: 17 min 38 sec
Tests result: FAILURE


But running: ./python -m test -v test_tools
...
Ran 223 tests in 2.503s

OK (skipped=2, expected failures=14)

== Tests result: SUCCESS ==

1 test OK.

Total duration: 2.6 sec
Tests result: SUCCESS


Any tips for a newbe?

Tests working now.  PR submitted.

I don't see why this should be considered a security issue.

This should likely have been done when io.open_code() was initially added, but now that 3.8 is out, I don't think backporting this would be wise.

It's a security issue because Python 3.8 says it will open files to be executed with io.open_code() instead of open(). This allows a way to bypass that.

That said, this appears to be a fallback case, so I'm not hugely concerned. I haven't quite figured out why it would fall back here (that involved reading the pkgutil sources ;) ).

I would vote for backporting to 3.8.1, but if Tal wants to push back and nobody else has an opinion then whatever.

New changeset e243bae9999418859106328d9fce71815b7eb2fe by Miss Islington (bot) (jsnklln) in branch 'master':
bpo-38722: Runpy use io.open_code() (GH-17234)
https://github.com/python/cpython/commit/e243bae9999418859106328d9fce71815b7eb2fe

Thanks Steve! I hadn't realized that we'd made such a declaration WRT opening of code files in general. In that case, this is certainly at least a bug fix, and should be backported.

Thanks for reporting this, Dominic!

Thanks for the PR, Jason!

> I hadn't realized that we'd made such a declaration WRT opening of code files in general.

It wasn't exactly a hugely publicised declaration :)

The relevant quote from PEP 578 is:

> All import and execution functionality involving code from a file will be changed to use open_code() unconditionally.

Which I admit is a big claim, and one that was not completely followed through with before 3.8.0. Calling it a "security" fix is borderline, as it isn't really a vulnerability by default, but calling it incorrect behaviour (i.e. a regular bug) is fine by me.

New changeset e37767bee1f7f1940b30768d21bfe2ae68c20a5f by Miss Islington (bot) in branch '3.8':
bpo-38722: Runpy use io.open_code() (GH-17234)
https://github.com/python/cpython/commit/e37767bee1f7f1940b30768d21bfe2ae68c20a5f

Thanks for the clarification Steve! I've backported this to 3.8.

messages: + msg356896

type: security -> enhancement