Issue 41520: codeop: 3.8.5 regression, warnings.simplefilter('error', SyntaxWarning) does not raise.

assuming 

    $ cat foo.py
    import warnings
    from codeop import compile_command

    warnings.simplefilter('error', SyntaxWarning)
    res = compile_command('1 is 1\n', symbol='exec')
    print('Res', res)

On 3.8.0...3.8.4 this correctly raises a SyntaxError:

 python  foo.py
Traceback (most recent call last):
  File "foo.py", line 5, in <module>
    res = compile_command('1 is 1\n', symbol='exec')
  File "/Users/bussonniermatthias/miniconda3/envs/38/lib/python3.8/codeop.py", line 122, in compile_command
    return _maybe_compile(_compile, source, filename, symbol)
  File "/Users/bussonniermatthias/miniconda3/envs/38/lib/python3.8/codeop.py", line 99, in _maybe_compile
    raise err1
  File "/Users/bussonniermatthias/miniconda3/envs/38/lib/python3.8/codeop.py", line 87, in _maybe_compile
    code1 = compiler(source + "\n", filename, symbol)
  File "/Users/bussonniermatthias/miniconda3/envs/38/lib/python3.8/codeop.py", line 102, in _compile
    return compile(source, filename, symbol, PyCF_DONT_IMPLY_DEDENT)
  File "<input>", line 1
SyntaxError: "is" with a literal. Did you mean "=="?



But will silently return None on 3.8.5

$ python  foo.py
Res None

seem to affect 3.8.4 as well.

Potentially due to 

https://bugs.python.org/issue40807
https://github.com/python/cpython/pull/20486

bisecting agrees with Matthias:

# first bad commit: [c067183605cf84bb1a246635f52827251d0476f8] bpo-40807: Show warnings once from codeop._maybe_compile (GH-20486)

I proposed a fix: PR 21838. Would you mind to review it?

New changeset 369a1cbdee14d9f27356fb3a8bb21e4fde289d25 by Victor Stinner in branch 'master':
bpo-41520: codeop no longer ignores SyntaxWarning (GH-21838)
https://github.com/python/cpython/commit/369a1cbdee14d9f27356fb3a8bb21e4fde289d25

New changeset afff51fc09993dff1693aacb440221314b163409 by Miss Islington (bot) in branch '3.8':
bpo-41520: codeop no longer ignores SyntaxWarning (GH-21838)
https://github.com/python/cpython/commit/afff51fc09993dff1693aacb440221314b163409

New changeset 90eff4ed4445a0fa9d8cdf74c0f86c50ed510dad by Miss Islington (bot) in branch '3.9':
bpo-41520: codeop no longer ignores SyntaxWarning (GH-21838)
https://github.com/python/cpython/commit/90eff4ed4445a0fa9d8cdf74c0f86c50ed510dad

Thanks Matthias Bussonnier for the fix. The regression should now be fixed. Also, I converted your reproducer into a regression test.

> Thanks Matthias Bussonnier for the fix. 

Thank *YOU* for the fix, and the bug report is initially from tcaswell. 

At least on 3.8 branch this is fixed for me.

Just for completeness, this was discovered as in IPython we try to guess whether "enter"  is "insert new line" or "execute", and `1 is 1<enter>` would keep adding new lines.

Much love for the fast turnaround; looking fwd to 3.9

Thanks tcaswell in this case ;-)

This 'fix' introduces another regression by undoing the previous fix in issue 40807 of emitting DeprecationWarning (for '\e', for instance) just once instead of thrice.

I suspect that the example of Matthias would fail in a debug build (with DeprecationWarnings on) for DeprecationWarning also.

PR-21848 removes the limitation of the first fix to SyntaxWarning and includes DeprecationWarning in the test for single emission.

The commit 369a1cbdee14d9f27356fb3a8bb21e4fde289d25 introduces a change in environment. I haven't checked the other PR though to see if this is fixed.

./python.exe -m test --fail-env-changed test_codeop -m test_warning
0:00:00 load avg: 2.30 Run tests sequentially
0:00:00 load avg: 2.30 [1/1] test_codeop
Warning -- warnings.filters was modified by test_codeop
  Before: (4331716096, [('default', None, <class 'DeprecationWarning'>, '__main__', 0), ('ignore', None, <class 'DeprecationWarning'>, None, 0), ('ignore', None, <class 'PendingDeprecationWarning'>, None, 0), ('ignore', None, <class 'ImportWarning'>, None, 0), ('ignore', None, <class 'ResourceWarning'>, None, 0)], [('default', None, <class 'DeprecationWarning'>, '__main__', 0), ('ignore', None, <class 'DeprecationWarning'>, None, 0), ('ignore', None, <class 'PendingDeprecationWarning'>, None, 0), ('ignore', None, <class 'ImportWarning'>, None, 0), ('ignore', None, <class 'ResourceWarning'>, None, 0)])
  After:  (4331716096, [('default', None, <class 'DeprecationWarning'>, '__main__', 0), ('ignore', None, <class 'DeprecationWarning'>, None, 0), ('ignore', None, <class 'PendingDeprecationWarning'>, None, 0), ('ignore', None, <class 'ImportWarning'>, None, 0), ('ignore', None, <class 'ResourceWarning'>, None, 0)], [('error', None, <class 'SyntaxWarning'>, None, 0), ('default', None, <class 'DeprecationWarning'>, '__main__', 0), ('ignore', None, <class 'DeprecationWarning'>, None, 0), ('ignore', None, <class 'PendingDeprecationWarning'>, None, 0), ('ignore', None, <class 'ImportWarning'>, None, 0), ('ignore', None, <class 'ResourceWarning'>, None, 0)])
test_codeop failed (env changed)

== Tests result: ENV CHANGED ==

1 test altered the execution environment:
    test_codeop

Total duration: 38 ms
Tests result: ENV CHANGED


On an unrelated note the test passes with verbose mode though fail-env-changed flag is passed.

./python.exe -m test --fail-env-changed test_codeop -m test_warning -vvv
== CPython 3.10.0a0 (heads/master:c51db0ea40d, Aug 13 2020, 15:47:27) [Clang 10.0.1 (clang-1001.0.46.4)]
== macOS-10.14.6-x86_64-i386-64bit little-endian
== cwd: /Users/kasingar/stuff/python/cpython/build/test_python_29945æ
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 1.53 Run tests sequentially
0:00:00 load avg: 1.53 [1/1] test_codeop
test_warning (test.test_codeop.CodeopTests) ... ok

----------------------------------------------------------------------

Ran 1 test in 0.001s

OK

== Tests result: SUCCESS ==

1 test OK.

Total duration: 58 ms
Tests result: SUCCESS

> The commit 369a1cbdee14d9f27356fb3a8bb21e4fde289d25 introduces a change in environment. (...)

Right. It's fixed by PR 21848.

New changeset c818b15fa59039de67022c29085d439fa5d3ef95 by Terry Jan Reedy in branch 'master':
bpo-41520: Fix second codeop regression (GH-21848)
https://github.com/python/cpython/commit/c818b15fa59039de67022c29085d439fa5d3ef95

> New changeset c818b15fa59039de67022c29085d439fa5d3ef95 by Terry Jan Reedy in branch 'master':
> bpo-41520: Fix second codeop regression (GH-21848)
https://github.com/python/cpython/commit/c818b15fa59039de67022c29085d439fa5d3ef95

I confirm that this change fixed the "test altered the execution environment" issue.

New changeset f24430f1542ea2768793b48704ae2d4e241892ae by Terry Jan Reedy in branch '3.9':
[3.9] bpo-41520: Fix second codeop regression (GH-21848)
https://github.com/python/cpython/commit/f24430f1542ea2768793b48704ae2d4e241892ae

New changeset a3416c13b51af25675e175d4ffe07d8b925e7dbf by Miss Islington (bot) in branch '3.8':
[3.9] bpo-41520: Fix second codeop regression (GH-21848)
https://github.com/python/cpython/commit/a3416c13b51af25675e175d4ffe07d8b925e7dbf

Lukasz, please cherry-pick the 3.9 commits for this issue
90eff4ed4445a0fa9d8cdf74c0f86c50ed510dad
f24430f1542ea2768793b48704ae2d4e241892ae
into 3.9.0rc2 before release.
They fix a regression introduced by a bug fix last June.

stage: patch review -> resolved

nosy: + cheryl.sabella
messages: + msg375273
resolution: fixed -> (no value)
stage: resolved -> needs patch

pull_requests: + pull_request20975

messages: + msg375253

stage: patch review -> resolved

pull_requests: + pull_request20966
stage: patch review