Issue 36272: Recursive logging crashes Interpreter in Python 3

Following code logs an error and calls itself leading to stack overflow and eventually core dump in Python 3.6.

>>> import logging
>>> def rec():
...     logging.error("foo")
...     rec()
>>> rec()

[1]    101641 abort (core dumped)  python3
FTR, this doesn't crash Python 2.7.

Attaching the error (condensed) in Python 3.6:

ERROR:root:foo
...
--- Logging error ---
Traceback (most recent call last):
...
RecursionError: maximum recursion depth exceeded in comparison
...
Fatal Python error: Cannot recover from stack overflow.
...
[1]    101641 abort (core dumped)  python3
Python 2.7:

RuntimeError: maximum recursion depth exceeded
But no core dump in Python 2.7.

FTR, the error above with Python 3.6 will come into play if the log level is set to logging.ERROR. Similarly for other log levels.

I am not sure this is related to logging and looks similar to issue35542 except stack (depends on OS) is exhausted without setrecursionlimit(). What does below return?

def rec():
    rec()

rec()

Stack exhaustion doesn't seem to be due to be the root cause. A simple recursive function doesn't crash the interpreter in Python 3.6.

>>> def rec(): rec()
>>> rec()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in rec
  File "<stdin>", line 1, in rec
  File "<stdin>", line 1, in rec
  [Previous line repeated 995 more times]
RecursionError: maximum recursion depth exceeded

I'm not sure issue35542. I think this happens because while logging the recursion limit is hit which calls https://github.com/python/cpython/blob/master/Python/ceval.c#L535-L539.

The RecursionError is then handled by https://github.com/python/cpython/blob/master/Lib/logging/__init__.py#L1000 and cleared.

On subsequent calls the exception is not set anymore because `tstate->overflowed` equals 1 so we exit early before setting the exception again at https://github.com/python/cpython/blob/master/Python/ceval.c#L531.

This goes on until the condition on https://github.com/python/cpython/blob/master/Python/ceval.c#L531 pass which abort the interpreter.

I think there is two ways to solve the issue, either handle RecursionError explicitly in the logging module so we don't clear it inadvertently as there is no way to recover from it anyway or check if the exception has been cleared at https://github.com/python/cpython/blob/master/Python/ceval.c#L531 and set it again.

Handling it explictly in the logging module would not help for code doing this elsewhere:

def rec():
    try:
        rec()
    except:
        rec()
rec()


I can submit a patch if you want.

The following patch fixes the issue and raise RecursionError as expecting without core dump:

diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py
index b4659af7cc..7457549cb9 100644
--- a/Lib/logging/__init__.py
+++ b/Lib/logging/__init__.py
@@ -1094,6 +1094,8 @@ class StreamHandler(Handler):
             # issue 35046: merged two stream.writes into one.
             stream.write(msg + self.terminator)
             self.flush()
+        except RecursionError as e:
+            raise
         except Exception:
             self.handleError(record)

> Stack exhaustion doesn't seem to be due to be the root cause. A simple recursive function doesn't crash the interpreter in Python 3.6.

Yes, sorry I got misleaded. I have added logging module author, @vinay.sajip to the issue.

Limiting the version scope to 3.6 until someone reproduces on 3.7 and/or 3.8.

Hi Brett, I confirm the test case breaks both Python3.7 and 3.8. I opened a PR to fix the problem.

New changeset 65f64b1903ae85b97a30f514bbc1b7ce940c3af2 by Vinay Sajip (Rémi Lapeyre) in branch 'master':
bpo-36272: Logging now propagates RecursionError (GH-12312)
https://github.com/python/cpython/commit/65f64b1903ae85b97a30f514bbc1b7ce940c3af2

New changeset 6a7a9f1d83cef628d2bacd71ee568b93f53fd6b4 by Vinay Sajip (Miss Islington (bot)) in branch '3.7':
bpo-36272: Logging now propagates RecursionError (GH-12312) (GH-12391)
https://github.com/python/cpython/commit/6a7a9f1d83cef628d2bacd71ee568b93f53fd6b4

messages: + msg337879
versions: - Python 3.7, Python 3.8

messages: + msg337843
versions: + Python 3.7, Python 3.8