Issue 38644: Pass explicitly tstate to function calls

Follow-up of bpo-36710 for function calls.

PR 16997 is a first step. The second step will be to pass tstate to:

* _PyObject_Vectorcall()
* _PyObject_MakeTpCall()
* _Py_CheckFunctionResult()

(I have a local branch that I have to rewrite on top of PR 16997.)

New changeset f4b1e3d7c64985f5d5b00f6cc9a1c146bbbfd613 by Victor Stinner in branch 'master':
bpo-38644: Add Py_EnterRecursiveCall() to the limited API (GH-17046)
https://github.com/python/cpython/commit/f4b1e3d7c64985f5d5b00f6cc9a1c146bbbfd613

New changeset be434dc0380d9f5c7c800de9943cc46d55fd9491 by Victor Stinner in branch 'master':
bpo-38644: Pass tstate to Py_EnterRecursiveCall() (GH-16997)
https://github.com/python/cpython/commit/be434dc0380d9f5c7c800de9943cc46d55fd9491

New changeset 17269090940aa20f6079a6b9f27ae319f8cdae14 by Victor Stinner in branch 'master':
bpo-38644: Pass tstate to _Py_CheckFunctionResult() (GH-17050)
https://github.com/python/cpython/commit/17269090940aa20f6079a6b9f27ae319f8cdae14

New changeset 7e433733175e76627d46ed9bdab543860cd1452d by Victor Stinner in branch 'master':
bpo-38644: Add _PyObject_VectorcallTstate() (GH-17052)
https://github.com/python/cpython/commit/7e433733175e76627d46ed9bdab543860cd1452d

I started a thread on python-dev about this issue:
"Pass the Python thread state to internal C functions"
https://mail.python.org/archives/list/python-dev@python.org/thread/PQBGECVGVYFTVDLBYURLCXA3T7IPEHHO/

New changeset b9e681261cd5ce6db0a79461c58d7cc52cfa4902 by Victor Stinner in branch 'master':
bpo-38644: Add _PyEval_EvalFrame() with tstate (GH-17131)
https://github.com/python/cpython/commit/b9e681261cd5ce6db0a79461c58d7cc52cfa4902

New changeset 4d231bcc77ac8ce7d11bda0804130dcdd678f710 by Victor Stinner in branch 'master':
bpo-38644: Add _PyObject_Call() (GH-17089)
https://github.com/python/cpython/commit/4d231bcc77ac8ce7d11bda0804130dcdd678f710

New changeset b5e170f127b57d5b0a4fb58f316acd6191509dce by Victor Stinner in branch 'master':
bpo-38644: Add _PyEval_EvalCode() (GH-17183)
https://github.com/python/cpython/commit/b5e170f127b57d5b0a4fb58f316acd6191509dce

New changeset 51edf8aaa2e17626f9690ed29d25945fc03016b9 by Victor Stinner in branch 'master':
bpo-38644: Cleanup ceval.h (GH-17185)
https://github.com/python/cpython/commit/51edf8aaa2e17626f9690ed29d25945fc03016b9

One further step would be to change the VECTORCALL/FASTCALL calling convention to pass tstate. But I am not sure what is the risk to do that in Python 3.9? Cython uses FASTCALL internally for example.

> One further step would be to change the VECTORCALL/FASTCALL calling convention to pass tstate. But I am not sure what is the risk to do that in Python 3.9? Cython uses FASTCALL internally for example.

I started a thread on python-dev:
https://mail.python.org/archives/list/python-dev@python.org/thread/PIXJAJPWKDGHSQD65VOO2B7FDLU2QLHH/

I also wrote an article on this issue, "Pass the Python thread state explicitly":
https://vstinner.github.io/cpython-pass-tstate.html

New changeset 2b1df4592e1691017414337514c6e378eb639498 by Victor Stinner in branch 'master':
bpo-38644: Pass tstate to _Py_FinishPendingCalls() (GH-17990)
https://github.com/python/cpython/commit/2b1df4592e1691017414337514c6e378eb639498

New changeset 61f4db8c56ca4b7e60050d96ecc255cfb03d92a0 by Victor Stinner in branch 'master':
bpo-38644: Pass tstate in ceval.c (GH-18222)
https://github.com/python/cpython/commit/61f4db8c56ca4b7e60050d96ecc255cfb03d92a0

New changeset 877ea88934a5164be4d9f15207694fad4173d87d by Victor Stinner in branch 'master':
bpo-38644: Add Py_EnterRecursiveCall() to python3.def (GH-18399)
https://github.com/python/cpython/commit/877ea88934a5164be4d9f15207694fad4173d87d

> One further step would be to change the VECTORCALL/FASTCALL calling convention to pass tstate.

My first strong motivation is to get None singleton from tstate. It's now being discussed in bpo-39511: "[subinterpreters] Per-interpreter singletons (None, True, False, etc.)".

New changeset f3fda374685dffa31ebda9e681e00ef7032b8a1d by Victor Stinner in branch 'master':
bpo-38644: Rephrase What's New entry (GH-18461)
https://github.com/python/cpython/commit/f3fda374685dffa31ebda9e681e00ef7032b8a1d

New changeset 08faf0016e1ee590c78f64ddb244767c7801866a by Victor Stinner in branch 'master':
bpo-38644: Add _PySys_Audit() which takes tstate (GH-19180)
https://github.com/python/cpython/commit/08faf0016e1ee590c78f64ddb244767c7801866a

New changeset 728189884e0e128c4ffc57b785b04584d57a90c0 by Victor Stinner in branch 'master':
bpo-38644: Pass tstate explicitly in signalmodule.c (GH-19184)
https://github.com/python/cpython/commit/728189884e0e128c4ffc57b785b04584d57a90c0

New changeset 71a3522ef85df06a3acc718107360e37e4116a15 by Victor Stinner in branch 'master':
bpo-38644: Make tstate more explicit inside pystate.c (GH-19182)
https://github.com/python/cpython/commit/71a3522ef85df06a3acc718107360e37e4116a15

New changeset 1c1e68cf3e3a2a19a0edca9a105273e11ddddc6e by Victor Stinner in branch 'master':
bpo-38644: Use _PySys_Audit(): pass tstate explicitly (GH-19183)
https://github.com/python/cpython/commit/1c1e68cf3e3a2a19a0edca9a105273e11ddddc6e

This adds cost to our most critical code paths.  For example, type_call() will now be slower for every single object instantiation.

Raymond:
> This adds cost to our most critical code paths.  For example, type_call() will now be slower for every single object instantiation.

This issue is now closed. Please open a new issue if you consider that the code should be modified. It would also help to have a benchmark if it's a performance regression ;-)