Issue 28805: Add documentation for METH_FASTCALL and _PyObject_FastCall*()

It looks like METH_FASTCALL gives nice speedups (#28754).

Is this an internal interface or can it be documented like
METH_KEYWORDS etc.?

I suggest to keep it internal. It's meaning may be changed in future. I have some ideas about making it more convenient and faster (but I'm not sure that they would work).

Is it possible to use Argument Clinic for third party extensions? If yes, I
suggest to use it.

It would also be nice if Cython could use it automatically. So we can
suggest to use Cython. I don't know well Cython, maybe your idea is very
dumb :-)

Serhiy Storchaka added the comment:
> I suggest to keep it internal. It's meaning may be changed in future. I
have some ideas about making it more convenient and faster (but I'm not
sure that they would work).

Ah? Can you please elaborate your secret plan? :-)

There are a couple of problems with using Argument Clinic for third
party projects.

First, it makes no stability promises:

"Currently Argument Clinic is considered internal-only for CPython. ..."


Then, for large projects that already use some generated code (like
NumPy), IMO the result would be unmaintainable.


So it would be nice to have a public stable API -- of course there
is no hurry if Serhiy has plans to improve it.

> Ah? Can you please elaborate your secret plan? :-)

I already said about this. Move the part of parsing keyword argument outside of the function. The caller should unpack keyword arguments and pass just a raw array of PyObject*. Missed arguments are set to NULL. This could make _PyArg_ParseStack() simpler and might even allow to inline arguments parsing code.

This and the _PyObject_FastCall* APIs really do need to be documented, especially if they're going to be used in the interpreter itself.  The leading underscore signifies that it's not part of the public API (well, METH_FASTCALL doesn't have a leading underscore).

When these are not documented, then they aren't discoverable, so there are cases where it may be useful but will be overlooked.  It also means that folks will be inclined to cargo cult the few existing uses without perhaps understanding the full semantics.

Can we please document these for Python 3.7 at least?

> Can we please document these for Python 3.7 at least?

I chose to not document FASTCALL on purpose in Python 3.6, I wanted to keep everything private, until we get enough feedback and stabilize the API.

Recently, the FASTCALL API changed: METH_FASTCALL doesn't support keywords anymore. You have to pass METH_FASTCALL|METH_KEYWORDS.

It seems like the API became stable.

I don't know if we should make the API public or not. PyPy would complain as usual that we give them more work to do :-D

I suggest to document the following 4 functions/macros:

PyAPI_FUNC(PyObject *) _PyObject_FastCallDict(
    PyObject *callable,
    PyObject **args,
    Py_ssize_t nargs,
    PyObject *kwargs);

PyAPI_FUNC(PyObject *) _PyObject_FastCallKeywords(
    PyObject *callable,
    PyObject **args,
    Py_ssize_t nargs,
    PyObject *kwnames);

#define _PyObject_FastCall(func, args, nargs) \
    _PyObject_FastCallDict((func), (args), (nargs), NULL)

#define _PyObject_CallNoArg(func) \
    _PyObject_FastCallDict((func), NULL, 0, NULL)

And the METH_FASTCALL and METH_FASTCALL|METH_KEYWORDS calling convention.

> It would also be nice if Cython could use it automatically.

Cython is using FASTCALL since Python 3.6.

Victor, of the four functions you mentioned:

- _PyObject_FastCallDict: documented by PEP 590

- _PyObject_FastCallKeywords: renamed _PyObject_Vectorcall and documented by PEP 590

- _PyObject_FastCall: not sure if it's useful enough (just use _PyObject_Vectorcall with kwnames=NULL)

- _PyObject_CallNoArg: see #37194

So that leaves documenting METH_FASTCALL.

New changeset 5600b5e1b24a3491e83f1b3038a7ea047a34c0bf by Inada Naoki (Jeroen Demeyer) in branch 'master':
bpo-28805: document METH_FASTCALL (GH-14079)
https://github.com/python/cpython/commit/5600b5e1b24a3491e83f1b3038a7ea047a34c0bf

New changeset b101fa7783615051a89500e488708b955eac94c5 by Miss Islington (bot) in branch '3.7':
bpo-28805: document METH_FASTCALL (GH-14079)
https://github.com/python/cpython/commit/b101fa7783615051a89500e488708b955eac94c5

New changeset e784f9f1c3fdd2102aae3fc0fe226408ff3a6029 by Miss Islington (bot) in branch '3.8':
bpo-28805: document METH_FASTCALL (GH-14079)
https://github.com/python/cpython/commit/e784f9f1c3fdd2102aae3fc0fe226408ff3a6029

PR 14079 has been merged, is there anything left as unresolved about this issue?

messages: + msg303701
versions: + Python 3.7