Issue 37926: regression: PySys_SetArgvEx(0, NULL, 0): SystemError: Python-3.8.0b3/Objects/unicodeobject.c:2089: bad argument to internal function

There is a regression between Python 3.7 and 3.8 when using PySys_SetArgvEx(0, NULL, 0).

Consider this example:

#include <Python.h>

int main() {
    Py_Initialize();
    PySys_SetArgvEx(0, NULL, 0);  /* HERE */
    PyRun_SimpleString("from time import time,ctime\n"
                       "print('Today is', ctime(time()))\n");
    Py_FinalizeEx();
    return 0;
}


This works in 3.7 but no longer does in 3.8:

$ gcc $(python3.7-config --cflags --ldflags) example.c 
$ ./a.out 
Today is Fri Aug 23 07:59:52 2019

$ gcc $(python3.8-config --cflags --ldflags --embed) example.c 
$ ./a.out 
Fatal Python error: no mem for sys.argv
SystemError: /builddir/build/BUILD/Python-3.8.0b3/Objects/unicodeobject.c:2089: bad argument to internal function

Current thread 0x00007f12c78b9740 (most recent call first):
Aborted (core dumped)


The documentation https://docs.python.org/3/c-api/init.html#c.PySys_SetArgvEx explicitly mentions passing 0 to PySys_SetArgvEx:

"if argc is 0..."

So I guess this is not something you shouldn't do.

Oops, it's my fault! PR 15415 fix the crash.

The regression was introduced by:

commit 74f6568bbd3e70806ea3219e8bacb386ad802ccf
Author: Victor Stinner <vstinner@redhat.com>
Date:   Fri Mar 15 15:08:05 2019 +0100

    bpo-36301: Add _PyWstrList structure (GH-12343)

Simplified change:

-        static wchar_t *empty_argv[1] = {L""};
+        wchar_t* empty_argv[1] = {L""};

It's a deliberate change to not waste memory: PySys_SetArgvEx() (make_sys_argv()) is only called once, whereas static keeps the memory for the whole lifecycle of the process.

But I didn't notice a subtle issue with memory allocated on the stack: the code works well with gcc -O0! The bug only triggers with gcc -O3.

New changeset c48682509dc49b43fe914fe6c502bc390345d1c2 by Victor Stinner in branch 'master':
bpo-37926: Fix PySys_SetArgvEx(0, NULL, 0) crash (GH-15415)
https://github.com/python/cpython/commit/c48682509dc49b43fe914fe6c502bc390345d1c2

New changeset ca9ae94a2aba35d94ac1ec081f9bcac3a13aebd3 by Victor Stinner in branch '3.8':
bpo-37926: Fix PySys_SetArgvEx(0, NULL, 0) crash (GH-15415) (GH-15420)
https://github.com/python/cpython/commit/ca9ae94a2aba35d94ac1ec081f9bcac3a13aebd3

I tested my fix manually using example from msg350262 and gcc -O3 (./configure && make): I can reproduce the crash without the change, and I confirm that my change fix the crash.

Thanks Miro for the bug report: the fix will be included in next Python 3.8 beta release. I fixed the bug in 3.8 and master branches (3.7 is not affected).

FYI this bug was found in paraview in Fedora Rawhide: https://bugzilla.redhat.com/show_bug.cgi?id=1743896

resolution: fixed
stage: patch review -> resolved