Created on 2014-07-10 17:38 by David.Edelsohn, last changed 2014-09-11 11:35 by serhiy.storchaka. This issue is now closed.
The patch for Issue #21881 causes CPython test_tcl to crash on AIX. $ ./python -m test -v test_tcl == CPython 3.5.0a0 (default:d1f89eb9ea1e+, Jul 10 2014, 10:21:22) [GCC 4.8.1] == AIX-1-00F84C0C4C00-powerpc-32bit big-endian == hash algorithm: siphash24 32bit == /home/dje/src/cpython/build/test_python_27984522 Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0) [1/1] test_tcl patchlevel = 8.4.7 testCall (test.test_tcl.TclTest) ... ok testCallException (test.test_tcl.TclTest) ... ok testCallException2 (test.test_tcl.TclTest) ... ok testEval (test.test_tcl.TclTest) ... ok testEvalException (test.test_tcl.TclTest) ... ok testEvalException2 (test.test_tcl.TclTest) ... ok testEvalFile (test.test_tcl.TclTest) ... ok testEvalFileException (test.test_tcl.TclTest) ... ok testGetVar (test.test_tcl.TclTest) ... ok testGetVarArray (test.test_tcl.TclTest) ... ok testGetVarArrayException (test.test_tcl.TclTest) ... ok testGetVarException (test.test_tcl.TclTest) ... ok testLoadWithUNC (test.test_tcl.TclTest) ... skipped 'Requires Windows' testPackageRequireException (test.test_tcl.TclTest) ... ok testSetVar (test.test_tcl.TclTest) ... ok testSetVarArray (test.test_tcl.TclTest) ... ok testUnsetVar (test.test_tcl.TclTest) ... ok testUnsetVarArray (test.test_tcl.TclTest) ... ok testUnsetVarException (test.test_tcl.TclTest) ... ok test_eval_null_in_result (test.test_tcl.TclTest) ... ok test_evalfile_null_in_result (test.test_tcl.TclTest) ... ok test_exprboolean (test.test_tcl.TclTest) ... ok test_exprdouble (test.test_tcl.TclTest) ... ok test_exprlong (test.test_tcl.TclTest) ... ok test_exprstring (test.test_tcl.TclTest) ... ok test_getboolean (test.test_tcl.TclTest) ... ok test_getdouble (test.test_tcl.TclTest) ... ok test_getint (test.test_tcl.TclTest) ... ok test_passing_values (test.test_tcl.TclTest) ... ok test_split (test.test_tcl.TclTest) ... ok test_splitlist (test.test_tcl.TclTest) ... ok test_user_command (test.test_tcl.TclTest) ... Assertion failed: __EX, file Objects/methodobject.c, line 94 Fatal Python error: Aborted Current thread 0x00000001 (most recent call first): File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 426 in check File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 458 in test_user_command File "/home/dje/src/cpython/Lib/unittest/case.py", line 577 in run File "/home/dje/src/cpython/Lib/unittest/case.py", line 625 in __call__ File "/home/dje/src/cpython/Lib/unittest/suite.py", line 125 in run File "/home/dje/src/cpython/Lib/unittest/suite.py", line 87 in __call__ File "/home/dje/src/cpython/Lib/unittest/suite.py", line 125 in run File "/home/dje/src/cpython/Lib/unittest/suite.py", line 87 in __call__ File "/home/dje/src/cpython/Lib/unittest/runner.py", line 168 in run File "/home/dje/src/cpython/Lib/test/support/__init__.py", line 1724 in _run_suite File "/home/dje/src/cpython/Lib/test/support/__init__.py", line 1758 in run_unittest File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 601 in test_main File "/home/dje/src/cpython/Lib/test/regrtest.py", line 1278 in runtest_inner File "/home/dje/src/cpython/Lib/test/regrtest.py", line 978 in runtest File "/home/dje/src/cpython/Lib/test/regrtest.py", line 763 in main File "/home/dje/src/cpython/Lib/test/regrtest.py", line 1562 in main_in_temp_cwd File "/home/dje/src/cpython/Lib/test/__main__.py", line 3 in <module> File "/home/dje/src/cpython/Lib/runpy.py", line 85 in _run_code File "/home/dje/src/cpython/Lib/runpy.py", line 170 in _run_module_as_main IOT/Abort trap (core dumped)
Is it crashes when comment out "check((), '')"? Is it crashes when comment out two previous checks (for inf and -inf)?
If I comment out all three tests, it runs.
--- a/Lib/test/test_tcl.py Thu Jul 10 01:17:11 2014 -0400
+++ b/Lib/test/test_tcl.py Sat Jul 12 16:59:33 2014 -0700
@@ -455,9 +455,9 @@
check(float('inf'), 'Inf', eq=float_eq)
check(-float('inf'), '-Inf', eq=float_eq)
# XXX NaN representation can be not parsable by float()
- check((), '')
- check((1, (2,), (3, 4), '5 6', ()), '1 2 {3 4} {5 6} {}')
- check([1, [2,], [3, 4], '5 6', []], '1 2 {3 4} {5 6} {}')
+ #check((), '')
+ #check((1, (2,), (3, 4), '5 6', ()), '1 2 {3 4} {5 6} {}')
+ #check([1, [2,], [3, 4], '5 6', []], '1 2 {3 4} {5 6} {}')
def test_splitlist(self):
splitlist = self.interp.tk.splitlist
Is following test passed?
check((1,), '1')
Is CPython crashes when change signature of testfunc?
--- a/Lib/test/test_tcl.py Sat Jul 12 18:26:03 2014 +0300
+++ b/Lib/test/test_tcl.py Sun Jul 13 19:21:55 2014 +0300
@@ -416,7 +416,7 @@
def test_user_command(self):
result = None
- def testfunc(arg):
+ def testfunc(arg=None, *args):
nonlocal result
result = arg
return arg
> Is following test passed?
> check((1,), '1')
That test succeeds.
> Is CPython crashes when change signature of testfunc?
--- a/Lib/test/test_tcl.py Sat Jul 12 18:26:03 2014 +0300
+++ b/Lib/test/test_tcl.py Sun Jul 13 19:21:55 2014 +0300
@@ -416,7 +416,7 @@
def test_user_command(self):
result = None
- def testfunc(arg):
+ def testfunc(arg=None, *args):
nonlocal result
result = arg
return arg
This change does not have any effect on the original failure. The assertion failure and crash continues to occur.
#5 0x1019bb90 in PyCFunction_Call (func=0x30533b5c, arg=0x305d8ab4, kw=0x0)
at Objects/methodobject.c:94
#6 0x1012534c in call_function (pp_stack=0x2ff16144, oparg=2)
at Python/ceval.c:4269
94 CHECK_RESULT(res);
(gdb) print res
$9 = (PyObject *) 0x0
(gdb) print meth
$10 = (PyCFunction) @0x20380888: 0xd8dd57dc <Tkapp_Call>
(gdb) print self
$11 = (PyObject *) 0x305d44d8
May be this patch will help.
No difference with the patch. The problem is not a a malloc() failure.
New changeset 7de64def6565 by Serhiy Storchaka in branch '2.7': Issue #21951: Temporary skip crashing test_user_command on AIX. http://hg.python.org/cpython/rev/7de64def6565 New changeset 31f4cb1fede9 by Serhiy Storchaka in branch '3.4': Issue #21951: Temporary skip crashing test_user_command on AIX. http://hg.python.org/cpython/rev/31f4cb1fede9 New changeset de32cd419174 by Serhiy Storchaka in branch 'default': Issue #21951: Temporary skip crashing test_user_command on AIX. http://hg.python.org/cpython/rev/de32cd419174
Link to log with a crash: http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/2442/steps/test/logs/stdio
The failing assertion means that the method raised an exception and returned a result, or failed but didn't raised an exception. According to the gdb trace, the function failed (res is NULL). You can display the current exception in gdb using: "print PyErr_Occurred()" (NULL if there is no exception). Please run test_tcl on AIX with attached test_tkinter.patch applied.
As far as I remember, both res and PyErr_Occurred() were NULL.
When the failing test is re-enabled, the _tkinter patch produces the following output: test_user_command (test.test_tcl.TclTest) ... Assertion failed: __EX, file /home/dje/src/cpython/Modules/_tkinter.c, line 1277 Fatal Python error: Aborted Current thread 0x00000001 (most recent call first): File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 435 in check File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 468 in test_user_command File "/home/dje/src/cpython/Lib/unittest/case.py", line 577 in run File "/home/dje/src/cpython/Lib/unittest/case.py", line 625 in __call__ File "/home/dje/src/cpython/Lib/unittest/suite.py", line 125 in run File "/home/dje/src/cpython/Lib/unittest/suite.py", line 87 in __call__ File "/home/dje/src/cpython/Lib/unittest/suite.py", line 125 in run File "/home/dje/src/cpython/Lib/unittest/suite.py", line 87 in __call__ File "/home/dje/src/cpython/Lib/unittest/runner.py", line 168 in run File "/home/dje/src/cpython/Lib/test/support/__init__.py", line 1750 in _run_suite File "/home/dje/src/cpython/Lib/test/support/__init__.py", line 1784 in run_unittest File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 615 in test_main File "/home/dje/src/cpython/Lib/test/regrtest.py", line 1280 in runtest_inner File "/home/dje/src/cpython/Lib/test/regrtest.py", line 978 in runtest File "/home/dje/src/cpython/Lib/test/regrtest.py", line 763 in main File "/home/dje/src/cpython/Lib/test/regrtest.py", line 1564 in main_in_temp_cwd File "/home/dje/src/cpython/Lib/test/__main__.py", line 3 in <module> File "/home/dje/src/cpython/Lib/runpy.py", line 85 in _run_code File "/home/dje/src/cpython/Lib/runpy.py", line 170 in _run_module_as_main IOT/Abort trap (core dumped)
New changeset 9ab404cdcaa1 by Victor Stinner in branch 'default': Issue #21951: Fix AsObj() of the _tkinter module: raise MemoryError on memory http://hg.python.org/cpython/rev/9ab404cdcaa1
> When the failing test is re-enabled, the _tkinter patch produces the following output: Thanks. I found and fixed a bug in AsObj() for tuple and list. It looks like ckalloc(size) returns NULL if size is 0. Please update Python, compile and run test_tcl to see if you get a MemoryError exception instead of a crash. I just created the issue #22336 which may fix this issue if my analysis was correct. It uses PyMem_Malloc(size) which has a well-defined behaviour when size is 0.
$ ./python -m test -v test_tcl == CPython 3.5.0a0 (default:9ab404cdcaa1+, Sep 4 2014, 10:06:33) [GCC 4.8.1] == AIX-1-00F84C0C4C00-powerpc-32bit big-endian == hash algorithm: siphash24 32bit == /home/dje/src/cpython/build/test_python_48694044 Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0) [1/1] test_tcl patchlevel = 8.4.7 testCall (test.test_tcl.TclTest) ... ok testCallException (test.test_tcl.TclTest) ... ok testCallException2 (test.test_tcl.TclTest) ... ok testEval (test.test_tcl.TclTest) ... ok testEvalException (test.test_tcl.TclTest) ... ok testEvalException2 (test.test_tcl.TclTest) ... ok testEvalFile (test.test_tcl.TclTest) ... ok testEvalFileException (test.test_tcl.TclTest) ... ok testGetVar (test.test_tcl.TclTest) ... ok testGetVarArray (test.test_tcl.TclTest) ... ok testGetVarArrayException (test.test_tcl.TclTest) ... ok testGetVarException (test.test_tcl.TclTest) ... ok testLoadWithUNC (test.test_tcl.TclTest) ... skipped 'Requires Windows' testPackageRequireException (test.test_tcl.TclTest) ... ok testSetVar (test.test_tcl.TclTest) ... ok testSetVarArray (test.test_tcl.TclTest) ... ok testUnsetVar (test.test_tcl.TclTest) ... ok testUnsetVarArray (test.test_tcl.TclTest) ... ok testUnsetVarException (test.test_tcl.TclTest) ... ok test_eval_null_in_result (test.test_tcl.TclTest) ... ok test_evalfile_null_in_result (test.test_tcl.TclTest) ... ok test_exprboolean (test.test_tcl.TclTest) ... ok test_exprdouble (test.test_tcl.TclTest) ... ok test_exprlong (test.test_tcl.TclTest) ... ok test_exprstring (test.test_tcl.TclTest) ... ok test_getboolean (test.test_tcl.TclTest) ... ok test_getdouble (test.test_tcl.TclTest) ... ok test_getint (test.test_tcl.TclTest) ... ok test_passing_values (test.test_tcl.TclTest) ... ok test_split (test.test_tcl.TclTest) ... ok test_splitlist (test.test_tcl.TclTest) ... ok test_user_command (test.test_tcl.TclTest) ... ERROR testFlattenLen (test.test_tcl.TkinterTest) ... ok test_huge_string_builtins (test.test_tcl.BigmemTclTest) ... skipped 'needs UINT_MAX < SIZE_MAX' test_huge_string_call (test.test_tcl.BigmemTclTest) ... skipped 'needs UINT_MAX < SIZE_MAX' ====================================================================== ERROR: test_user_command (test.test_tcl.TclTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 468, in test_user_command check((), '') File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 435, in check r = self.interp.call('testfunc', value) MemoryError ---------------------------------------------------------------------- Ran 35 tests in 0.133s FAILED (errors=1, skipped=3) test test_tcl failed 1 test failed: test_tcl
Thanks David! Your ouput confirms my analysis. asobj_empty_sequence.patch should fix the test on AIX. I read Tcl source code, Tcl_NewListObj() doesn't use the second parameter if the first parameter (length) is zero.
The patch can be simpler. And there is yet one similar bug, for unicode strings.
As Victor noted on IRC, ckalloc() panics and doesn't returns NULL in case of error. We should use attemptckalloc() instead.
I'm not sure that tkinter_ckallock.patch is correct. Extract of attemptckalloc() manual page: "If the allocation fails, these functions will return NULL. Note that on some platforms, but not all, attempting to allocate a zero-sized block of memory will also cause these functions to return NULL." http://linux.die.net/man/3/attemptckalloc It looks like you are fixing two different issues in the same patch: fix AIX and enhance handling of memory allocation failure. Can you please split your patch in two parts? For AIX, adding these two lines for list/tuple should be enough: if (size == 0) return Tcl_NewListObj(0, NULL);
"I'm not sure that tkinter_ckallock.patch is correct." Oh, when I read my message, I realized that I was not explicit enough. It looks like attemptckalloc() can be called with 0 at some places with tkinter_ckallock.patch. But I didn't check carefully, the patch is maybe correct.
New changeset ee969a717cb5 by Serhiy Storchaka in branch '2.7': Issue #21951: Use attemptckalloc() instead of ckalloc() in Tkinter. http://hg.python.org/cpython/rev/ee969a717cb5 New changeset 1223c882253f by Serhiy Storchaka in branch '3.4': Issue #21951: Use attemptckalloc() instead of ckalloc() in Tkinter. http://hg.python.org/cpython/rev/1223c882253f New changeset 499b60b7d067 by Serhiy Storchaka in branch 'default': Issue #21951: Use attemptckalloc() instead of ckalloc() in Tkinter. http://hg.python.org/cpython/rev/499b60b7d067 New changeset d6c7ab5a2065 by Serhiy Storchaka in branch '2.7': Issue #21951: Fixed a crash in Tkinter on AIX when called Tcl command with http://hg.python.org/cpython/rev/d6c7ab5a2065 New changeset 6a96c28f9474 by Serhiy Storchaka in branch '3.4': Issue #21951: Fixed a crash in Tkinter on AIX when called Tcl command with http://hg.python.org/cpython/rev/6a96c28f9474 New changeset 7b7bae546959 by Serhiy Storchaka in branch 'default': Issue #21951: Fixed a crash in Tkinter on AIX when called Tcl command with http://hg.python.org/cpython/rev/7b7bae546959
Committed these two changes as separate patches.
> Committed these two changes as separate patches. Thanks, it's clearer like that.
test_tcl now pass on AIX: http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/2592/steps/test/logs/stdio Thanks Serhiy for the fix.
Thank you Victor for great investigation of this issue.