Created on 2019-05-24 09:25 by vstinner, last changed 2020-01-01 06:12 by miss-islington. This issue is now closed.
Each type types.CodeType constructor changes, a lot of applications break. Python 3.8 added a new parameter which broke for example the Genshi project, just to name one.
I propose to add a new CodeType.replace() method, similar to datetime.datetime.replace and namedtuple._replace() for example:
$ python3
Python 3.7.3 (default, Mar 27 2019, 13:41:07)
>>> import datetime
>>> d=datetime.datetime.now()
>>> d2=d.replace(year=2010)
>>> d, d2
(datetime.datetime(2019, 5, 24, 11, 25, 3, 839966), datetime.datetime(2010, 5, 24, 11, 25, 3, 839966))
>>> import collections
>>> Point = collections.namedtuple("Point", "x y")
>>> p=Point(1, 2)
>>> p2=p._replace(y=3)
>>> p, p2
(Point(x=1, y=2), Point(x=1, y=3))
> Python 3.8 added a new parameter which broke for example the Genshi project Pablo fixed it in Genshi: https://github.com/edgewall/genshi/pull/19
See also bpo-37034: "Argument Clinic omits name of keyword-only parameter on _PyArg_BadArgument() call".
Fix in Hypothesis: https://github.com/HypothesisWorks/hypothesis/commit/8f47297fa2e19c426a42b06bb5f8bf1406b8f0f3#diff-a097452b8ffe8323641ccccac335fbf9
ipython fix: https://github.com/ipython/ipython/commit/248128dfaabb33e922b1e36a298fd7ec0c730069
It's not like Python 3.8 is the first type in Python history that code constructor changed. The PyCode_New() function slowly growed from 3 parameters in 1990 to 16 paramters in 2019 :-) History of PyCode_New(). New "posonlyargcount" parameter: commit 8c77b8cb9188165a123f2512026e3629bf03dc9b Author: Pablo Galindo <Pablogsal@gmail.com> Date: Mon Apr 29 13:36:57 2019 +0100 bpo-36540: PEP 570 -- Implementation (GH-12701) New "kwonlyargcount" parameter, bpo-1549670: commit 4f72a78684bbfcdc43ceeabb240ceee54706c4b0 Author: Guido van Rossum <guido@python.org> Date: Fri Oct 27 23:31:49 2006 +0000 Jiwon Seo's PEP 3102 implementation. See SF#1549670. The compiler package has not yet been updated. New "freevars" and "cellvars" parameters, PEP 227: commit 64949cb753f206c0ca1d83f55d07afd3c179b81a Author: Jeremy Hylton <jeremy@alum.mit.edu> Date: Thu Jan 25 20:06:59 2001 +0000 PEP 227 implementation ... Include/compile.h Add co_freevars and co_cellvars slots to code objects. Update PyCode_New() to take freevars and cellvars as arguments New "firstlineno" and "lnotab" parameters: commit da4eb5c3b55df9b7d6957ddd0458313102f0a92e Author: Guido van Rossum <guido@python.org> Date: Fri Jan 24 03:43:35 1997 +0000 Instead of emitting SET_LINENO instructions, generate a line number table which is incorporated in the code object. This way, the runtime overhead to keep track of line numbers is only incurred when an exception has to be reported. New "stacksize" parameter: commit 8b993a98db507cc3a75067af4c865ffc8afbada1 Author: Guido van Rossum <guido@python.org> Date: Fri Jan 17 21:04:03 1997 +0000 Add co_stacksize field to codeobject structure, and stacksize argument to PyCode_New() argument list. Move MAXBLOCKS constant to conpile.h. Added accurate calculation of the actual stack size needed by the generated code. Also commented out all fprintf statements (except for a new one to diagnose stack underflow, and one in #ifdef'ed out code), and added some new TO DO suggestions (now that the stacksize is taken of the TO DO list). New "argcount", "nlocals", "flags" and "varnames" parameters: commit 681d79aaf397850778608f35585d091fa7fe370a Author: Guido van Rossum <guido@python.org> Date: Tue Jul 18 14:51:37 1995 +0000 keyword arguments and faster calls New "name" parameter: commit 9bfef44d97d1ae24e03717e3d59024b44626a9de Author: Guido van Rossum <guido@python.org> Date: Mon Mar 29 10:43:31 1993 +0000 ... * Added function name to code object. Print it for code and function objects. THIS MAKES THE .PYC FILE FORMAT INCOMPATIBLE (the version number has changed accordingly) ... New "filename" parameter: commit 3f5da24ea304e674a9abbdcffc4d671e32aa70f1 Author: Guido van Rossum <guido@python.org> Date: Thu Dec 20 15:06:42 1990 +0000 "Compiling" version The very first version of the function was: static codeobject * newcodeobject(code, consts, names) object *code; object *consts; object *names; From this commit: commit 10dc2e8097e7a431367e72f46ddba91be93aa159 Author: Guido van Rossum <guido@python.org> Date: Sun Nov 18 17:27:39 1990 +0000 Initial revision
> It's not like Python 3.8 is the first type in Python history that code constructor changed. Oops, typo: It's not like Python 3.8 is the first *time* in Python history that the code constructor changed.
Fix in Cloud Pickle: https://github.com/cloudpipe/cloudpickle/commit/b9dc17fc5f723ffbfc665295fafdd076907c0a93
See bpo-36886 "Failed to construct CodeType on Python-3.8.0a4" for other failures caused by the addition of the "posonlyargcount" parameter to code constructor.
New changeset a9f05d69ccbf3c75cdd604c25094282697789a62 by Victor Stinner in branch 'master': bpo-37032: Add CodeType.replace() method (GH-13542) https://github.com/python/cpython/commit/a9f05d69ccbf3c75cdd604c25094282697789a62
Once Python 3.8 beta1 will be released, it would be interesting to patch all projects that I had to be fixed to handle the new "posonlyargcount" to use the new "replace()" method is available. Maybe even remove the fix for Python 3.8 alpha 4 and only use use replace() on Python 3.8 and newer.
Is there anything more on this issue?
Pablo: "Is there anything more on this issue?" I proposed "Once Python 3.8 beta1 will be released, it would be interesting to patch all projects that I had to be fixed to handle the new "posonlyargcount" to use the new "replace()" method is available." But that only concerns third party project,s no Python directly, so I close the issue :-)
Thanks for this great improvement! Was about to suggest having something more futureproof when noticing this with pdbpp, found this, and submitted PRs to use it for ipython and hypothesis also.
> submitted PRs to use it for ipython and hypothesis also Cool! Thank you for doing that ;-)
stage: patch review -> resolved