Issue 40726: ast.Call end_lineno is defined and returns None

Reporting an issue from https://github.com/xonsh/xonsh/issues/3581; boils down to, ast.Call used to not define end_lineno in 3.8:

py -3.8 -c 'import ast; type(ast.Call().end_lineno)'                                                                                                                                                                 Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: 'Call' object has no attribute 'end_lineno'

However in 3.9 is defined with None:

py -3.9 -c 'import ast; type(ast.Call().end_lineno)'

This messes with some other operations in ast, namely https://github.com/python/cpython/blob/master/Lib/ast.py#L233

The issue is not limited to ast.Call. Other AST nodes are also affected (e.g. ast.BinOp). It's also not limited to end_lineno attribute. The same applies to end_col_offset.

This is because the 'end_lineno' and 'end_col_offset' are declared as optional attributes in the ASDL spec. The commit 'b7e9525f9c7ef02a1d2ad8253afdeb733b0951d4' made all optional fields and attributes auto initalized with None.

See here for the complete spec: https://docs.python.org/3.9/library/ast.html#abstract-grammar

This was done in b7e9525f9c7ef02a1d2ad8253afdeb733b0951d4 for issue 36287, all attribute should now be defined, even when they are not set. It looks like some parts of the ast module where not updated.

If nobody works on this I will send a PR to update the rest of AST later today.

> If nobody works on this I will send a PR to update the rest of AST later today.

:/ I'm terribly sorry for that, I just sent a PR without refreshing the tab.

There's current expectation in a lot of linters / code formatters that the *lineno and *col_offset attributes will be missing if they are not attached to the node

setting them to None is going to break a lot of those, if possible I'd suggest going back to when they were missing

> setting them to None is going to break a lot of those, if possible I'd suggest going back to when they were missing

'lineno' and 'col_offset' will never be none, since both are declared as normal integers. But on the other hand, 'end_lineno' and 'end_col_offset' are declared as optional integers which would make sense to auto initalize them with *None*, and I dont think it would ever break some code that complies with the ASDL declaration.

Actually, Xonsh (http://github.com/xonsh/xonsh) tests show that keyword AST nodes are missing 'lineno' attribute, but that could be our fault.

> Actually, Xonsh tests show that keyword AST nodes are missing 'lineno' attribute, but that could be our fault.

Yes, our fault. Sorry for the noise.

New changeset 8f4380d2f5839a321475104765221a7394a9d649 by Batuhan Taskaya in branch 'master':
bpo-40726: handle uninitalized end_lineno on ast.increment_lineno (GH-20312)
https://github.com/python/cpython/commit/8f4380d2f5839a321475104765221a7394a9d649

New changeset a1320989f5350439e0677450f49b36f2c10583d2 by Miss Islington (bot) in branch '3.9':
bpo-40726: handle uninitalized end_lineno on ast.increment_lineno (GH-20312)
https://github.com/python/cpython/commit/a1320989f5350439e0677450f49b36f2c10583d2

New changeset b24c9d2b0656764bef48120d9511faf833bd7ead by Batuhan Taskaya in branch '3.8':
[3.8] bpo-40726: handle uninitalized end_lineno on ast.increment_lineno (GH-21745)
https://github.com/python/cpython/commit/b24c9d2b0656764bef48120d9511faf833bd7ead

pull_requests: + pull_request19581
stage: patch review

messages: + msg369577

components: + Library (Lib)
type: behavior