@serhiy-storchaka, thanks for your PR! By analyzing the history of the files in this pull request, we identified @benjaminp, @loewis and @tim-one to be potential reviewers.

@serhiy-storchaka if Parser/token.c is auto-generated by the tool, why do you include it in this PR ?

Nice cleaning up of the code. Generally LGTM.

Tools/scripts/generate_token_h.py and Tools/scripts/generate_token_rst.py are very similar, it would be neater to merge them.

Created alternate PR #10370 which takes all information from Grammar/Tokens and use a single generating script.

Tools/scripts/generate_token_h.py, Tools/scripts/generate_token_rst.py and Tools/scripts/generate_token_c.py have been merged into Tools/scripts/generate_token.py. See also #10370.

Closed in favor of #10370.