Issue38236
Created on 2019-09-20 16:44 by vstinner, last changed 2019-09-23 14:01 by vstinner. This issue is now closed.
| Pull Requests | |||
|---|---|---|---|
| URL | Status | Linked | Edit |
| PR 16300 | merged | vstinner, 2019-09-20 17:07 | |
| PR 16332 | merged | vstinner, 2019-09-23 12:57 | |
| PR 16333 | merged | vstinner, 2019-09-23 13:13 | |
| Messages (8) | |||
|---|---|---|---|
| msg352869 - (view) | Author: STINNER Victor (vstinner) * | Date: 2019-09-20 16:44 | |
When Python path configuration is not properly configured, Python fails with an error looking like: --- Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding Python runtime state: core initialized ModuleNotFoundError: No module named 'encodings' Current thread 0x00007ff0065b2740 (most recent call first): <no Python frame> --- This error is very common and hard to debug. It can happen in various cases for various reasons: * Python is embedded in an application * PYTHONPATH and/or PYTHONHOME environment variable is set * pyenv.cfg configuration for virtual environment * Local .py files which override the standard library A few examples: * bpo-38234: Py_SetPath() regression in Python 3.8 which impacts PyInstaller * https://bugs.python.org/issue35978#msg335363 virtual env * https://bugs.python.org/issue35969#msg335249 "encodings.py" file in the current directory * https://stackoverflow.com/questions/55657593/python-3-8-0a3-failed-to-get-the-python-codec-of-the-filesystem-encoding embedded Python * https://askubuntu.com/questions/640010/fatal-python-error-py-initialize-unable-to-get-the-locale-encoding-syntaxe Python 3 installed by Anaconda which uses Python 2.7 stdlib because of PYTHONPATH * https://github.com/GNS3/gns3-gui/issues/2267 Python embedded in "gns3" * https://github.com/JuliaPy/PyCall.jl/issues/410 venv * https://techglimpse.com/error-executing-python3-5-command-solution/ another random error Attached PR dumps the Python path configuration if importing the Python codec of the filesystem encoding fails. In the Python initialization, it's the first import which is done. |
|||
| msg352873 - (view) | Author: STINNER Victor (vstinner) * | Date: 2019-09-20 17:08 | |
Without my change: --- vstinner@apu$ PYTHONHOME=/xxx ./python Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding Python runtime state: core initialized ModuleNotFoundError: No module named 'encodings' Current thread 0x00007fa9c32d6740 (most recent call first): <no Python frame> --- With my PR 16300: --- $ PYTHONHOME=/xxx ./python Python path configuration: PYTHONHOME = '/xxx' PYTHONPATH = '/usr/share/qa-tools/python-modules' isolated = 0 environment = 1 user site = 1 import site = 1 sys.executable = '/home/vstinner/python/master/python' sys.prefix = '/xxx' sys.exec_prefix = '/xxx' sys.path = [ '/usr/share/qa-tools/python-modules', '/xxx/lib/python39.zip', '/xxx/lib/python3.9', '/xxx/lib/python3.9/lib-dynload', ] Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding Python runtime state: core initialized ModuleNotFoundError: No module named 'encodings' Current thread 0x00007f9cef949740 (most recent call first): <no Python frame> --- |
|||
| msg352874 - (view) | Author: STINNER Victor (vstinner) * | Date: 2019-09-20 17:13 | |
> user site = 1 > import site = 1 I'm not sure that dumping these configuration variables are useful to debug this very specific error: "Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding" They only have an effect on sys.path later. |
|||
| msg352925 - (view) | Author: Stéphane Wirtel (matrixise) * | Date: 2019-09-21 05:39 | |
+1 |
|||
| msg353013 - (view) | Author: STINNER Victor (vstinner) * | Date: 2019-09-23 12:46 | |
New changeset fcdb027234566c4d506d6d753c7d5638490fb088 by Victor Stinner in branch 'master': bpo-38236: Dump path config at first import error (GH-16300) https://github.com/python/cpython/commit/fcdb027234566c4d506d6d753c7d5638490fb088 |
|||
| msg353015 - (view) | Author: STINNER Victor (vstinner) * | Date: 2019-09-23 13:35 | |
New changeset 88e6447451fb5525e83e802c66c3e51b4a45bf86 by Victor Stinner in branch 'master': bpo-38236: Fix init_dump_ascii_wstr() (GH-16333) https://github.com/python/cpython/commit/88e6447451fb5525e83e802c66c3e51b4a45bf86 |
|||
| msg353016 - (view) | Author: STINNER Victor (vstinner) * | Date: 2019-09-23 13:59 | |
New changeset c5c642565e260477ae2fb29d0c86a91e19702ae3 by Victor Stinner in branch '3.8': bpo-38236: Dump path config at first import error (GH-16300) (GH-16332) https://github.com/python/cpython/commit/c5c642565e260477ae2fb29d0c86a91e19702ae3 |
|||
| msg353017 - (view) | Author: STINNER Victor (vstinner) * | Date: 2019-09-23 14:01 | |
I backported my change to Python 3.8.
Example with PYTHONPATH set to Python 2.7 with Python 3.8:
$ PYTHONPATH=/usr/lib64/python2.7/ ./python -c pass
Python path configuration:
PYTHONHOME = (not set)
PYTHONPATH = '/usr/lib64/python2.7/'
program name = './python'
isolated = 0
environment = 1
user site = 1
import site = 1
sys._base_executable = '/home/vstinner/python/3.8/python'
sys.base_prefix = '/usr/local'
sys.base_exec_prefix = '/usr/local'
sys.executable = '/home/vstinner/python/3.8/python'
sys.prefix = '/usr/local'
sys.exec_prefix = '/usr/local'
sys.path = [
'/usr/lib64/python2.7/',
'/usr/local/lib/python38.zip',
'/home/vstinner/python/3.8/Lib',
'/home/vstinner/python/3.8/build/lib.linux-x86_64-3.8-pydebug',
]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
File "/usr/lib64/python2.7/encodings/__init__.py", line 123
raise CodecRegistryError,\
^
SyntaxError: invalid syntax
Current thread 0x00007fe3ade4d740 (most recent call first):
<no Python frame>
|
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2019-09-23 14:01:37 | vstinner | set | status: open -> closed resolution: fixed stage: patch review -> resolved |
| 2019-09-23 14:01:26 | vstinner | set | messages:
+ msg353017 versions: + Python 3.8 |
| 2019-09-23 13:59:08 | vstinner | set | messages: + msg353016 |
| 2019-09-23 13:36:00 | vstinner | set | messages: + msg353015 |
| 2019-09-23 13:13:40 | vstinner | set | pull_requests: + pull_request15910 |
| 2019-09-23 12:57:52 | vstinner | set | pull_requests: + pull_request15909 |
| 2019-09-23 12:46:03 | vstinner | set | messages: + msg353013 |
| 2019-09-21 05:39:55 | matrixise | set | nosy:
+ matrixise messages: + msg352925 |
| 2019-09-20 17:13:10 | vstinner | set | messages: + msg352874 |
| 2019-09-20 17:08:39 | vstinner | set | messages: + msg352873 |
| 2019-09-20 17:07:19 | vstinner | set | keywords:
+ patch stage: patch review pull_requests: + pull_request15886 |
| 2019-09-20 16:44:55 | vstinner | create | |