PEP 394 – The “python” Command on Unix-Like Systems | peps.python.org

Abstract

This PEP outlines the behavior of Python scripts when the python command is invoked. Depending on a distribution or system configuration, python may or may not be installed. If python is installed its target interpreter may refer to python2 or python3. End users may be unaware of this inconsistency across Unix-like systems. This PEP’s goal is to reduce user confusion about what python references and what will be the script’s behavior.

The recommendations in the next section of this PEP will outline the behavior when:

• using virtual environments
• writing cross-platform scripts with shebangs for either python2 or python3

The PEP’s goal is to clarify the behavior for script end users, distribution providers, and script maintainers / authors.

Recommendation

Our recommendations are detailed below. We call out any expectations that these recommendations are based upon.

History of this PEP

In 2011, the majority of distributions aliased the python command to Python 2, but some started switching it to Python 3 ([5]). As some of the former distributions did not provide a python2 command by default, there was previously no way for Python 2 code (or any code that invokes the Python 2 interpreter directly rather than via sys.executable) to reliably run on all Unix-like systems without modification, as the python command would invoke the wrong interpreter version on some systems, and the python2 command would fail completely on others. This PEP originally provided a very simple mechanism to restore cross-platform support, with minimal additional work required on the part of distribution maintainers. Simplified, the recommendation was:

1. The python command was preferred for code compatible with both Python 2 and 3 (since it was available on all systems, even those that already aliased it to Python 3).
2. The python command should always invoke Python 2 (to prevent hard-to-diagnose errors when Python 2 code is run on Python 3).
3. The python2 and python3 commands should be available to specify the version explicitly.

However, these recommendations implicitly assumed that Python 2 would always be available. As Python 2 is nearing its end of life in 2020 (PEP 373, PEP 404), distributions are making Python 2 optional or removing it entirely. This means either removing the python command or switching it to invoke Python 3. Some distributors also decided that their users were better served by ignoring the PEP’s original recommendations, and provided system administrators with the freedom to configure their systems based on the needs of their particular environment.

Current Rationale

As of 2019, activating a Python virtual environment (or its functional equivalent) prior to script execution is one way to obtain a consistent cross-platform and cross-distribution experience.

Accordingly, publishers can expect users of the software to provide a suitable execution environment.

Future Changes to this Recommendation

This recommendation will be periodically reviewed over the next few years, and updated when the core development team judges it appropriate. As a point of reference, regular maintenance releases for the Python 2.7 series will continue until January 2020.

Migration Notes

This section does not contain any official recommendations from the core CPython developers. It’s merely a collection of notes regarding various aspects of migrating to Python 3 as the default version of Python for a system. They will hopefully be helpful to any distributions considering making such a change.

• The main barrier to a distribution switching the python command from python2 to python3 isn’t breakage within the distribution, but instead breakage of private third party scripts developed by sysadmins and other users. Updating the python command to invoke python3 by default indicates that a distribution is willing to break such scripts with errors that are potentially quite confusing for users that aren’t familiar with the backwards incompatible changes in Python 3. For example, while the change of print from a statement to a builtin function is relatively simple for automated converters to handle, the SyntaxError from attempting to use the Python 2 notation in Python 3 may be confusing for users that are not aware of the change:

  $ python3 -c 'print "Hello, world!"'
    File "<string>", line 1
      print "Hello, world!"
            ^
  SyntaxError: Missing parentheses in call to 'print'. Did you mean print("Hello, world!")?
  

  While this might be obvious for experienced Pythonistas, such scripts might even be run by people who are not familiar with Python at all. Avoiding breakage of such third party scripts was the key reason this PEP used to recommend that python continue to refer to python2.

• The error message python: command not found tends to be surprisingly actionable, even for people unfamiliar with Python.
• The pythonX.X (e.g. python3.6) commands exist on modern systems, on which they invoke specific minor versions of the Python interpreter. It can be useful for distribution-specific packages to take advantage of these utilities if they exist, since it will prevent code breakage if the default minor version of a given major version is changed. However, scripts intending to be cross-platform should not rely on the presence of these utilities, but rather should be tested on several recent minor versions of the target major version, compensating, if necessary, for the small differences that exist between minor versions. This prevents the need for sysadmins to install many very similar versions of the interpreter.
• When the pythonX.X binaries are provided by a distribution, the python2 and python3 commands should refer to one of those files rather than being provided as a separate binary file.
• It is strongly encouraged that distribution-specific packages use python3 (or python2) rather than python, even in code that is not intended to operate on other distributions. This will reduce problems if the distribution later decides to change the version of the Python interpreter that the python command invokes, or if a sysadmin installs a custom python command with a different major version than the distribution default.
• If the above point is adhered to and sysadmins are permitted to change the python command, then the python command should always be implemented as a link to the interpreter binary (or a link to a link) and not vice versa. That way, if a sysadmin does decide to replace the installed python file, they can do so without inadvertently deleting the previously installed binary.
• Even as the Python 2 interpreter becomes less common, it remains reasonable for scripts to continue to use the python3 convention, rather than just python.
• If these conventions are adhered to, it will become the case that the python command is only executed in an interactive manner as a user convenience, or else when using a virtual environment or similar mechanism.

Backwards Compatibility

A potential problem can arise if a script adhering to the python2/python3 convention is executed on a system not supporting these commands. This is mostly a non-issue, since the sysadmin can simply create these symbolic links and avoid further problems. It is a significantly more obvious breakage than the sometimes cryptic errors that can arise when attempting to execute a script containing Python 2 specific syntax with a Python 3 interpreter or vice versa.

Application to the CPython Reference Interpreter

While technically a new feature, the make install and make bininstall command in the 2.7 version of CPython were adjusted to create the following chains of symbolic links in the relevant bin directory (the final item listed in the chain is the actual installed binary, preceding items are relative symbolic links):

python -> python2 -> python2.7
python-config -> python2-config -> python2.7-config

Similar adjustments were made to the macOS binary installer.

This feature first appeared in the default installation process in CPython 2.7.3.

The installation commands in the CPython 3.x series already create the appropriate symlinks. For example, CPython 3.2 creates:

python3 -> python3.2
idle3 -> idle3.2
pydoc3 -> pydoc3.2
python3-config -> python3.2-config

And CPython 3.3 creates:

python3 -> python3.3
idle3 -> idle3.3
pydoc3 -> pydoc3.3
python3-config -> python3.3-config
pysetup3 -> pysetup3.3

The implementation progress of these features in the default installers was managed on the tracker as issue #12627 ([3]).

Impact on PYTHON* Environment Variables

The choice of target for the python command implicitly affects a distribution’s expected interpretation of the various Python related environment variables. The use of *.pth files in the relevant site-packages folder, the “per-user site packages” feature (see python -m site) or more flexible tools such as virtualenv are all more tolerant of the presence of multiple versions of Python on a system than the direct use of PYTHONPATH.

Exclusion of MS Windows

This PEP deliberately excludes any proposals relating to Microsoft Windows, as devising an equivalent solution for Windows was deemed too complex to handle here. PEP 397 and the related discussion on the python-dev mailing list address this issue.

References

Copyright

This document has been placed in the public domain.