Issue 18018: SystemError: Parent module '' not loaded, cannot perform relative import

When executing a submodule, there's a SystemError in 3.3 where we used to receive a ValueError.

mkdir marsu
touch marsu/__init__.py
echo "from .houba import bi" >> marsu/pilami.py

./python marsu/pilami.py

Traceback (most recent call last):
  File "marsu/pilami.py", line 2, in <module>
    from .houba import bi
SystemError: Parent module '' not loaded, cannot perform relative import


In Python 3.2 (or Python 2.7):

./python3.2 marsu/pilami.py

Traceback (most recent call last):
  File "marsu/pilami.py", line 2, in <module>
    from .houba import bi
ValueError: Attempted relative import in non-package

For the specific case given, both error messages are misleading anyway - the problem is attempting to directly execute a module inside a package instead of using "-m".

However, for the case of attempting a relative import from a top level module, the old message is indeed preferable.

The line raising the exception is http://hg.python.org/cpython/file/f7992397e98d/Lib/importlib/_bootstrap.py#l1518 .

If you're sure it's a regression feel free to change what exception is raised.

User code should generally not see any SystemErrors (even when choosing wacky module names :-)), so IMHO this is a regression.

This still happens in current CPython, and doesn't even require a package:

rosuav@sikorsky:~$ echo 'from . import x' >syserr.py
rosuav@sikorsky:~$ python3 syserr.py
Traceback (most recent call last):
  File "syserr.py", line 1, in <module>
    from . import x
SystemError: Parent module '' not loaded, cannot perform relative import


This just caused some confusion for a student of mine who had migrated from Python 2. Converting intra-package imports from "import spam" to "from . import spam" made them work; converting a non-package import the same way caused this problem. ImportError would be massively preferable.

But I'm not understanding something here. If I set __package__ inside the module, the behaviour changes accordingly:

rosuav@sikorsky:~$ cat syserr.py 
__package__ = 'sys'
from . import version
print(version)
rosuav@sikorsky:~$ python3 syserr.py 
3.6.0a0 (default:ac94418299bd+, Jan 15 2016, 08:44:02) 
[GCC 4.9.2]


Leaving __package__ unset has it implicitly be None. Whether it's None or '', the package checks should simply not be made - see three lines above the line Brett linked to (in today's code, that's https://hg.python.org/cpython/file/ac94418299b/Lib/importlib/_bootstrap.py#l925 - hasn't changed). So I'm not sure why the checks are even happening. However, that probably means more about my exploration and testing than it does about the code; editing the text of the message doesn't result in a change, so I'm obviously not updating the frozen version.

The problem is in Python/import.c: https://hg.python.org/cpython/file/default/Python/import.c#l1453 . While the Python code has a check that `package` is a truthy value, the accelerated C code doesn't. Adding a check that the string isn't empty or truthy should do the trick as there seems to already be a check for None earlier.

The other question is whether this should change back to ValueError for Python 2 compatibility or turn into ImportError as Chris and Eric Snow have independently suggested?

Either way I'm only going to change this in Python 3.6 since the current semantics have been this way since Python 3.3, so it's beyond the realm of regression and into fixing a design mistake.

If someone made a new way of importing and had it raise ValueError on some issue or other, I think there'd be complete consensus that that's the wrong exception. Yes, this is incompatible with Python 2 - but there are a lot of corner cases in the 3.3+ import code that differ from 2.7. What we have is a code error ("relative imports outside of packages don't make sense, dummy!") and the only difference is which exception is being raised (ValueError, SystemError, ImportError). It's not like working code has different semantics.

But even if ValueError is chosen, I would still greatly prefer that to SystemError. When my student told me he was getting SystemError, I started trying to diagnose a corrupted venv, not a buggy script.

New changeset c4e4886c6052 by Brett Cannon in branch 'default':
Issue #18018: Raise an ImportError if a relative import is attempted
https://hg.python.org/cpython/rev/c4e4886c6052

Thanks to everyone for providing feedback. I went with ImportError in the end as that's what the pure Python implementation of __import__() already raised.

New changeset c824cc8426a16dd9f3949a3ed135523d37787bae by Serhiy Storchaka in branch '3.5':
[3.5] Backport bpo-30876 (GH-2639), bpo-18018 and bpo-26367. (#2677)
https://github.com/python/cpython/commit/c824cc8426a16dd9f3949a3ed135523d37787bae

stage: test needed -> resolved

messages: + msg258230
versions: + Python 3.5, Python 3.6