Issue 37409: relative import without parent succeeds with builtins.__import__

>>> from curses import ascii
>>> from . import ascii

The second line should raise an ImportError but instead succeeds (tested cpython 3.6.7, 3.7.0 and 3.7.3, and from interactive interpreter and scripts). Specifically, builtins.__import__ does not reproduce the behaviour of importlib._bootstrap.__import__; maybe ceval.c:import_from is neglecting to check that there are parent packages when attempting a relative import?

More details here: https://stackoverflow.com/a/56768129/5104777

I can't reproduce:

>>> from importlib import abc
>>> from . import util
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'util' from '__main__' (unknown location)
>>> import importlib.util
>>>

Did you happen to do an `import *` prior to this in your REPL? If so that might explain it as you could have pulled in __package__ and such and that's used to resolve relative imports.

>>> foo = 'oops'
>>> from . import foo as fubar   # should raise ImportError
>>> fubar
'oops'

After further investigation, the problem is that builtins.__import__ (the c port version) does not replicate the behaviour of importlib.__import__ (the python reference version):

>>> import builtins, importlib
>>> __package__ is None
True
>>> importlib.__import__('', globals(), locals(), ('foo',), 1)
ImportError
>>> builtins.__import__('', globals(), locals(), ('foo',), 1)
<module '__main__' (built-in)>

A further discrepancy is that for deeper relative imports, builtins.__import__ raises a ValueError instead of ImportError (contrary to expectation/spec):

>>> from ...... import foo
ValueError

A simple work around uses the python implementation to restore expected behaviour:

>>> builtins.__import__ = importlib.__import__
>>> from ...... import foo
ImportError
>>> from curses import ascii
>>> from . import ascii
ImportError

PS: Brett Cannon, to replicate please copy and paste lines in correct order :-)

Please open a separate issue for the relative import issue.

I opened bpo-37444 for the relative import beyond top-level package issue.

The code doing the sanity check for importlib can be found at https://github.com/python/cpython/blob/f9f8e3ce709ceb15c8db8c8dde940daf1febf13d/Lib/importlib/_bootstrap.py#L943-L948 .

If you run with `-Xdev`/warnings turned on you get an idea of what's happening:

>>> builtins.__import__('', globals(), locals(), ('foo',), 1)
<stdin>:1: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
<module '__main__' (built-in)>

The check is being done in resolve_name() in import.c (https://github.com/python/cpython/blob/f9f8e3ce709ceb15c8db8c8dde940daf1febf13d/Python/import.c#L1543). My guess is there's an off-by-one error in the sanity check logic for attempting a relative import beyond the top-level package.

I'll look into this further and open a PR initially for the regression tests.

New changeset 92420b3e679959a7d0ce875875601a4cee45231e by Brett Cannon (Ben Lewis) in branch 'master':
bpo-37409: fix relative import with no parent (#14956)
https://github.com/python/cpython/commit/92420b3e679959a7d0ce875875601a4cee45231e

New changeset 0a6693a469cfb1dd5c8048d8cb4231a7b5883251 by Brett Cannon in branch '3.8':
[3.8] bpo-37409: fix relative import with no parent (GH-14956) (GH-15913)
https://github.com/python/cpython/commit/0a6693a469cfb1dd5c8048d8cb4231a7b5883251

New changeset f3480ad08823a9bc7df490bb5b54593d9483be70 by Miss Islington (bot) in branch '3.7':
[3.8] bpo-37409: fix relative import with no parent (GH-14956) (GH-15913)
https://github.com/python/cpython/commit/f3480ad08823a9bc7df490bb5b54593d9483be70

The test added seems to have created an ImportWarning in test_builtin.BuiltinTest.test_import .

./python.exe -Wall -m unittest -v test.test_builtin.BuiltinTest.test_import
test_import (test.test_builtin.BuiltinTest) ... /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/case.py:202: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
  callable_obj(*args, **kwargs)
ok

----------------------------------------------------------------------
Ran 1 test in 0.002s

OK

The test is same as below and given that __spec__ an __name__ are passed as None where ImportWarning is raised in Lib/importlib/_bootstrap.py 1074 . can we just use self.assertWarns(ImportWarning) in the test?


>>> __import__('', {'__package__': None, '__spec__': None, '__name__': '__main__'}, locals={}, fromlist=('foo',), level=1)
<stdin>:1: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: attempted relative import with no known parent package

Thanks for catching the warning and the fix, Karthikeyan!

title: relative import_from without parent -> relative import without parent

stage: resolved