[proxy] web.archive.org← back | site home | direct (HTTPS) ↗ | proxy home | ◑ dark◐ light
/ cpython Public
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-31642: Restore blocking "from" import by setting None in sys.modules. #3834

Merged

Conversation

Copy link
Member

serhiy-storchaka commented Sep 30, 2017

serhiy-storchaka added needs backport to 3.6 type-bug An unexpected behavior, bug, or error labels Sep 30, 2017
serhiy-storchaka requested a review from a team Sep 30, 2017
if exc.name == from_name:
if (exc.name == from_name and
from_name not in sys.modules):
#sys.modules.get(from_name, _NEEDS_LOADING) is not None):
Copy link
Member Author

serhiy-storchaka Sep 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know what condition is better. Tests are passed in both cases.

Copy link
Member Author

serhiy-storchaka Sep 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On further thoughts, perhaps the only reliable way is testing the error message. There is possible race condition with testing sys.modules.

Copy link
Member

brettcannon Oct 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I vote for the latter since it's more explicit what you're actually worrying about.

Copy link
Member Author

serhiy-storchaka Oct 3, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't there a race condition? More precisely, is a race condition here bad?

Copy link
Member

brettcannon Oct 3, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the import lock released at this point? @pitrou might have an opinion on how bad the race condition might be (I personally don't as I've avoided the import lock as much as possible 😄 ).

Copy link
Contributor

ncoghlan Oct 4, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're definitely not holding any of the import locks here (not the global one, and not the one for from_name either), so there are two race conditions to consider:

  1. Some other thread inserted None into sys.modules while this thread's failed import attempt was running
  2. Some other thread successfully imported the module, or inserted a value other than None while this thread's import attempt was running

In neither case do we actually care whether that state change was made before our import attempt, or by a different thread while our import attempt was running, but we do want to treat the cases differently: we want the first one to be considered a failed import, and we want the second one to be considered a successful import.

So I think the sys.modules.get(from_name, _NEEDS_LOADING) is not None check would be the right one to use.

Copy link
Member Author

serhiy-storchaka Oct 4, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is yet one possibility. Some other thread started importing the module. It added the module into sys.modules, but still not set the attribute in the parent module.

Copy link
Contributor

ncoghlan Oct 4, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fine, as whether it fails or not will then depend on whether subsequent code uses LOAD_ATTR or IMPORT_FROM, just as it does for circular imports.

# If fromlist entry is None, let a ModuleNotFoundError propagate.
# issue31642
mod = types.ModuleType(PKG_NAME)
mod.__path__ = ['XXX']
Copy link
Member

brettcannon Oct 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can actually just set __path__ to [] and it will still count as a package.

if exc.name == from_name:
if (exc.name == from_name and
from_name not in sys.modules):
#sys.modules.get(from_name, _NEEDS_LOADING) is not None):
Copy link
Member

brettcannon Oct 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I vote for the latter since it's more explicit what you're actually worrying about.

serhiy-storchaka merged commit f07e2b6 into python:master Oct 8, 2017
Copy link
Contributor

miss-islington commented Oct 8, 2017

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.6.
🐍🍒🤖

serhiy-storchaka deleted the import-blocked-fromlist branch Oct 8, 2017
Copy link
Contributor

miss-islington commented Oct 8, 2017

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.6 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker f07e2b64df6304a36fb5e29397d3c77a7ba17704 3.6

serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this pull request Oct 8, 2017
Copy link

bedevere-bot commented Oct 8, 2017

GH-3923 is a backport of this pull request to the 3.6 branch.

serhiy-storchaka added a commit that referenced this pull request Oct 8, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants