Issue 31903: `_scproxy` calls SystemConfiguration functions in a way that can cause deadlocks [proxy]

The Wayback Machine - https://web.archive.org/web/20210307021553/https://bugs.python.org/issue31903

Issue31903

classification

Title:	`_scproxy` calls SystemConfiguration functions in a way that can cause deadlocks
Type:	crash	Stage:	resolved
Components:	macOS	Versions:	Python 3.7, Python 3.6, Python 3.5, Python 2.7

process

Status:	closed	Resolution:	fixed
Dependencies:		Superseder:
Assigned To:		Nosy List:	Maxime Belanger, benjamin.peterson, miss-islington, ned.deily, ronaldoussoren
Priority:	normal	Keywords:	patch

Created on 2017-10-30 19:38 by Maxime Belanger, last changed 2018-09-12 15:38 by miss-islington. This issue is now closed.

Pull Requests
URL	Status	Linked	Edit
PR 4178	merged	maxbelanger, 2017-10-30 19:59
PR 9215	open	miss-islington, 2018-09-12 15:20
PR 9216	merged	miss-islington, 2018-09-12 15:20

Messages (6)
msg305247 - (view)	Author: Maxime Belanger (Maxime Belanger)	Date: 2017-10-30 19:38
Through the use of various Python packages (such as `pyobjc`), it is possible for a deadlock to occur due to how `_scproxy.c` calls `SCDynamicStoreCopyProxies`. In more recent versions of macOS (10.7+), this function can block on acquiring a lock deep inside `NSUserPreferences`. As `pyobjc` allows Python-wrapped `NSString`s to be stored in `CFPreferences`, it is thus possible for one thread to hold the `CFPreferences` lock and block on the GIL while another thread holds the GIL and blocks on the `CFPreferences` lock. We've observed this on a significant number of macOS devices before fixing ourselves by wrapping the calls to `SCDynamicStoreCopyProxies` with `Py_BEGIN/END_ALLOW_THREADS`.
msg325090 - (view)	Author: Benjamin Peterson (benjamin.peterson) *	Date: 2018-09-11 23:14
New changeset 4859ba0d2ce4506fddc3f55f90f8dce031b3804f by Benjamin Peterson (Max Bélanger) in branch 'master': closes bpo-31903: Release the GIL when calling into SystemConfiguration (GH-4178) https://github.com/python/cpython/commit/4859ba0d2ce4506fddc3f55f90f8dce031b3804f
msg325104 - (view)	Author: Ned Deily (ned.deily) *	Date: 2018-09-12 00:21
Thanks, Benjamin! Ronald, do you think this should be backported?
msg325135 - (view)	Author: Ronald Oussoren (ronaldoussoren) *	Date: 2018-09-12 12:33
Yes it should, the same problem has been present from the start.
msg325140 - (view)	Author: miss-islington (miss-islington)	Date: 2018-09-12 15:37
New changeset 72c34cf6dd5fb206fe4d407ff603468af466fd2e by Miss Islington (bot) in branch '3.7': closes bpo-31903: Release the GIL when calling into SystemConfiguration (GH-4178) https://github.com/python/cpython/commit/72c34cf6dd5fb206fe4d407ff603468af466fd2e
msg325141 - (view)	Author: miss-islington (miss-islington)	Date: 2018-09-12 15:38
New changeset aa12534acff4868f6dce9e2364a66fbb0ebcb3ca by Miss Islington (bot) in branch '3.6': closes bpo-31903: Release the GIL when calling into SystemConfiguration (GH-4178) https://github.com/python/cpython/commit/aa12534acff4868f6dce9e2364a66fbb0ebcb3ca

History
Date	User	Action	Args
2018-09-12 15:38:39	miss-islington	set	messages: + msg325141
2018-09-12 15:37:43	miss-islington	set	nosy: + miss-islington messages: + msg325140
2018-09-12 15:20:47	miss-islington	set	pull_requests: + pull_request8648
2018-09-12 15:20:37	miss-islington	set	pull_requests: + pull_request8647
2018-09-12 12:33:27	ronaldoussoren	set	messages: + msg325135
2018-09-12 00:21:04	ned.deily	set	messages: + msg325104
2018-09-11 23:14:03	benjamin.peterson	set	status: open -> closed nosy: + benjamin.peterson messages: + msg325090 resolution: fixed stage: patch review -> resolved
2017-10-30 19:59:15	maxbelanger	set	keywords: + patch stage: patch review pull_requests: + pull_request4148
2017-10-30 19:38:53	Maxime Belanger	create