Issue35715
Created on 2019-01-11 08:36 by dchevell, last changed 2019-03-16 22:29 by pablogsal. This issue is now closed.
| Pull Requests | |||
|---|---|---|---|
| URL | Status | Linked | Edit |
| PR 11514 | merged | dchevell, 2019-01-11 08:41 | |
| PR 11514 | merged | dchevell, 2019-01-11 08:41 | |
| PR 11514 | merged | dchevell, 2019-01-11 08:41 | |
| PR 11514 | merged | dchevell, 2019-01-11 08:41 | |
| Messages (2) | |||
|---|---|---|---|
| msg333444 - (view) | Author: David Chevell (dchevell) * | Date: 2019-01-11 08:36 | |
ProcessPoolExecutor workers will hold onto the return value of their last task in memory until the next task is received. Since the return value has already been propagated to the parent process's `Future` or else effectively discarded, this is holding onto objects unnecessarily.
Simple case to reproduce:
import concurrent.futures
import time
executor = concurrent.futures.ProcessPoolExecutor(max_workers=1)
def big_val():
return [{1:1} for i in range(1, 1000000)]
executor.submit(big_val)
# Observe the memory usage of the process worker during the sleep interval
time.sleep(10)
This should be easily fixed by having the worker explicitly `del r` after calling `_sendback_result` as it already does this for `call_item`
|
|||
| msg338105 - (view) | Author: Pablo Galindo Salgado (pablogsal) * | Date: 2019-03-16 22:28 | |
New changeset 962bdeab191ee64459caa199209331005797ea7a by Pablo Galindo (Dave Chevell) in branch 'master': bpo-35715: Liberate return value of _process_worker (GH-11514) https://github.com/python/cpython/commit/962bdeab191ee64459caa199209331005797ea7a |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2019-03-16 22:29:06 | pablogsal | set | keywords:
patch, patch, patch, patch status: open -> closed resolution: fixed stage: patch review -> resolved |
| 2019-03-16 22:28:53 | pablogsal | set | nosy:
+ pablogsal messages: + msg338105 |
| 2019-01-13 18:25:56 | xtreak | set | keywords:
patch, patch, patch, patch nosy: + bquinlan, pitrou |
| 2019-01-11 08:42:00 | dchevell | set | keywords:
+ patch stage: patch review pull_requests: + pull_request11078 |
| 2019-01-11 08:41:56 | dchevell | set | keywords:
+ patch stage: (no value) pull_requests: + pull_request11077 |
| 2019-01-11 08:41:52 | dchevell | set | keywords:
+ patch stage: (no value) pull_requests: + pull_request11076 |
| 2019-01-11 08:41:48 | dchevell | set | keywords:
+ patch stage: (no value) pull_requests: + pull_request11075 |
| 2019-01-11 08:36:21 | dchevell | create | |