Issue 21817: `concurrent.futures.ProcessPoolExecutor` swallows tracebacks

When you use `concurrent.futures.ProcessPoolExecutor` and an exception is raised in the created process, it doesn't show you the traceback. This makes debugging very annoying.

Example file:

    #!python
    
    import sys
    import concurrent.futures 
    
    def f():
        # Successful assert:
        assert True
        return g()
    
    def g():
        # Hard-to-find failing assert:
        assert False
    
        
    if __name__ == '__main__':
        with concurrent.futures.ProcessPoolExecutor(max_workers=1) as executor:
            assert isinstance(executor, concurrent.futures.Executor)
            future = executor.submit(f)
            future.result()
        print('Main process finished')
        
        
If you run this under Windows, you get this: 

    Traceback (most recent call last):
      File "./bug.py", line 20, in <module>
        future.result()
      File "c:\python34\lib\concurrent\futures\_base.py", line 402, in result
        return self.__get_result()
      File "c:\python34\lib\concurrent\futures\_base.py", line 354, in __get_result
        raise self._exception
    AssertionError

This is the traceback of the main process, while we want the traceback of the process that failed.

Hello. Here's a patch based on c4f92b597074, which adds something similar to multiprocessing.pool.
The output after the patch is:

concurrent.futures.process.RemoteTraceback:
"""
Traceback (most recent call last):
  File "D:\Projects\cpython\lib\concurrent\futures\process.py", line 153, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "D:\Projects\cpython\PCbuild\a.py", line 9, in f
    return g()
  File "D:\Projects\cpython\PCbuild\a.py", line 13, in g
    assert False
AssertionError
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "a.py", line 20, in <module>
    future.result()
  File "D:\Projects\cpython\lib\concurrent\futures\_base.py", line 402, in result
    return self.__get_result()
  File "D:\Projects\cpython\lib\concurrent\futures\_base.py", line 354, in __get_result
    raise self._exception
AssertionError


It's a little better than the current output, even though it's a little verbose.

Any type of feedback regarding this approach will be appreciated.

Hi Claudiu, sorry for the silence.

This output looks great. I'd love to see that go into Python.

Here's the updated version of this patch.

Hi, here are some comments about the patch:

- RemoteTraceback, ExceptionWithTraceback, rebuild_exc should be private (i.e. with a leading underscore)
- in test_traceback, you can use the context manager form of assertRaises instead of the try..except..else construct

Thanks, Antoine! Here's the new version, with your comments addressed.

New changeset a36b402b099b by Antoine Pitrou in branch 'default':
Issue #21817: When an exception is raised in a task submitted to a ProcessPoolExecutor, the remote traceback is now displayed in the parent process.
https://hg.python.org/cpython/rev/a36b402b099b

Thank you very much. Everything is now committed.

stage: patch review -> resolved

messages: + msg234186

versions: - Python 3.4

versions: + Python 3.5
keywords: + patch
nosy: + Claudiu.Popa

messages: + msg221218
stage: patch review