Issue 39148: DatagramProtocol + IPv6 does not work with ProactorEventLoop

Receiving a UDP datagram using DatagramProtocol on the Proactor event loop results in error_received() being called with WinError 87 (Invalid Parameter). The low-level sock_recv() works fine, but naturally loses the sender address information. The attached script works fine as-is on Linux, and on Windows if ::1 is replaced with 127.0.0.1.

There were extensive tests added for UDP support on IOCP, but unfortunately all of them use only IPv4 sockets so they could not catch this problem.

I tried some debugging with Python 3.9.0a4 and it looks like unparse_address in overlapped.c (https://github.com/python/cpython/blob/v3.9.0a4/Modules/overlapped.c#L689) raises the error. Almost seems like ENABLE_IPV6 is not set...

Just to confirm, for some reason ENABLE_IPV6 is not set when compiling _overlapped. I'm not familiar enough with the windows build infrastructure to submit a PR though.

Well, I found this: https://github.com/python/cpython/blob/3c97e1e457033bbb8bbe0b7198bd13fc794a12b0/configure.ac#L3278-L3279
Could it be that the official binaries were compiled without --enable-ipv6?

./configure is not run on Windows, but I believe the define should go here:

https://github.com/python/cpython/blob/8510f430781118d9b603c3a2f06945d6ebc5fe42/PC/pyconfig.h#L678

Would somebody be a champion for the issue?

Main thing we need is a PR (straightforward) and confirming that the relevant tests all run. We can use the buildbot fleet to make sure that all supported platforms have the necessary APIs, but it wouldn't surprise me if there's some weird edge cases here.

PR is up. Any chance we get this reviewed for inclusion in 3.9.0a6 / 3.8.3rc1?

Please let me know if there is anything else I can do to get this going.

Which PR is it?

Oh, it's https://github.com/python/cpython/pull/19121. I think it would be prudent to add a test as well to make sure this doesn't happen again.

Fair enough, PR updated.

Thanks, looks good to me now!

The PR is still awaiting for a core developer to be reviewed. It's too bad we missed the 3.8.3 window, but perhaps this can get included in 3.9.0 at least.

New changeset 442634c42fcaf31c636f693951a97734042c3e7b by Kjell Braden in branch 'master':
bpo-39148: enable ipv6 for datagrams in Proactor (GH-19121)
https://github.com/python/cpython/commit/442634c42fcaf31c636f693951a97734042c3e7b

New changeset 94d9c5e5afdee4f46be9d9faaa39d6be40c34849 by Miss Islington (bot) in branch '3.8':
bpo-39148: enable ipv6 for datagrams in Proactor (GH-19121)
https://github.com/python/cpython/commit/94d9c5e5afdee4f46be9d9faaa39d6be40c34849

Bot failed for AIX     https://buildbot.python.org/all/#builders/227/builds/978 with:

0:07:11 Re-running test_asyncio in verbose mode
Failed to import test module: test.test_asyncio.test_events
Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_asyncio/test_events.py", line 239, in <module>
    class EventLoopTestsMixin:
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_asyncio/test_events.py", line 1262, in EventLoopTestsMixin
    @unittest.skipUnless(support.IPV6_ENABLED, 'IPv6 not supported or enabled')
AttributeError: module 'test.support' has no attribute 'IPV6_ENABLED'
test test_asyncio crashed -- Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/libregrtest/runtest.py", line 270, in _runtest_inner
    refleak = _runtest_inner2(ns, test_name)
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/libregrtest/runtest.py", line 234, in _runtest_inner2
    test_runner()
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/libregrtest/runtest.py", line 208, in _test_module
    raise Exception("errors while loading tests")
Exception: errors while loading tests
1 test failed again:
    test_asyncio

New changeset 58205a0217e91232cc1e945dbfe4e387d636eb76 by Nathaniel J. Smith in branch 'master':
bpo-39148: fixup to account for IPV6_ENABLED being moved (GH-20170)
https://github.com/python/cpython/commit/58205a0217e91232cc1e945dbfe4e387d636eb76

I think I fixed the buildbot issues in GH-20170, but I can't seem to reach the buildbot site right now, so it's hard to know for sure!

I could not "fathom" the buildbot test results - however, a manual build of PR29170 on 3.9 works: I'll try 3.8, but then from master (assuming it is already part of master) -- and that works as well.

Thanks for the quick update!

messages: + msg367131