[proxy] bugs.python.org — ← back | site home | direct (HTTPS) ↗ | proxy home | ◑ dark◐ light

Issue 35934: Add socket.create_server() utility function

Issue35934

classification
Title: Add socket.create_server() utility function
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, cheryl.sabella, giampaolo.rodola, jaraco, josiah.carlson, loewis, neologix, pablogsal, pitrou, vstinner
Priority: normal Keywords: patch, patch, patch

Created on 2019-02-07 19:15 by giampaolo.rodola, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 11784 merged giampaolo.rodola, 2019-02-07 19:17
PR 11784 merged giampaolo.rodola, 2019-02-07 19:17
PR 11784 merged giampaolo.rodola, 2019-02-07 19:17
Messages (13)
msg335034 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * Date: 2019-02-07 19:15 
The main point of this patch is to automatize all the necessary tasks which are usually involved when creating a server socket, amongst which:

* determining the right family based on address, similarly to socket.create_connection()
* whether to use SO_REUSEADDR depending on the platform
* set AI_PASSIVE flag for getaddrinfo()
* set a more optimal default backlog

This is somewhat related to issue17561 which I prefer to leave pending for now (need to think about it more carefully). issue17561 is complementary to this one so it appears it can be integrated later (or never) without altering the base functionality implemented in here.
msg335286 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * Date: 2019-02-12 04:45 
Closing this out as duplicate. Will continue in issue35934.
msg335534 - (view) Author: Cheryl Sabella (cheryl.sabella) * Date: 2019-02-14 14:20 
The previous comment is a self-reference.  I believe the intention was to mark this as a duplicate of #17561.
msg335535 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * Date: 2019-02-14 14:39 
Correct. Sorry for the typo.
msg339677 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * Date: 2019-04-08 22:34 
New changeset eb7e29f2a9d075accc1ab3faf3612ac44f5e2183 by Giampaolo Rodola in branch 'master':
bpo-35934: Add socket.create_server() utility function (GH-11784)
https://github.com/python/cpython/commit/eb7e29f2a9d075accc1ab3faf3612ac44f5e2183
msg339779 - (view) Author: STINNER Victor (vstinner) * Date: 2019-04-09 16:54 
The change broke multiple buildbots, please fix buildbots or the change will be reverted:
https://github.com/python/cpython/pull/11784#issuecomment-481036369

Revert on fail: https://pythondev.readthedocs.io/ci.html#revert-on-fail
msg339780 - (view) Author: STINNER Victor (vstinner) * Date: 2019-04-09 16:55 
AMD64 Ubuntu Shared 3.x:

https://buildbot.python.org/all/#/builders/141/builds/1498

FAIL: test_wait_socket (test.test_multiprocessing_fork.TestWait)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/_test_multiprocessing.py", line 4377, in test_wait_socket
    self.assertEqual(b''.join(v), expected)
AssertionError: b'1\n2\n3\n4\n5\n6\n7\n8\n9\n' != b'0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n'
msg339782 - (view) Author: STINNER Victor (vstinner) * Date: 2019-04-09 16:57 
AMD64 FreeBSD CURRENT Shared 3.x:

https://buildbot.python.org/all/#/builders/168/builds/885

test_wait (test.test_multiprocessing_spawn.TestWait) ... ok
test_wait_integer (test.test_multiprocessing_spawn.TestWait) ... ok
test_wait_slow (test.test_multiprocessing_spawn.TestWait) ... ok
Process Process-62:
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/multiprocessing/process.py", line 302, in _bootstrap
    self.run()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/_test_multiprocessing.py", line 4337, in _child_test_wait_socket
    s.connect(address)
ConnectionResetError: [Errno 54] Connection reset by peer
/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/multiprocessing/process.py:318: ResourceWarning: unclosed <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 2397)>
  traceback.print_exc()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Process Process-60:
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/multiprocessing/process.py", line 302, in _bootstrap
    self.run()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/_test_multiprocessing.py", line 4337, in _child_test_wait_socket
    s.connect(address)
ConnectionResetError: [Errno 54] Connection reset by peer
/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/multiprocessing/process.py:318: ResourceWarning: unclosed <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 2398)>
  traceback.print_exc()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Process Process-59:
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/multiprocessing/process.py", line 302, in _bootstrap
    self.run()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/_test_multiprocessing.py", line 4337, in _child_test_wait_socket
    s.connect(address)
ConnectionResetError: [Errno 54] Connection reset by peer
/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/multiprocessing/process.py:318: ResourceWarning: unclosed <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 2399)>
  traceback.print_exc()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Timeout (0:25:00)!
Thread 0x0000000800acc000 (most recent call first):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/socket.py", line 212 in accept
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/_test_multiprocessing.py", line 4361 in test_wait_socket
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/case.py", line 680 in run
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/case.py", line 740 in __call__
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/suite.py", line 122 in run
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/suite.py", line 84 in __call__
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/suite.py", line 122 in run
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/suite.py", line 84 in __call__
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/suite.py", line 122 in run
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/suite.py", line 84 in __call__
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/runner.py", line 176 in run
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/support/__init__.py", line 1968 in _run_suite
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/support/__init__.py", line 2064 in run_unittest
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/libregrtest/runtest.py", line 178 in test_runner
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/libregrtest/runtest.py", line 182 in runtest_inner
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/libregrtest/runtest.py", line 137 in runtest
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/libregrtest/main.py", line 305 in rerun_failed_tests
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/libregrtest/main.py", line 623 in _main
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/libregrtest/main.py", line 586 in main
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/libregrtest/main.py", line 640 in main
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/__main__.py", line 2 in <module>
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/runpy.py", line 85 in _run_code
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/runpy.py", line 192 in _run_module_as_main
test_wait_socket (test.test_multiprocessing_spawn.TestWait) ... *** Error code 1
msg339783 - (view) Author: STINNER Victor (vstinner) * Date: 2019-04-09 16:58 
s390x SLES 3.x:
https://buildbot.python.org/all/#/builders/16/builds/2625

FAIL: test_wait_socket (test.test_multiprocessing_fork.TestWait)
FAIL: test_wait_socket (test.test_multiprocessing_spawn.TestWait)
Re-running failed tests in verbose mode
Re-running test 'test_multiprocessing_fork' in verbose mode
FAIL: test_wait_socket (test.test_multiprocessing_fork.TestWait)
Re-running test 'test_multiprocessing_spawn' in verbose mode
msg339784 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * Date: 2019-04-09 17:01 
Those are old failures (+/- 17 hours ago). I committed:
https://github.com/python/cpython/pull/12735
...which fixed them.
msg339787 - (view) Author: STINNER Victor (vstinner) * Date: 2019-04-09 17:19 
> Those are old failures (+/- 17 hours ago). I committed:
> https://github.com/python/cpython/pull/12735

Oh, you wrote "BPO-" rather than "bpo-" and so your commit wasn't mentioned here:

commit 8702b67dad62a9084f6c1823dce10653743667c8
Author: Giampaolo Rodola <g.rodola@gmail.com>
Date:   Tue Apr 9 04:42:06 2019 +0200

    BPO-17561: set create_server backlog default to None (GH-12735)
    
    It turns out doing socket.listen(0) does not equal to "choose a
    reasonable default". It actually means "set backlog to 0".
    As such set backlog=None as the default for socket.create_server.
    Fixes the following BB failures:
    https://github.com/python/cpython/pull/11784#issuecomment-481036369
    Ref. BPO-1756, GH-11784.

> ...which fixed them.

Oh, great! Thanks for the fix.

Sorry, I was 10 days late to handle emails from the buildbot-status mailing list. I only reported failures, I didn't check if it has been fixed in the meanwhile.

I close again the issue.
msg339788 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * Date: 2019-04-09 17:21 
No problem. My bad that I mistyped the "BPO" thing.
msg339792 - (view) Author: STINNER Victor (vstinner) * Date: 2019-04-09 17:32 
I was sure that I already reported the issue... In fact, I commented  bpo-17561 :-)
History
Date User Action Args
2022-04-11 14:59:11adminsetgithub: 80115
2019-04-09 17:32:30vstinnersetkeywords: patch, patch, patch

messages: + msg339792

2019-04-09 17:21:33giampaolo.rodolasetkeywords: patch, patch, patch

messages: + msg339788

2019-04-09 17:19:36vstinnersetstatus: open -> closed

resolution: fixed
messages: + msg339787
keywords: patch, patch, patch

2019-04-09 17:01:11giampaolo.rodolasetkeywords: patch, patch, patch

messages: + msg339784

2019-04-09 16:58:40vstinnersetkeywords: patch, patch, patch

messages: + msg339783

2019-04-09 16:57:03vstinnersetkeywords: patch, patch, patch

messages: + msg339782

2019-04-09 16:55:06vstinnersetkeywords: patch, patch, patch

messages: + msg339780

2019-04-09 16:54:21vstinnersetstatus: closed -> open

nosy: + vstinner, pablogsal
messages: + msg339779

keywords: patch, patch, patch
resolution: duplicate -> (no value)

2019-04-08 22:34:07giampaolo.rodolasetmessages: + msg339677
2019-02-14 14:39:51giampaolo.rodolasetkeywords: patch, patch, patch

messages: + msg335535

2019-02-14 14:20:38cheryl.sabellasetkeywords: patch, patch, patch

messages: + msg335534

2019-02-14 13:46:32vstinnersetkeywords: patch, patch, patch
title: Add socket.bind_socket() utility function -> Add socket.create_server() utility function
2019-02-12 04:45:54giampaolo.rodolasetstatus: open -> closed
messages: + msg335286

keywords: patch, patch, patch
resolution: duplicate
stage: patch review -> resolved

2019-02-07 19:17:38giampaolo.rodolasetkeywords: + patch
pull_requests: + pull_request11771
2019-02-07 19:17:25giampaolo.rodolasetkeywords: + patch
pull_requests: + pull_request11770
2019-02-07 19:17:10giampaolo.rodolasetkeywords: + patch
pull_requests: + pull_request11769
2019-02-07 19:15:02giampaolo.rodolacreate