Issue 21491: race condition in SocketServer.py ForkingMixIn collect_children

collect_children routine in SocketServer.py contains two possible race conditions. First one is in while loop "while len(self.active_children) >= self.max_children:". If status of child is collected outside of Socket server (for example in signal handler or something), then the variable self.active_children will not match actual child processes and the os.waitpid(0, 0) within the while loop will raise os.error, errno=10 (ECHILD) "No Child Processes". self.active_children should be emptied in this case, otherwise you'll end up with an endless loop comsuming 100% CPU (as happened to us).

The second possible race condition is below in the collect_children routine in the "for child in self.active_children" which contains a statement self.active_children.remove(pid) which would modify the iterator. I do not now about python 2.7, but before this would result in "incorrect iteration".
Original code:
        for child in self.active_children:
            try:
                pid, status = os.waitpid(child, os.WNOHANG)
            except os.error:
                pid = None
            if not pid: continue
            try:
                self.active_children.remove(pid)
            except ValueError, e:
                raise ValueError('%s. x=%d and list=%r' % (e.message, pid, self.active_children))

Fixed code:
        to_remove = []
        for child in self.active_children:
            try:
                pid, status = os.waitpid(child, os.WNOHANG)
            except os.error:
                pid = None
            if not pid: continue
            to_remove.append(pid)

        for pid in to_remove:
            try:
                self.active_children.remove(pid)
            except ValueError, e:
                raise ValueError('%s. x=%d and list=%r' % (e.message, pid, self.active_children))

Here's a patch fixing both issues.

New changeset aa5e3f7a5501 by Charles-François Natali in branch '2.7':
Issue #21491: SocketServer: Fix a race condition in child processes reaping.
http://hg.python.org/cpython/rev/aa5e3f7a5501

New changeset 2a7375bd09f9 by Charles-François Natali in branch '3.4':
Issue #21491: socketserver: Fix a race condition in child processes reaping.
http://hg.python.org/cpython/rev/2a7375bd09f9

New changeset ae0b572ced20 by Charles-François Natali in branch 'default':
Issue #21491: socketserver: Fix a race condition in child processes reaping.
http://hg.python.org/cpython/rev/ae0b572ced20

Committed, thanks!

resolution: fixed
stage: patch review -> resolved

nosy: + pitrou, vstinner
messages: + msg219930

keywords: + patch, needs review
stage: patch review