Issue 32539: os.listdir(...) on deep path on windows in python2.7 fails with errno 123

On windows, a deep path can be accessed by prefixing the path with \\?\

https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx?f=255&MSPPError=-2147217396#maxpath

The call to `listdir()` fails because it uses a posix separator.

A quick patch to fix this is to use `SEP` here: https://github.com/python/cpython/blob/ab95b3074ee43098edf3f23b07fb18ef57ee614d/Modules/posixmodule.c#L2388 (I can submit a patch for this)

Here's a stacktrace

>>> os.listdir(r'\\?\C:\Temp')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
WindowsError: [Error 123] The filename, directory name, or volume label syntax is incorrect: '\\\\?\\C:\\Temp/*.*'


This works fine in python3, as the code has been refactored to use `SEP` already.

>>> os.listdir(r'\\?\C:\Temp')
[]

This should be fixed. That said, we have to use a unicode string for a long path anyway. Prior to Windows 8, the conversion from ANSI to Unicode in the system runtime library uses a static MAX_PATH buffer, so the ANSI API is inherently limited to MAX_PATH. You'll see this in the documentation of all functions that allow using \\?\ extended paths:

    In the ANSI version of this function, the name is limited to 
    MAX_PATH characters. To extend this limit to 32,767 wide 
    characters, call the Unicode version of the function and 
    prepend "\\?\" to the path.

Decoding ANSI strings uses a dynamically-sized buffer in Windows 8, but the change is undocumented and should not be relied upon.

Unfortunately, we can't reliably use a raw unicode string literal in Python 2, since \u and \U escapes are still evaluated. We can instead use forward slashes and normalize via os.path.normpath.

New changeset 27f32e938ff51fd5d90a29abbbabc6b81d156f33 by Serhiy Storchaka (Anthony Sottile) in branch '2.7':
bpo-32539: Fix OSError for os.listdir() for extended-length paths on Windows (#5169)
https://github.com/python/cpython/commit/27f32e938ff51fd5d90a29abbbabc6b81d156f33

Thank you Anthony for your patch!

stage: patch review -> resolved

messages: + msg309866
stage: patch review -> (no value)