Issue 34011: Default preference not given to venv DLL's

I don't know if this is a bug or an odd design decision or just something that hasn't been considered or maybe I'm missing something.

On Windows and I create a venv with Python 3.6.3:

> python -m venv venv

This creates a subdirectory called /venv. Inside this, there's:
./venv/Scripts/sqlite3.dll

This is the sqlite library - except it's not, because Python isn't using this file. If I upgrade this library by replacing it with a newer sqlite3.dll version, Python keep using the original version of the library. Turns out, Python is by default reading the DLL in the root Python install:

c:\Python36\DLLs\sqlite3.dll

Now, I can change that file and sure enough my VENV (*all* VENVs) then get the newer version of SQLite, or I can delete that file and the VENV's will all use their local versions, or I can possibly play with some sys.path to try and get the VENV version loaded first.

But this raises a few questions:
1) This seems like a rather odd default - copying a file that is never used by default.
2) Surely either the correct option is to use the VENV version by default?
3) Otherwise, what's the point in copying across this DLL file into the VENV by default?

Offhand I don't know why it copies PYD and DLL files from the DLLs directory. It's part of the standard library. The virtual environment should only need to copy or symlink the binaries in the application directory, such as python.exe, pythonw.exe, vcruntime<###>.dll, python<##>.dll, and python<#>.dll. Currently a virtual environment doesn't use the binaries that get copied or symlinked from the DLLs directory, since the system DLLs directory is in sys.path.

Also, why does it copy over init.tcl? _tkinter uses Py_GetPrefix() to to get the real prefix directory in order to find the TCL library at "tcl\tclX.y". For a Python 3.7 virtual environment, I verified using Sysinternals procmon that TCL loads the init file from "C:\Program Files\Python37\tcl\tcl8.6\init.tcl", not the virtual environment.

The code that does the copy is the original PEP 405 implementation code (from 26 May 2012). I'm fairly sure that these files were copied over only because (at least prior to the Python 3.3 release in September 2012) they were needed to be in the venv for the venv to work correctly. Python internals may have changed subsequently, making these copies unnecessary.

I can't easily recreate a Python 3.3.0 on Windows to confirm this, but I'm pretty sure I only coded it to copy over what was actually needed at the time.

Hmmm. I managed to find a machine to install Python3.3.0 on, and it appears that even with that copy suite removed, the tests still work. So possibly it's something that changed between 3.3.0 alpha and 3.3.0 final.

I can try removing the copy code for 3.8 but I'm not sure if any code out there is relying on the copied files being in the venv, so it may not be a good idea to backport.

New changeset 94487d45707772723ef19e86700a40a12743baa1 by Vinay Sajip in branch 'master':
bpo-34011: Update code copying DLLs and init.tcl into venvs. (GH-8253)
https://github.com/python/cpython/commit/94487d45707772723ef19e86700a40a12743baa1

This change breaks a number of tests when run on a proper installation, since we still need to copy pythonXY.dll or else python.exe refuses to start.

Since I'm fixing these tests today, I'll also fix this issue.

New changeset f14c28f39766855420dd58d209da4ad847f3030e by Steve Dower in branch 'master':
bpo-34011: Fixes missing venv files and other tests (GH-9458)
https://github.com/python/cpython/commit/f14c28f39766855420dd58d209da4ad847f3030e

I agree with this only applying to 3.8 - removing the copied files on earlier versions seems like an unnecessary risk to compatibility.

This buildbot is failing when raising when creating subprocess.CalledProcessError:

https://buildbot.python.org/all/#/builders/40/builds/1135/steps/3/logs/stdio

test_defaults (test.test_venv.BasicTest) ... ok
test_executable (test.test_venv.BasicTest) ... ok
test_executable_symlinks (test.test_venv.BasicTest) ... skipped 'Needs symlinks'
test_isolation (test.test_venv.BasicTest) ... ok
test_overwrite_existing (test.test_venv.BasicTest) ... ok
test_prefixes (test.test_venv.BasicTest) ... ok
test_prompt (test.test_venv.BasicTest) ... ok
test_symlinking (test.test_venv.BasicTest) ... skipped 'Needs symlinks'
test_unicode_in_batch_file (test.test_venv.BasicTest) ... ERROR
test_unoverwritable_fails (test.test_venv.BasicTest) ... ok
test_upgrade (test.test_venv.BasicTest) ... ok
test_devnull (test.test_venv.EnsurePipTest) ... ok
test_explicit_no_pip (test.test_venv.EnsurePipTest) ... ok
test_no_pip_by_default (test.test_venv.EnsurePipTest) ... ok
test_with_pip (test.test_venv.EnsurePipTest) ... ok
======================================================================
ERROR: test_unicode_in_batch_file (test.test_venv.BasicTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_venv.py", line 302, in test_unicode_in_batch_file
    out, err = check_output(
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_venv.py", line 37, in check_output
    raise subprocess.CalledProcessError(
TypeError: __init__() takes from 3 to 5 positional arguments but 6 were given
----------------------------------------------------------------------
I will prepare a PR fixing this

resolution: fixed
stage: patch review -> resolved

components: + Library (Lib), Windows
type: behavior
stage: needs patch