Issue 28027: Remove Lib/plat-*/* files

After discussion at the core dev sprints, Guido has decreed that the files in Lib/plat-* can and should be removed.

New changeset 14d89359a389 by Zachary Ware in branch 'default':
Issue #28027: Remove Lib/plat-* files
https://hg.python.org/cpython/rev/14d89359a389

Will close this after the buildbots have confirmed they don't hate it.

New changeset e256583e6062 by Zachary Ware in branch 'default':
Issue #28027: Mention the names of the removed modules in Misc/NEWS
https://hg.python.org/cpython/rev/e256583e6062

> Issue #28027: Mention the names of the removed modules in Misc/NEWS

Thanks :-)

None of the buildbots have complained about this (they've been too busy complaining about other things), so I'm closing it.

Yeah! Thanks Zach :-)

YES, finally. Thanks :)

FYI I first proposed to remove plat-* in 2011:
https://mail.python.org/pipermail/python-dev/2011-October/114105.html

That's why I added os.RTDL_* constants:
https://docs.python.org/dev/library/os.html#os.RTLD_LAZY

the removal of these modules should follow the normal deprecation process, for 3.6, then be removed for 3.7.  There is no harm for this approach.

Why? Do you have a dependency on these? None of them are documented.
Do you propose to document them just so we can mark them deprecated?

Since the 3.6 cycle, these modules are regenerated for the build, so they are not outdated anymore.  They are not a maintenance burden either.  So I'd like to see these available in 3.6, and documented as deprecated.  Then removed in 3.7.

Apparently the proposers for this removal don't have the time for an analysis before the beta, I don't have the time either. So pretty please let's revert that change, and instead have the deprecation documented.

No, you need to demonstrate that they are used (despite undocumented).

On Fri, Sep 9, 2016 at 2:26 PM, Matthias Klose <report@bugs.python.org> wrote:
>
> Matthias Klose added the comment:
>
> Since the 3.6 cycle, these modules are regenerated for the build, so they are not outdated anymore.  They are not a maintenance burden either.  So I'd like to see these available in 3.6, and documented as deprecated.  Then removed in 3.7.
>
> Apparently the proposers for this removal don't have the time for an analysis before the beta, I don't have the time either. So pretty please let's revert that change, and instead have the deprecation documented.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue28027>
> _______________________________________

I have been searching sources.debian.net, and have found no evidence of anything that would be broken by this.  Every use of these modules that I have found has been already wrapped in try/except ImportError, or is in code that is fairly obviously not meant for Python 3.

whatever you searched, there are regressions:

https://codesearch.debian.net/search?q=import+DLFCN

some of these have conditional imports, however even boost uses unconditional imports, and is often used as an embedded copy.

Those boost imports will never be triggered in Python 3, because sys.platform will never be 'linux2'.

> https://codesearch.debian.net/search?q=import+DLFCN

I see 8 projects: boost, trilinos, vtk6, openbabel, gdcm, paraview, ecasound, yade. It doesn't like a giant project to update them.


> boost1.58_1.58.0+dfsg-5.1/libs/mpi/build/__init__.py
>
> import sys
> if sys.platform == 'linux2':
>    import DLFCN as dl

It's not like this code already works on Python 3 :-) sys.platform was replace with "linux" on Python 3.

It's easy to replace this import with:

try:
   import DLFCN as dl
except ImportError
   import os as dl

these projects are the ones packaged by Debian; there are surely others.  Now again, why can't the actual removal for 3.7, following the usual deprecation rules?

Because in every release of Python they have been undocumented, in any recent release of Python they have been inaccurate (at best) and probably unnecessary, they have been unmaintained for years, the presence of the plat-* dir on sys.path slows down imports from any directory after it on sys.path, the problem they purport to solve has other much better solutions, and this discussion started 5 years ago with only the the slightest of defenses from MvL.

> Since the 3.6 cycle, these modules are regenerated for the build, so they are not outdated anymore.  They are not a maintenance burden either.

Indeed, they are regenerated by the changes made in issue 23968 ("rename the platform directory from plat-$(MACHDEP) to plat-$(PLATFORM_TRIPLET)"). And they are a maintenance burden as these changes fail to regenerate these modules on Android cross-builds as explained in msg269360, the generation being done with the headers of the build system instead of the target host (Android) system. The problem is not limited to Android cross-compilation of course.

The FreeBSD packager told me that he is *very* happy to see these modules go away, it was also a pain for him to maintain a patch for them. The maintaince burden is even wider than Python source code.

With no solid evidence against the change and much joy with the change, I'm reclosing the issue.  doko, if people complain to you about being broken by this, feel free to refer them to me.

you asked for evidence, you got evidence. re-opening the issue again.  feel free to close it again without any evidence, and stating that you ignore the usual deprecation rules.

Matthias Klose: Maybe you didn't understand something: breaking the backward compatibility was a deliberate choice. It's a matter of maintenance burden versus the number of broken applications.

According to your search, there are only 8 projects in the wild which will break on Python 3.6. Please compare this number to the number of Python modules on PyPI: 88 298 packages, according to https://pypi.python.org/pypi

Moreover, it seems like some of these 8 projects are not compatible with Python 3 yet, so it's not like this specific change broke them...

Finally, I don't like repeating myself, but: fixing these project requires to modify A SINGLE LINE OF CODE. I know well that Debian likes patching packages, it's not like it is something technically impossible.

Please don't reopen the issue again without a very strong argument.

"It's a matter of maintenance burden". No I don't think so.

As agreed on IRC with zware, I'll restore the removed modules, including a deprecation notice.

"Agreed" is a bit strong, I still don't agree that the change was bad or should be reverted.  I'm just not ruling out the possibility of a partial (because there is absolutely no reason to re-add the files for any platform but Linux and *maybe* OSX) reversion if you put forth the effort to do it after #28046 and add deprecation notice.

Sorry, I'm leaving this discussion now.

The modules should remain deleted for 3.6b1 so we can see if there are
actually any complaints -- if we get any we can consider restoring
them for b2. I fail to be convinced by the arguments against deletion
brought up so far -- the "evidence" produced is way too thin to
matter.

Since we've reached 3.6.3 with no complaints that have made it to me, I'm going to go ahead and close the issue.

Thank you Zachary for this removal ;-)

nosy: + doko
messages: + msg275443

resolution: fixed -> (no value)

stage: resolved