Issue 35772: test_tarfile fails on ppc64le when using tmpfs filesystem

The following test_tarfile tests fail on ppc64 when using tmpfs filesystem (which is the case on RHEL package build server):

* test_sparse_file_00 (test.test_tarfile.GNUReadTest)
* test_sparse_file_01 (test.test_tarfile.GNUReadTest)
* test_sparse_file_10 (test.test_tarfile.GNUReadTest)
* test_sparse_file_old (test.test_tarfile.GNUReadTest)

Example of failure:

======================================================================
FAIL: test_sparse_file_00 (test.test_tarfile.GNUReadTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.6.6/Lib/test/test_tarfile.py", line 964, in test_sparse_file_00
    self._test_sparse_file("gnu/sparse-0.0")
  File "/builddir/build/BUILD/Python-3.6.6/Lib/test/test_tarfile.py", line 958, in _test_sparse_file
    self.assertLess(s.st_blocks * 512, s.st_size)
AssertionError: 131072 not less than 86016

Bug first report on RHEL8:
https://bugzilla.redhat.com/show_bug.cgi?id=1639490


test_tarfile has _fs_supports_holes() function to check if the filesystem supports sparse files with holes. The function returns True on:

* ext4 filesystem on x86_64 on my Fedora 29 (kernel 4.19)
* ext4 filesystem on x86_64 on my Fedora 29 (kernel 4.19)
* XFS filesystem on ppc64le (kernel 4.18)
* tmpfs filesystem on ppc64le (kernel 4.18)

In short, it always return True on x86_64 and ppc64le Linux kernels.

Problem: in practice, "tmpfs filesystem on ppc64le (kernel 4.18)" doesn't fully support sparse files.

--

Example from:
https://bugzilla.redhat.com/show_bug.cgi?id=1639490#c5

# ls -lhs ~/sparse 
48K -rw-r--r--. 1 root root 84K Jan 18 05:36 /root/sparse

Copy a sparse file from XFS to tmpfs: cp --sparse=always and fallocate --dig fail to punch holes, the file always take 128K on disk on tmpfs.

# cp sparse /root/mytmp/sparse --sparse=always
# ls -lhs /root/mytmp/sparse 
128K -rw-r--r--. 1 root root 84K Jan 18 06:10 /root/mytmp/sparse
# fallocate --dig /root/mytmp/sparse
# ls -lhs /root/mytmp/sparse 
128K -rw-r--r--. 1 root root 84K Jan 18 06:10 /root/mytmp/sparse

Counter example on XFS, source and destionation files use 48K on disk fo 84K of data:

# cp sparse sparse2 --sparse=always
# ls -lhs sparse*
48K -rw-r--r--. 1 root root 84K Jan 18 05:36 sparse
48K -rw-r--r--. 1 root root 84K Jan 18 06:13 sparse2

--

Attached PR fix the _fs_support_holes() heuristic to return properly False on tmpfs on ppc64le.

Oops, I forgot to attached my test script: create_sparse.py

offsets = ( 4096, 12288, 20480, 28672, 36864, 45056, 53248, 61440, 69632,
        77824,)

These offsets are less than 64 KiB apart.  On systems with a 64 KiB page size, this will not result in a sparse file on tmpfs because the effective block size is the page size (tmpfs lives in the page cache).

Red Hat Enterprise Linux uses 64 KiB pages on aarch64, ppc64, ppc64le.  Only s390x and x86_64 use 4 KiB pages.

Ah yes, I confirm that ppc64le uses 64 KiB page size:

# python3
Python 3.6.8 (default, Jan 11 2019, 01:44:37) 
[GCC 8.2.1 20180905 (Red Hat 8.2.1-3)] on linux
>>> import resource
>>> resource.getpagesize()
65536

whereas my x86_64 laptop uses 4 KiB page size:

$ python3
Python 3.7.2 (default, Jan  3 2019, 09:14:01) 
[GCC 8.2.1 20181215 (Red Hat 8.2.1-6)] on linux
>>> import resource
>>> resource.getpagesize()
4096

I updated PR 11606 to mention the link between tmpfs and the page size.

New changeset b2385458ceddaf3d0d91456923716259d3915023 by Victor Stinner in branch 'master':
bpo-35772: Fix test_tarfile on ppc64 (GH-11606)
https://github.com/python/cpython/commit/b2385458ceddaf3d0d91456923716259d3915023

New changeset d1dd6be613381b996b9071443ef081de8e5f3aff by Miss Islington (bot) in branch '3.7':
bpo-35772: Fix test_tarfile on ppc64 (GH-11606)
https://github.com/python/cpython/commit/d1dd6be613381b996b9071443ef081de8e5f3aff

messages: + msg333973

messages: + msg333972