Issue 36993: zipfile: tuple IndexError on extract

The following code throws an IndexError when attempting to extract a malformed archive (attached):

import zipfile
import sys

zf = zipfile.ZipFile(sys.argv[1])
for info in zf.infolist():
  zf.extract(info.filename)

Result:
Traceback (most recent call last):
  File "code.py", line 4, in <module>
    zf = zipfile.ZipFile(sys.argv[1])
  File "/usr/local/lib/python3.8/zipfile.py", line 1230, in __init__
    self._RealGetContents()
  File "/usr/local/lib/python3.8/zipfile.py", line 1353, in _RealGetContents
    x._decodeExtra()
  File "/usr/local/lib/python3.8/zipfile.py", line 480, in _decodeExtra
    self.file_size = counts[idx]
IndexError: tuple index out of range

unzip index_tuple.zip -x
Archive:  index_tuple.zip

caution:  zipfile comment truncated
error [index_tuple.zip]:  missing 3992977728 bytes in zipfile
  (attempting to process anyway)
   skipping: zipfile_extract/        unsupported compression method 211


I think the issue is not with Python but with your ZIP file. Did you try to uncompress it with unzip?\

Thank you

This report is valid. Serhiy has improved error reporting of the extra field in feccdb2a249a71be330765be77dee57121866779.

counts can indeed be an empty tuple:

    elif ln == 0:
        counts = ()

If I'm reading section 4.5.3 of https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT correctly, I think we need to raise BadZipFile if ln == 0.

It is not enough. IndexError can be raised for ln == 8 or 16 when file_size, compress_size and header_offset are all set to 0xffffffff.

@alter-bug-tracer, could you please create test files for the cases Serhiy has just mentioned?

@berker.peksag, first of all sorry for the late reply. 
We are not sure that we know how to do that. Our tests are generated automatically. What we can do is retest the lib with your temporary fixes in place, to see if they fix all the problems our software can detect. Would that help you?

I've pushed a PR which adds a test that generates corrupt zip64 files with different combinations of zip64 extra data lengths and zip64 flags (which determines how many fields are required in the extra data).

It now raises a BadZipFile with a message naming the first missing field.

New changeset da6ce58dd5ac109485af45878fca6bfd265b43e9 by Serhiy Storchaka (Daniel Hillier) in branch 'master':
bpo-36993: Improve error reporting for zipfiles with bad zip64 extra data. (GH-14656)
https://github.com/python/cpython/commit/da6ce58dd5ac109485af45878fca6bfd265b43e9

New changeset f7d50f8f997fbfce1556991a3700826536871fe7 by Miss Skeleton (bot) in branch '3.7':
bpo-36993: Improve error reporting for zipfiles with bad zip64 extra data. (GH-14656)
https://github.com/python/cpython/commit/f7d50f8f997fbfce1556991a3700826536871fe7

New changeset 3801b2699eb9441ca31c6ec8fa956fc0fe755ef7 by Miss Skeleton (bot) in branch '3.8':
bpo-36993: Improve error reporting for zipfiles with bad zip64 extra data. (GH-14656)
https://github.com/python/cpython/commit/3801b2699eb9441ca31c6ec8fa956fc0fe755ef7

Thank you for your contribution Daniel.

resolution: fixed
stage: patch review -> resolved

type: behavior
components: + Library (Lib)
versions: - Python 3.6
nosy: + berker.peksag, serhiy.storchaka

messages: + msg344181
resolution: not a bug -> (no value)
stage: resolved -> needs patch

nosy: + matrixise
messages: + msg343152

resolution: not a bug
stage: resolved