Re: bug: git-archive does not use the zip64 extension for archives with more than 16k entries

[Johannes Schauer <josch@debian.org>]

[-- Attachment #1: Type: text/plain, Size: 2435 bytes --]

Hi,

Quoting René Scharfe (2015-08-12 21:40:48)
> Am 11.08.2015 um 12:40 schrieb Johannes Schauer:
> > for repositories with more than 16k files and folders, git-archive will create
> > zip files which store the wrong number of entries. That is, it stores the
> > number of entries modulo 16k. This will break unpackers that do not include
> > code to support this brokenness.
> 
> The limit is rather 65535 entries, isn't it?  The entries field has two 
> bytes, and they are used fully.

seems to be 65535 indeed.

I just forwarded the number Dieter Baron (libzip contributor) told me when they
replied (off list) to my bug report against libzip:
http://nih.at/listarchive/libzip-discuss/msg00554.html

But reading https://en.wikipedia.org/wiki/Zip_(file_format)#ZIP64 the limit
indeed seems to be 65535.

> Which programs are affected? InfoZIP Zip 3.0 and 7-Zip 9.20 seem to handle an
> archive with more entries just fine.  The built-in functionality of Windows
> 10 doesn't.

In my case I discovered this because libzip http://nih.at/libzip does not
implement reading an archive with more than 65535 entries without zip64.

> Besides, 64K entries should be enough for anybody. ;-)

:P

> Seriously, though: What kind of repository has that many files and uses the
> ZIP format to distribute snapshots?  Just curious.

I have not searched for any.

In my case I was using git to keep track of the modifications our tools do to a
directory of files to detect regressions. These tools are also able to read
data from zip archives instead from a directory. I created the zip archive
using git-archive because the files already were in git so that seemed most
convenient to me. That's when I discovered the problem because our tools use
libzip. The easy workaround was to use another packager instead of git-archive.
We use the zip format because Windows has support for it.

> > Instead, git-archive should use the zip64 extension to handle more than 16k
> > files and folders correctly.
> 
> That seems to be what InfoZIP does and Windows 10 handles it just fine. If
> lower Windows versions and other popular extractors can unzip such archives
> as well then this might indeed be the way to go.

The wikipedia page above claims that windows versions starting with vista have
support for zip64. It also lists some other software with support for it.

Thanks!

cheers, josch

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAABCAAGBQJVzAApAAoJEPLLpcePvYPhLUEP/i3r+wbhhHBrapMjLLeIupJy
4I7fItTXxzSWyX8g8MH1RLdd/ckKJ0zsMQZTMtSbKwjEatE56ChvEvm2PIadX8Qg
DDn51mTFABDpPaItl8IxSmMVDzHePnkzTnFRbZzFC/jaCgm2Rb4AGCr9GTTvvlNi
37XQ/YyE2af2WiGtn2TLGO8ZamHaxoov4n9DYBEcgi9yUVg3CUvL5+Reb7spf0sI
PEOMkeuLmaBKDQsb4N4IPz6c0vTGgvfe9rmD4dtUKuB0R/7S2Tz366IM1zuypW3y
Q3hg53Pq2OaXtB5ukYYQ3mi5/jEyUIciF8mklbXfTx+vaaPoFXUwEgjJde+49WOl
lHfIpqk1w2WqYyzgwgf328of7lUeEh3ABhFdpAdNPOYf2amSUAy+xl614cu4Y/ld
wcoKn28H2pspfiux/SPljVbo9xhQbkd2Po8p0J2lLVhwXqjJv5MLBITPUSH0sZRL
MuI+1wNpOiQaIZsaxyiSmNUpMVdmeb2ZHUw38jkKCcpvYhtfE3ItCDs8R28s6bP8
yUEWXVrZk1hKmID1plLkCPAc5sHSZJdA7n66xv8ZdTyWHbykto45FPY6Kt0Lc6/q
B+riiP6rEuX6TeUnOvHc3lQ49lwGXfBrRFJbYvY/XQs99izD/Xp9o0SnJ3p9InC4
xPYEMc1IsRSxfssr0wNa
=6mn4
-----END PGP SIGNATURE-----