Git bundle create produces inconsistent bundle

Git bundle create produces inconsistent bundle

[Andre Loconte]

Hello,

I am running into an issue where "git bundle create" doesn't always produce the same bundle, provided the exact same arguments and the exact same repo.

Where it gets tricky is that:

- I cannot reproduce the issue with random repos from Github (tried with ~10 repos and 30 bundles per repo);
- I cannot reproduce the issue with the same OS and git version on different hardware ;
- I can reproduce the issue on colleagues' workstation (environment is identical down to the hardware, see below).

Environment:

- GNU/Linux 5.3.0-24-generic Ubuntu 19.10 x86_64 ;
- git version 2.20.1 ;
- Intel Xeon Silver 4114 CPU @ 2.20GHz × 20 ;
- 4 × Hynix HMA81GR7CJR8N-VK 8GB DDR4-2666 ECC.

I have put together a bash PoC [1]. And there are some example output in the comments (consistent [2], inconsistent [3]).

[1] https://gist.github.com/alct/05cc9a2b4657d51669c96cb22cd5c4a6
[2] https://gist.github.com/alct/05cc9a2b4657d51669c96cb22cd5c4a6#gistcomment-3130206
[3] https://gist.github.com/alct/05cc9a2b4657d51669c96cb22cd5c4a6#gistcomment-3130213

I cannot share the affected repos but I can run any diagnostic tool against them.

Would you have any clue on what could produce such behavior?

Best,

A.L.

Re: Git bundle create produces inconsistent bundle

[Jeff King]

On Tue, Jan 07, 2020 at 05:44:47PM +0000, Andre Loconte wrote:

> I am running into an issue where "git bundle create" doesn't always
> produce the same bundle, provided the exact same arguments and the
> exact same repo.

This is expected.  A bundle contains a packfile, and the packfile
generation is sensitive to several things:

  - the delta search is threaded (to the same number of threads as you
    have CPUs). Try "git -c pack.threads=1 bundle create ...".

  - we try to reuse on-disk deltas when possible. So the exact set of
    packs you have (or which objects are loose) may impact the result
    slightly. So between two different clones of the same repository,
    I'd expect different results.

Possibly repacking first with "git -c pack.threads=1 repack -adf" might
get you an identical on-disk state in two different clones (assuming you
have the same refs and reflogs in each!).

But as you can see, we don't in general expect packfiles to be
byte-stable (this is also the reason that git clones aren't easily
resumable).

-Peff