Re: [BUG/FEATURE] Git pushing and fetching many more objects than strictly required

[Jeff King <peff@peff.net>]

On Fri, Nov 08, 2019 at 09:54:02PM +0100, Paul van Loon wrote:

> >> $ git push -v origin 'refs/replace/*:refs/replace/*'
> >> Pushing to XXXX
> >> Enumerating objects: 2681, done.
> >> Counting objects: 100% (2681/2681), done.
> >> Delta compression using up to 8 threads
> >> Compressing objects: 100% (1965/1965), done.
> >> Writing objects: 100% (2582/2582), 1.96 MiB | 1024 bytes/s, done.
> >> Total 2582 (delta 95), reused 1446 (delta 58)
> >> remote: Resolving deltas: 100% (95/95), completed with 33 local objects.
> >> To XXXX
> >>  * [new branch]            refs/replace/XXXX -> refs/replace/XXXX
> >
> > Could you verify that refs/replace/XXXX (or one of its close ancestors)
> > was fetched by the "git fetch --all" command? "--all" fetches all
> > remotes, not all refs.
> 
> No, it was not fetched. HOWEVER, the ONLY thing the replace commit (1 single object) does is point to an existing parent object. No other new objects are referenced.
> Those 'ancestor' objects were all fetched.

Was it a parent object at the tip of a ref?

The push protocol, unlike the fetch protocol, doesn't expend any effort
to negotiate to find a common base. It just feeds the ref tips of the
receiver to pack-objects (which then does traverse down to a merge base,
but it can't always do so if the sender doesn't have all of the
objects).

It's hard to say more without having a reproducible case to look at.

Some possible things to poke at:

  - record the stdin from the local push to the local pack-objects,
    which shows which objects we're planning to send and which we're
    claiming the other side has. That would help determine if the push
    isn't feeding enough information to pack-objects, or if pack-objects
    isn't trying hard enough to find the minimal set of objects

    There's not really an easy way to do this, but something like strace
    might help.

  - try building reachability bitmaps (e.g., "git repack -adb") in the
    local clone. When those are present, pack-objects will compute the
    object set more thoroughly (because it can do so efficiently).

I don't _think_ the fact that it's in refs/replace should matter to push
(in terms of what it feeds to pack-objects). But obviously another thing
to try is whether pushing to or from a different ref has any impact.

-Peff