Re: Race condition between repack and loose-objects maintenance task

[Taylor Blau <me@ttaylorr.com>]

Hi Greg,

On Wed, Jun 29, 2022 at 09:55:54AM -0700, Gregory Szorc wrote:
> 1. `git repack -A` creates loose objects
> 2. `git maintenance`'s `loose-objects` task deletes those loose objects
> 3. `git repack -A` fails to find the loose objects it just created and
> aborts with `fatal: unable to add recent objects`

This is a somewhat well-known race that occurs when one Git process
decides unreachable objects are safe to be deleted, but an incoming push
or reference update makes those to-be-deleted objects reachable before
they are actually removed, leaving the repository in a corrupt state.

I'm surprised that the loose-objects maintenance task deletes those
objects, though, since it just runs `prune-packed` or (the equivalent
of) `repack -d`, neither of which will actually delete objects from the
repository.

I see that Stolee is already on the CC list, so perhaps he could chime
in on the above.

In either case, my recommendation would be to keep those unreachable
objects which haven't yet aged out of the repository around for longer,
which will decrease the likelihood of seeing the race. If your
repository has a lot of unreachable objects (such that storing each
one of them individually as loose is impractical), I would recommend
using cruft packs (e.g., by running either `git repack --cruft -d` or
`git gc --cruft`) to collect those unreachable objects together into a
single pack.

See Documentation/technical/cruft-packs.txt for more information about
cruft packs.

Thanks,
Taylor