Re: There should have be git gc --repack-arguments

[Jeff King <peff@peff.net>]

On Wed, Apr 07, 2021 at 01:40:16PM -0700, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> >> ... git repack ...  --max-pack-size=<desired pack size> to create split and
> >> smaller packs instead.
> > ...
> > You can also set pack.packSizeLimit for the latter, though I do not
> > recommend it. It will not help with memory usage (neither while
> > repacking nor for later commands).
> 
> In other words, passing --max-pack-size, whether it is done with a
> new --repack-arguments option or it is done with the existing
> pack.packSizeLimit configuration, would make things worse.

Right. I wish we didn't have --max-pack-size at all. I do not think it
is ever a good idea, and it complicates the packing code quite a bit.

These days we have index v2 to let us address more than 4GB in a
packfile. I suppose it's possible you could have a filesystem whose max
file size is smaller than your total packfile, but that seems pretty
unlikely these days (even 32-bit systems tend to have large file
support).

But that's all a tangent. :)

> So in conclusion:
> 
>  - attempting to repack everything into one pack on a memory starved
>    box would be helped with reduced window memory size.

Yes, though less than you might think. It is only trying to keep the
memory used by delta compression at bay. The per-object book-keeping
tends to be quite high by itself. If you are under memory pressure
during delta compression, you may also be better off reducing the number
of threads (since each thread is simultaneously using windowMemory
bytes).

>  - on a small box, it may make sense to avoid repacking everything
>    into one in the first place, but we do not want the number of
>    packs to grow unbounded.
> 
> Would the new geometric repack feature help here, especially for the
> latter?

Yes, I think it would. You'd perhaps want to generate a multi-pack-index
file, too, to avoid having to look for objects in multiple packs
sequentially (we have a "git repack --write-midx" option on the way, as
well).

-Peff