Re: There should have be git gc --repack-arguments

[Jeff King <peff@peff.net>]

On Wed, Apr 07, 2021 at 07:10:43PM +0700, Bagas Sanjaya wrote:

> I request that git gc should have --repack-arguments option. The value
> of this option should be passed to git repack.

I think in general we prefer to make individual options configurable,
rather than having a blanket "pass along these options" argument, for
two reasons:

  - some options may cause the sub-program to behave unexpectedly. E.g.,
    if you put "-a" in the repack-arguments, that may be subverting
    git-gc's assumptions about how repack will behave

  - arguments are a list, not a string. So you have to provide some
    mechanism for splitting them (presumably on whitespace, but what if
    we need quoting)?

> The use case is when I have very large repos (such as GCC and Linux kernel)
> on a server with small RAM (1-2 GB). When doing gc on such repo, the repack
> step may hang because git-repack have to create single large packfile which
> can be larger than available memory (RAM+swap), so it must be necessary to
> do git repack --window-memory=<desired memory usage> --max-pack-size=<desired
> pack size> to create split and smaller packs instead.
> 
> There should also git config item gc.repackArguments, which have the same
> effect as git gc --repack-arguments, with the option takes precedence over
> the config.

You can set pack.windowMemory in your config already, to solve the first
part.

You can also set pack.packSizeLimit for the latter, though I do not
recommend it. It will not help with memory usage (neither while
repacking nor for later commands). We do mmap() the resulting packfiles,
but we rely on the operating system to manage the actual in-RAM working
set (but that is also true with multiple packfiles; we are happy to map
several of them at once). And it may make your on-disk size much larger.
We don't allow deltas between on-disk packs, which means some objects
which could be stored as deltas won't be. That in turn hurts on a
memory-starved system because we'll need more block cache to perform the
same task. It also results in extra CPU when serving fetches or pushing,
since we'll try to find new deltas between the packs on the fly.

-Peff