Re: [PATCH] fast-import: implement --min-pack-size parameter

[Jeff King <peff@peff.net>]

On Sat, Apr 23, 2016 at 02:42:25AM +0000, Eric Wong wrote:

> With many incremental imports, small packs become highly
> inefficient due to the need to readdir scan and load many
> indices to locate even a single object.  Frequent repacking and
> consolidation may be prohibitively expensive in terms of disk
> I/O, especially in large repositories where the initial packs
> were aggressively optimized and marked with .keep files.
> 
> In those cases, users may be better served with loose objects
> and relying on "git gc --auto".
> 
> Signed-off-by: Eric Wong <normalperson@yhbt.net>
> ---
>   There should be a matching config file directive, but I'm
>   not sure how/if it should affect other commands.  So I'm
>   not sure if it should be "pack.packSizeMin" or
>   "fastimport.packSizeMin" or something else.

This same concept exists for pushing/fetching, but there we measure it
not in bytes but by the number of objects. Which is probably a better
measure. A single 10MB blob is better as a loose object than as a pack,
but a thousand 10KB blobs should be a pack.

There we have fetch.unpackLimit and receive.unpackLimit for the two
operations, plus transfer.unpackLimit to control both of them. This
doesn't necessarily need to be tied to that config, but you could
certainly consider it in the same boat. It's a way of transferring a
load of objects into the repository.

So it would make some sense to me to have fastimport.unpackLimit,
falling back to transfer.unpackLimit.

-Peff