Re: Something is broken in repack

[Nicolas Pitre <nico@cam.org>]

On Fri, 7 Dec 2007, Jon Smirl wrote:

> Using this config:
> [pack]
>         threads = 4
>         deltacachesize = 256M
>         deltacachelimit = 0
> 
> And the 330MB gcc pack for input
>  git repack -a -d -f  --depth=250 --window=250
> 
> complete seconds RAM
> 10%  47 1GB
> 20%  29 1Gb
> 30%  24 1Gb
> 40%  18 1GB
> 50%  110 1.2GB
> 60%  85 1.4GB
> 70%  195 1.5GB
> 80%  186 2.5GB
> 90%  489 3.8GB
> 95%  800 4.8GB
> I killed it because it started swapping
> 
> The mmaps are only about 400MB in this case.
> At the end the git process had 4.4GB of physical RAM allocated.
> 
> Starting from a highly compressed pack greatly aggravates the problem.
> Starting with a 2GB pack of the same data my process size only grew to
> 3GB with 2GB of mmaps.

You said having reproduced the issue, albeit not as severe, with the 
Linux kernel repo.  I did just that:

# to get the default pack:
$ git repack -a -f -d

# first measurement with a repack from a default pack
$ /usr/bin/time git repack -a -f --window=256 --depth=256
2572.17user 5.87system 22:46.80elapsed 188%CPU (0avgtext+0avgdata 0maxresident)k
15720inputs+356640outputs (71major+264376minor)pagefaults 0swaps

# do it again to start from a highly packed pack
$ /usr/bin/time git repack -a -f --window=256 --depth=256
2573.53user 5.62system 22:45.60elapsed 188%CPU (0avgtext+0avgdata 0maxresident)k
29176inputs+356664outputs (210major+274887minor)pagefaults 0swaps

This is with pack.threads=2 on a P4 with HT, and I'm using the machine 
for other tasks as well, but all measured time is sensibly the same for 
both cases.  Virtual memory allocation never reached 700MB in both cases 
either.


Nicolas