Re: [PATCH] RFC: git lazy clone proof-of-concept

["Jon Smirl" <jonsmirl@gmail.com>]

On 2/8/08, Harvey Harrison <harvey.harrison@gmail.com> wrote:
> On Fri, 2008-02-08 at 15:24 -0500, Jon Smirl wrote:
> > On 2/8/08, Harvey Harrison <harvey.harrison@gmail.com> wrote:
> > > On Fri, 2008-02-08 at 14:26 -0500, Jon Smirl wrote:
> > > > On 2/8/08, Jakub Narebski <jnareb@gmail.com> wrote:
> > > > > Jan Holesovsky <kendy@suse.cz> writes:
> > > > > One of the reasons why 'lazy clone' was not implemented was the fact
> > > > > that by using large enough window, and larger than default delta
> > > > > length you can repack "archive pack" (and keep it from trying to
> > > > > repack using .keep files, see git-config(1)) much tighter than with
> > > > > default (time and CPU conserving) options, and much, much tighter than
> > > > > pack which is result of fast-import driven import.
> > > > >
> > > > > Both Mozilla import, and GCC import were packed below 0.5 GB. Warning:
> > > > > you would need machine with large amount of memory to repack it
> > > > > tightly in sensible time!
> > > >
> > > > A lot of memory is 2-4GB. Without this much memory you will trigger
> > > > swapping and the pack process will finish in about a month.
> > >
> > > Well, my modest little Celeron M laptop w/ 1GB of ram did the full
> > > repack overnight on the gcc repo, so a month is a bit of an
> > > exaggeration.
> >
> > Try it again with window=250 and depth=250. That's how you get the
> > really small packs.
> >
>
> Yes, I know, and I did if you remember back to the gcc discussion.

Now that you mention it I seem to recall some changes were made to git
during that discussion that reduced the memory footprint and made the
optimized gcc repack fit into 1GB. I've forgotten the exact timings
and git is a moving target. When I was working on Mozilla it needed
2.4GB to avoid swapping but that was with a much older git.

The rule is: if it starts swapping it is going to take way longer that
you are probably willing to wait. Buying more RAM is a cheap and easy
fix.

If people are having trouble with large repositories please let the
git community know and your issues will probably get quickly fixed.
We can't fix something we don't know about.

-- 
Jon Smirl
jonsmirl@gmail.com