Re: [PATCH] RFC: git lazy clone proof-of-concept

[Johannes Schindelin <Johannes.Schindelin@gmx.de>]

Hi,

On Sun, 10 Feb 2008, David Symonds wrote:

> On Feb 10, 2008 4:08 AM, Johannes Schindelin 
> <Johannes.Schindelin@gmx.de> wrote:
>
> > On Sun, 10 Feb 2008, Marco Costalba wrote:
> >
> > > Linux git repository is not very big and can be downloaded with 
> > > easy. On the other end Linux history spans many more years then the 
> > > repo does.
> > >
> > > The design choice here is two have *two repositories*, one with 
> > > recent stuff and one historical, with stuff older then version 
> > > 2.6.12
> >
> > I do not think that this is an option: Jan already tried a shallow 
> > clone (which would amount to something like what you propose), and it 
> > was still too large.
> 
> I think that was still pulling all the branches, so a shallow clone of 
> just a couple of branches might be feasible.

Indeed:

$ git ls-remote git://o3-build.services.openoffice.org/git/ooo.git|wc -l
3970
$ git ls-remote --heads git://o3-build.services.openoffice.org/git/ooo.git|
	wc -l
751

Fetching just master is a little hard on the server (it spends quite a 
lot of time deltifying -- minutes! -- especially between 80% and 95%, 
and indexing is even slower), but other than 
that:

$ /usr/bin/time git fetch --depth=1 \
	git://o3-build.services.openoffice.org/git/ooo.git \
	master:refs/remotes/origin/master
warning: no common commits
remote: Generating pack...
remote: Done counting 79934 objects.
remote: Deltifying 79934 objects...
remote:  100% (79934/79934) done
Indexing 79934 objects...
remote: Total 79934 (delta 34549), reused 51323 (delta 20737)
 100% (79934/79934) done
Resolving 34549 deltas...
 100% (34549/34549) done
* refs/remotes/origin/master: storing branch 'master' of 
git://o3-build.services.openoffice.org/git/ooo
  commit: 29990e4
46.48user 4.60system 16:48.29elapsed 5%CPU (0avgtext+0avgdata 
0maxresident)k
0inputs+0outputs (0major+941205minor)pagefaults 0swaps

$ du .git/objects/pack/
464688  .git/objects/pack/
$ /usr/bin/time git repack -a -d -f --window=250 --depth=250
Generating pack...
Done counting 79934 objects.
Deltifying 79934 objects...
 100% (79934/79934) done
Writing 79934 objects...
 100% (79934/79934) done
Total 79934 (delta 40013), reused 0 (delta 0)
Pack pack-350e4edca93ee75ef3d85269284a24775bf6b24f created.
Removing unused objects 100%...
Done.
1869.78user 6.66system 31:36.50elapsed 98%CPU (0avgtext+0avgdata 
0maxresident)k
0inputs+0outputs (2031major+1753824minor)pagefaults 0swaps
$ du .git/objects/pack/
454636  .git/objects/pack/

Of course, the clone time would be reduced dramatically if the repository 
you clone from has only "master", and is fully (re-)packed.

So I was not completely correct in my assumption that a clear cut a la 
linux-2.6 (possibly grafting historical-linux) would not help.

Ciao,
Dscho