Re: Index/hash order

[Linus Torvalds <torvalds@osdl.org>]

On Wed, 13 Apr 2005, Florian Weimer wrote:
> 
> I've run a few tests, just to get a few numbers of the overhead
> involved.  I used the last ~8,000 changesets from the BKCVS kernel
> repository.  With cold cache, a checkout from cold cache takes about
> 250 seconds on my laptop.  I don't have git numbers, but a mere copy
> of the kernel tree needs 40 seconds.

I will bet you that a git checkout is _faster_ than a kernel source tree
copy. The time will be dominated by the IO costs (in particular the read
costs), and the IO costs are lower thanks to compression. So I think that
the cold-cache case will beat your 40 seconds by a clear margin. It
generally compresses to half the size, so 20 seconds is not impossible
(although seek costs would tend to stay constant, so I'd expect it to be
somewhere in between the two).

> For the hot-cache case, the difference is 140 seconds vs. 2.5 seconds
> (or 6 seconds with decompression).
> 
> Uh-oh.  I wouldn't have imaged the difference would be *that*
> dramatic.  The file system layer is *fast*.

Did I mention that I designed git for speed?

Yes. The whole damn design is really about performance, distribution, and 
built-in integrity checking. 

> On the storage front, we have 220 MB for the skip deltas vs. 106 MB
> for pure deltas-to-previous vs. 1.1 GB for uncompressed files
> (directories are always delta-compressed, so to speak[1]).

That's actually pretty encouraging. Your 1.1GB number implies to me that a
compressed file setup should be about half that, which in turn says that
the cost of full-file is not at all outrageous. Sure, it's 2-3 times
larger than your skip deltas, but considering that the performance is
about fifty times faster (and I can do distributed stuff without any
locking synchronization and you can't), that's a tradeoff I'm more than
happy with.

Or maybe I misunderstood what you were comparing?

Of course, the numbers will all depend on how the history looks etc, so 
this is all pretty much just guidelines.

			Linus