Re: Index/hash order

[Florian Weimer <fw@deneb.enyo.de>]

* Linus Torvalds:

> I will bet you that a git checkout is _faster_ than a kernel source tree
> copy. The time will be dominated by the IO costs (in particular the read
> costs), and the IO costs are lower thanks to compression. So I think that
> the cold-cache case will beat your 40 seconds by a clear margin. It
> generally compresses to half the size, so 20 seconds is not impossible
> (although seek costs would tend to stay constant, so I'd expect it to be
> somewhere in between the two).

It's indeed slightly faster (34 seconds).  The hot-cache case is about
6 seconds.  Still okay.

However, I should redo these tests with a real git.  The numbers could
be quite different because seek overhead is a bit hard to predict.
Which version should I try?

> That's actually pretty encouraging. Your 1.1GB number implies to me that a
> compressed file setup should be about half that, which in turn says that
> the cost of full-file is not at all outrageous.

I usually try to avoid the typical O(f(n)) fallacy because constant
factors do matter in practice.  But the way you put it -- maybe delta
compression isn't worth the complexity after all.  At least I'm
beginning to have doubts.

Especially since the same Subversion repository, stored by the
Berkeley DB backend, requires a whopping 1.3 GB of disk space.

> Or maybe I misunderstood what you were comparing?

My estimates only cover file data, not metadata.  Based on the
Subversion dumps, it might be possible to get some rough estimates for
the cost of storing directory information.  What is the average size
of a directory blob?  Is it true that for each tree revision, you need
to store a new directory blob for each directory which indirectly
contains a modified file?

Does your 50% estimate include wasted space due to the file system
block size?