Re: Handling large files with GIT

[Linus Torvalds <torvalds@osdl.org>]

On Tue, 14 Feb 2006, Johannes Schindelin wrote:
> 
> Point is, if you want to read your email on different computers (like one 
> desktop and one laptop), you are quite well off managing them with git. Of 
> course, you could rsync them from/to the other computer. But rsync is slow 
> once you accumulated enough files, since it has to compare the hashes of 
> tons of files (or file chunks). Git knows if they have changed.

Yes. I actually think that git would be a _wonderful_ email tracking tool, 
but that may not mean that it's a wonderful tool for tracking all 
particular email layout possibilities. It clearly is _not_ a wonderful 
tool for tracking mbox-style email setups, for example ;)

I suspect we actually could make the "one linear directory" setup perform 
pretty well. It wouldn't be the best possible layout (by far), but I think 
our problems there are just because of some decisions we've (me, mostly) 
made that didn't take that layout into account. I don't think the problems 
are in any way fundamental.

That said, I think git could do much better if the layout was optimized 
for git. For example, in the maildir thing, there's two issues: the flat 
directory structure is sub-optimal, but the other thing seems to be that 
maildir apparently saves metadata in the filename.

Saving meta-data in the filename should actually work wonderfully well 
with git, but both merging and git-diff-tree consider the filename to be 
the "index", so they optimize for that. You could do indexing the other 
way around, and consider the contents to be the index (and the filename is 
the "status"), but that's obviously not sane for a sw project, even if it 
might be exactly what you want to do for mail handling.

		Linus