Handling large files with GIT

[Martin Langhoff <martin.langhoff@gmail.com>]

Roland Stigge recently pointed out a use case using very large files
where GIT has some serious limitations. He is one of several Debian
developers keeping their homedir under version control with SVN (blame
Joey Hess for this - http://www.kitenet.net/~joey/svnhome.html ).

SVN does reasonably well tracking his >1GB mbox file. Now, I don't
know if I like the idea of putting my own mbox file under version
control, but it looks like projects with large and slow-changing files
would be in trouble with GIT. Not literally trouble, but gross
inefficiencies.

The problems are two. At commit time, a full copy is stored in the
object database until git-repack && git-prune-packed are called. And
during the transfer over the git protocol we send the full object,
even if both ends have objects that are good candidates for a small
delta.

I'm not strong on either aspect of git (packfile format or git
protocol), and I don't personally deal with large files. So feel free
to ignore me for the time being. If it ever itches, you might get a
patch...

cheers,


martin