Re: [PATCH] rev-list: add "--full-objects" flag.

[Linus Torvalds <torvalds@osdl.org>]

On Thu, 7 Jul 2005, Junio C Hamano wrote:
> 
> However it does not automatically mean that the avenue I have
> been pursuing would not work; the server side preparation needs
> to be a bit more careful than what I sent, which unconditionally
> runs "prune-packed".  It instead should leave the files that
> "--whole-trees" would have packed as plain SHA1 files, so that
> the bulk is obtained by statically generated packs and the rest
> can be handled in the commit-chain walker as before.

I really think the commit-chain walker needs to run locally (ie at the 
server end, or after fetching all the objects from the server).

I don't know how much you've tried out the git-http-pull and git-ssh-pull 
things, but their performance was quite horrid for anything half-way 
bigger, because of the totally synchronized IO.

The "fetch one object, parse it, fetch the next one, parse that.." 
approach is just horrible.

I ended up preferring the "rsync" thing even though rsync sucked badly on
big object stores too, if only because when rsync got working, it at least
nicely pipelined the transfers, and would transfer things ten times faster
than git-ssh-pull did (maybe I'm exaggerating, but I don't think so, it
really felt that way).

And the thing is, if you purely follow one tree (which is likely the
common case for a lot of users), then you are actually always likely
better off with the "mirror it" model. Which is _not_ a good model for
developers (for example, me rsync'ing from Jeff's kernel repository always
got me hundreds of useless objects), but it's fine for somebody who
actually just wants to track somebody else.

And then you really can use just rsync or wget or ncftpget or anything
else that has a "fetch recursively, optimizing existing objects" mode.

Now, re-packing ends up causing some double transmissions, but I bet the
cost of those are going to be less than the cost of the "ping-pong for
each object" approach. Especially as most of the repacked objects will be 
deltas if the repacking is done properly.

		Linus