Re: Is anyone working on a next-gen Git protocol?

[Andreas Ericsson <ae@op5.se>]

On 10/07/2012 09:57 PM, Ævar Arnfjörð Bjarmason wrote:
> On Wed, Oct 3, 2012 at 9:13 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>>
>>> I'm creating a system where a lot of remotes constantly fetch from a
>>> central repository for deployment purposes, but I've noticed that even
>>> with a remote.$name.fetch configuration to only get certain refs a
>>> "git fetch" will still call git-upload pack which will provide a list
>>> of all references.
>>
>> It has been observed that the sender has to advertise megabytes of
>> refs because it has to speak first before knowing what the receiver
>> wants, even when the receiver is interested in getting updates from
>> only one of them, or worse yet, when the receiver is only trying to
>> peek the ref it is interested has been updated.
> 
> Has anyone started working on a next-gen Git protocol as a result of
> this discussion? If not I thought I'd give it a shot if/when I have
> time.
> 
> The current protocol is basically (S = Server, C = Client)
> 
>   S: Spew out first ref
>   S: Advertisement of capabilities
>   S: Dump of all our refs
>   C/S: Declare wanted refs, negotiate with server
>   S: Send pack to client, if needed
> 
> And I thought I'd basically turn it into:
> 
>   C: Connect to server, declare what protocol we understand
>   C: Advertisement of capabilities
>   S: Advertisement of capabilities
>   C/S: Negotiate what we want
>   C/S: Same as v1, without the advertisement of capabilities, and maybe
> don't dump refs at all
> 
> Basically future-proofing it by having the client say what it supports
> to begin with along with what it can handle (like in HTTP).
> 
> Then in the negotiation phase the client & server would go back &
> forth about what they want & how they want it. I'd planned to
> implement something like:
> 
>      C: want_refs refs/heads/*
>      S: OK to that
>      C: want_refs refs/tags/*
>      S: OK to that
> 
> Or:
> 
>      C: want_refs refs/heads/master
>      S: OK to that
>      C: want_refs refs/tags/v*
>      S: OK to that
> 

You'll want that to be a single "wants" message to avoid incurring
insane amounts of roundtrip latency with lots of refs. github and
other hosted services are quite popular, but with my 120ms ping
rtt I'd be spending half a minute just telling the other side what
I want when I fetch from a repo with 250 refs.

It's a flagday and a half to change the protocol though, so I expect
it'll have to wait for 2.0, unless the current client-side part of
it is dumb and ignores existing refs when requesting its "wants", in
which case the server can just stop advertising existing refs and
most of the speedup is already done.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.