Re: [RFC/PATCH 0/3] protocol v2

[Stefan Beller <sbeller@google.com>]

On Sun, Mar 1, 2015 at 3:32 AM, Duy Nguyen <pclouds@gmail.com> wrote:
> On Sun, Mar 1, 2015 at 3:41 PM, Junio C Hamano <gitster@pobox.com> wrote:
>>>  - Because the protocol exchange starts by the server side
>>>    advertising all its refs, even when the fetcher is interested in
>>>    a single ref, the initial overhead is nontrivial, especially when
>>>    you are doing a small incremental update.  The worst case is an
>>>    auto-builder that polls every five minutes, even when there is no
>>>    new commits to be fetched [*3*].
>
> Maybe you can elaborate about how to handle states X, Y... in your
> footnote 3. I just don't see how it's actually implemented. Or is it
> optional feature that will be provided (via hooks, maybe) by admin? Do
> we need to worry about load balancers? Is it meant to address the
> excessive state transfer due to stateless nature of smart-http?

The way I understand Junio here is to have predefined points which
makes it easier to communicate. There are lots of clients and they usually
want to catch up a different amount of commits, so we need to recompute it
all the time. The idea is then to compute a small pack from the original point
to one of these predefined points.
So a conversion might look like:
Client: My newest commit is dated 2014-11-17.
Server: ok here is a pack from 2014-11-17 until 2014-12-01 and then
I have prepared packs I sent out all the time of 2014-12 and 2015-01
and 2015-02 and then there will be another custom pack for you describing
changes of 2015-02-01 until now.

Mind that I choose dates instead of arbitrary sha1 values as I feel
that explains the
point better, the packs in between are precomputed because many
clients need them.

Personally I don't buy that idea, because it produces a lot of question, like
how large should these packs be? Depending on time or commit counts?

The idea I'd rather favor (I am repeating myself from another post,
but maybe a bit clearer now):

    Client: The last time I asked for "refs/heads/*" and I got a refs
        advertisement hashing to $SHA1
    Server: Ok, here is the diff from that old ref advertisement to the
        current refs advertisement.

I realize that these two ideas are not contradicting each other, but
could rather
help each other as they are orthogonal to each other. One is about
refs advertising
while the other is about object transmission.

>
>>> I'd like to see a new protocol that lets us overcome the above
>>> limitations (did I miss others? I am sure people can help here)
>>> sometime this year.
>>
>> Unfortunately, nobody seems to want to help us by responding to "did
>> I miss others?" RFH, here are a few more from me.
>
> Heh.. I did think about it, but I didn't see anything worth mentioning..
>
>>  - The semantics of the side-bands are unclear.
>>
>>    - Is band #2 meant only for progress output (I think the current
>>      protocol handlers assume that and unconditionally squelch it
>>      under --quiet)?  Do we rather want a dedicated "progress" and
>>      "error message" sidebands instead?
>>
>>    - Is band #2 meant for human consumption, or do we expect the
>>      other end to interpret and act on it?  If the former, would it
>>      make sense to send locale information from the client side and
>>      ask the server side to produce its output with _("message")?
>
> No producing _(...) is a bad idea. First the client has to verify
> placeholders and stuff, we can't just feed data from server straight
> to printf(). Producing _() could complicate server code a lot. And I
> don't like the idea of using client .po files to translate server
> strings. There could be custom strings added by admin, which are not
> available in client .po. String translation should happen at server
> side.
>
> If we want error messages to be handled by machine as well, just add a
> result code at the beginning, like ftp, http, ... do. Hmm.. this could
> be the reason to separate progress and error messages.
> --
> Duy