Git Smart HTTP with HTTP/2.0

Git Smart HTTP with HTTP/2.0

[ForceCharlie]

As we known, HTTP/2.0 has been released. All Git-Smart-HTTP are currently
implemented using HTTP/1.1.

Frequently used Git developers often feel Git HTTP protocol is not
satisfactory, slow and unstable.This is because the HTTP protocol itself
decides
When HTTP/2.0 is published. We might be able to git developers jointly,
based on HTTP/2.0 Git-Smart-HTTP service and client support.
HTTP/2.0: https://tools.ietf.org/html/rfc7540
Github Mirror: https://httpwg.github.io/specs/rfc7540.html
HTTP/2.0 has Flow Controller like SSH, 
HTTP/2.0 has Server Push POST upload-pack can download large more
object-pack

......
libcurl now has begun to support HTTP/2.0, git is also using curl, as well
libgit2 may use libcurl
I suggest the git of developer joint developer of libgit2 and jgit
developers discussion based on HTTP/2.0 Git-Smart-HTTP
Now Subversion developer begin write a New HTTP Protocol for svn ("HTTP
v2"):http://svn.apache.org/repos/asf/subversion/trunk/notes/http-and-webdav/
http-protocol-v2.txt
What about git ?

Re: Git Smart HTTP with HTTP/2.0

[Ilari Liusvaara]

On Sat, Jul 11, 2015 at 11:10:48AM +0800, ForceCharlie wrote:
> As we known, HTTP/2.0 has been released. All Git-Smart-HTTP are currently
> implemented using HTTP/1.1.

Nit: It is HTTP/2.
 
> Frequently used Git developers often feel Git HTTP protocol is not
> satisfactory, slow and unstable.This is because the HTTP protocol itself
> decides

Note that there are already two versions of HTTP transport, the old "dumb"
one and the newer "smart" one.

The smart one is difficult to speed up (due to nature of the negotiations),
but usually is pretty reliable (the efficiency isn't horrible).

Now, the old "dumb" protocol is pretty unreliable and slow. HTTP/2 probably
can't do anything with the reliability problems, but probably could improve
the speed a bit.

Websockets over HTTP/2 (a.k.a. "websockets2") has not been defined yet.
With Websockets(1), it would probably already be possible to tunnel the
native git smart transport protocol over it. Probably not worth it.

> When HTTP/2.0 is published. We might be able to git developers jointly,
> based on HTTP/2.0 Git-Smart-HTTP service and client support.
> HTTP/2.0: https://tools.ietf.org/html/rfc7540

Well, it is published already.
 

-Ilari

Re: Git Smart HTTP with HTTP/2.0

[Shawn Pearce]

On Sat, Jul 11, 2015 at 12:00 AM, Ilari Liusvaara
<ilari.liusvaara@elisanet.fi> wrote:
> On Sat, Jul 11, 2015 at 11:10:48AM +0800, ForceCharlie wrote:
>> As we known, HTTP/2.0 has been released. All Git-Smart-HTTP are currently
>> implemented using HTTP/1.1.
>
> Nit: It is HTTP/2.
>
>> Frequently used Git developers often feel Git HTTP protocol is not
>> satisfactory, slow and unstable.This is because the HTTP protocol itself
>> decides
>
> Note that there are already two versions of HTTP transport, the old "dumb"
> one and the newer "smart" one.
>
> The smart one is difficult to speed up (due to nature of the negotiations),
> but usually is pretty reliable (the efficiency isn't horrible).

The negotiation in smart-HTTP actually has some bad corner cases. Each
round of negotiation requires a new POST resupplying all previously
agreed upon SHA-1s, and a batch of new SHA-1s. We have observed many
rounds where this POST is MiBs in size because the peers can't quite
agree and have to keep digging through history.

The native protocol on git:// and SSH is not as bad. Negotiation is
still many rounds, but these are pipelined and each round does not
need to repeat the prior round, as the server has a single stream and
is saving state.

> Now, the old "dumb" protocol is pretty unreliable and slow. HTTP/2 probably
> can't do anything with the reliability problems, but probably could improve
> the speed a bit.
>
> Websockets over HTTP/2 (a.k.a. "websockets2") has not been defined yet.
> With Websockets(1), it would probably already be possible to tunnel the
> native git smart transport protocol over it. Probably not worth it.

Another option is to tunnel using gRPC (grpc.io). libcurl probably
can't do this. And linking grpc.io library into git-core is crazy. So
its probably a non-starter. But food for thought.

But, at $DAY_JOB we tunnel the native bidirectional protocol in
grpc.io's predecessor, and it works quite well for us.

Re: Git Smart HTTP with HTTP/2.0

[Ilari Liusvaara]

On Sat, Jul 11, 2015 at 10:23:09AM -0700, Shawn Pearce wrote:
> On Sat, Jul 11, 2015 at 12:00 AM, Ilari Liusvaara
> <ilari.liusvaara@elisanet.fi> wrote:
> > On Sat, Jul 11, 2015 at 11:10:48AM +0800, ForceCharlie wrote:
> >
> >> Frequently used Git developers often feel Git HTTP protocol is not
> >> satisfactory, slow and unstable.This is because the HTTP protocol itself
> >> decides
> >
> > Note that there are already two versions of HTTP transport, the old "dumb"
> > one and the newer "smart" one.
> >
> > The smart one is difficult to speed up (due to nature of the negotiations),
> > but usually is pretty reliable (the efficiency isn't horrible).
> 
> The negotiation in smart-HTTP actually has some bad corner cases. Each
> round of negotiation requires a new POST resupplying all previously
> agreed upon SHA-1s, and a batch of new SHA-1s. We have observed many
> rounds where this POST is MiBs in size because the peers can't quite
> agree and have to keep digging through history.

Oh yeah that... Well, that is artifact of HTTP semantics.

> > Now, the old "dumb" protocol is pretty unreliable and slow. HTTP/2 probably
> > can't do anything with the reliability problems, but probably could improve
> > the speed a bit.
> >
> > Websockets over HTTP/2 (a.k.a. "websockets2") has not been defined yet.
> > With Websockets(1), it would probably already be possible to tunnel the
> > native git smart transport protocol over it. Probably not worth it.
> 
> Another option is to tunnel using gRPC (grpc.io). libcurl probably
> can't do this. And linking grpc.io library into git-core is crazy. So
> its probably a non-starter. But food for thought.

Wouldn't it link into git-remote-http (and on the server side, one
could use pipes to talk to git)?

But supporting websockets in git-remote-http could get annoying,
especially for wss:// (https://). Dunno how bad gRPC would be.



-Ilari

Re: Git Smart HTTP with HTTP/2.0

[Shawn Pearce]

On Sat, Jul 11, 2015 at 11:26 AM, Ilari Liusvaara
<ilari.liusvaara@elisanet.fi> wrote:
> On Sat, Jul 11, 2015 at 10:23:09AM -0700, Shawn Pearce wrote:
>>
>> > Websockets over HTTP/2 (a.k.a. "websockets2") has not been defined yet.
>> > With Websockets(1), it would probably already be possible to tunnel the
>> > native git smart transport protocol over it. Probably not worth it.
>>
>> Another option is to tunnel using gRPC (grpc.io). libcurl probably
>> can't do this. And linking grpc.io library into git-core is crazy. So
>> its probably a non-starter. But food for thought.
>
> Wouldn't it link into git-remote-http (and on the server side, one
> could use pipes to talk to git)?
>
> But supporting websockets in git-remote-http could get annoying,
> especially for wss:// (https://). Dunno how bad gRPC would be.

We wrote it as git-remote-$THING, invoked with $THING:// URLs. And
git-remote-$THING just implements the "connect" helper protocol. Its
much simpiler than git-remote-http.

Maybe its done that way in git-core as http2:// aka git-remote-http2.

Or the git-remote-http helper connects to the remote system and tries
to guess if it supports Git on HTTP/2 before responding to the
capabilities request from transport code. If yes, it replies with
connect, if no, it does the current fetch and push protocol.