git+http:// proof-of-concept (not using CONNECT)

[Eric Wong <normalperson@yhbt.net>]

I've been toying around with an HTTP server for various things over the
past few months and a few weeks ago I came up with the idea of tunneling
arbitrary stream protocols over HTTP with "Transfer-Encoding: chunked"
bodies on both ends.  This allowed me to tunnel the git:// protocol
among other things over something that may conform to RFC 2616.

To facilitate this, curl needed only one minor modification (now in
their CVS repository) to make stdin with "-T-" non-blocking:

  http://cool.haxx.se/cvs.cgi/curl/src/main.c.diff?r1=1.520&r2=1.521

Since git clone already supports proxy commands via GIT_PROXY_COMMAND, I
just created a one line shell script to use as GIT_PROXY_COMMAND.  No
modifications were needed to git itself on either end:

cat > /path/to/script <<\EOF
#!/bin/sh
exec curl --no-buffer -sSfT- http://$1:$2/
EOF

chmod +x /path/to/script
GIT_PROXY_COMMAND=/path/to/script
export GIT_PROXY_COMMAND

# Then, to test it on a small project on my server:
git clone git://git.bogomips.org:8080/pcu

# Of course, a larger project like git should work, too:
git clone git://git.bogomips.org:8080/mirrors/git

Hopefully it doesn't blow up or die on you, this is only lightly
tested.  Let me know if you manage to break it permanently.


The origin server I'm running for this is the latest release of
Unicorn[2], which supports sending a chunked HTTP response as it is
receiving a chunked request.  Unicorn just dechunks the request and
pipes it to git-daemon.  When git-daemon writes to stdout, Unicorn just
grabs the output and chunks it (via Rack[3] middleware) for the client.


This doesn't work in the face of most HTTP-aware proxies[1], so it
probably doesn't help those who have trouble accessing git:// servers in
the first place...  However, this could potentially be useful in places
where a proxy providing CONNECT is not available.


[1] - I run nginx on port 80 on bogomips.org and nginx (attempts to)
fully buffer all requests before proxying it to the backend, so it
definitely won't fly here.  HTTP proxies are perfectly alright with
taking chunked requests/responses and remove chunking on them (or
vice versa).

[2] - http://unicorn.bogomips.org/
[3] - http://rack.rubyforge.org/

-- 
Eric Wong