From: Jeff King <peff@peff.net>
To: Tay Ray Chuan <rctay89@gmail.com>
Cc: git@vger.kernel.org
Subject: Re: [PATCH 0/14] resumable network bundles
Date: Sat, 12 Nov 2011 12:58:26 -0500 [thread overview]
Message-ID: <20111112175826.GA9427@sigill.intra.peff.net> (raw)
In-Reply-To: <CALUzUxqsLP11Tcsoc1tzGMfNcMqor2wQF+Yutu3FRiTin4Pnew@mail.gmail.com>
On Sun, Nov 13, 2011 at 12:11:31AM +0800, Tay Ray Chuan wrote:
> One thing I'm not comfortable with is the "flexibility" allowed in
> bundle fetching - servers are allowed to send bundles if they see fit,
> and we have to detect it when they do (if I'm reading the "surprised"
> scenario in patch 9 correctly).
Right.
> Perhaps we can expose bundle fetching through /objects/info/bundles?
But what if the server you are hitting doesn't have a git repo at all?
In the simplest case, a bundle provider should just be able to put a
file somewhere http-ccessible, without having any special directory
structure or other meta files.
Which means that we have to be prepared for the URL the user gave us to
be a bundle, not a git repo that contains bundles.
> It could possibly contain information about what bundles are available
> and what revs they contain. If bundles are found, fetch them;
> otherwise, go through the usual ref advertisement and other steps of
> the pack protocol.
This is "step 2" of my plan: hitting a git repo will provide a way of
redirecting to other, static storage. But I think it's important that
the other storage not just be a path in the existing repo, for two
reasons:
1. You might want to redirect the client off-server to a
higher-bandwidth static service like S3, or something backed by a
CDN.
2. The client might not be hitting you through http, so you can't
expect them to look at arbitrary repo files (like
objects/info/bundles). We need to provide the information over the
git protocol (my plan is to use a special ref name, like
"refs/mirrors" to encode the information).
> That way, we take out the "surprise" factor in the fetching protocol.
I don't think it's that big a deal. It influenced the way that patches 9
and 10 were written (patch 9 handles "surprise" bundles when fetching
info/refs, and then patch 10 falls back to fetching $URL without
info/refs). But even if we didn't have the "surprise" case, most of the
code in patch 9 would have just ended up in patch 10. That is, the
surprise case doesn't take much code, and doesn't have a negative impact
on the non-surprise case (i.e., until we see a bundle header, the
behavior is identical, just putting the refs into a memory buffer).
-Peff
prev parent reply other threads:[~2011-11-12 17:58 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-11-10 7:43 [PATCH 0/14] resumable network bundles Jeff King
2011-11-10 7:45 ` Jeff King
2011-11-10 7:46 ` [PATCH 01/14] t/lib-httpd: check for NO_CURL Jeff King
2011-11-10 7:48 ` [PATCH 02/14] http: turn off curl signals Jeff King
2011-11-10 8:43 ` Daniel Stenberg
2011-11-11 20:54 ` Jeff King
2011-11-10 7:48 ` [PATCH 03/14] http: refactor http_request function Jeff King
2011-11-10 7:49 ` [PATCH 04/14] http: add a public function for arbitrary-callback request Jeff King
2011-11-10 7:49 ` [PATCH 05/14] remote-curl: use http callback for requesting refs Jeff King
2011-11-10 7:49 ` [PATCH 06/14] transport: factor out bundle to ref list conversion Jeff King
2011-11-10 7:50 ` [PATCH 07/14] bundle: add is_bundle_buf helper Jeff King
2011-11-10 7:50 ` [PATCH 08/14] remote-curl: free "discovery" object Jeff King
2011-11-10 7:50 ` [PATCH 09/14] remote-curl: auto-detect bundles when fetching refs Jeff King
2011-11-10 7:51 ` [PATCH 10/14] remote-curl: try base $URL after $URL/info/refs Jeff King
2011-11-10 7:53 ` [PATCH 11/14] progress: allow pure-throughput progress meters Jeff King
2011-11-10 7:53 ` [PATCH 12/14] remote-curl: show progress for bundle downloads Jeff King
2011-11-10 7:53 ` [PATCH 13/14] remote-curl: resume interrupted bundle transfers Jeff King
2011-11-10 7:56 ` [PATCH 14/14] clone: give advice on how to resume a failed clone Jeff King
2011-11-10 21:21 ` Junio C Hamano
2011-11-11 20:52 ` Jeff King
2011-11-11 13:13 ` [PATCH 0/14] resumable network bundles David Michael Barr
2011-11-12 16:11 ` Tay Ray Chuan
2011-11-12 17:58 ` Jeff King [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20111112175826.GA9427@sigill.intra.peff.net \
--to=peff@peff.net \
--cc=git@vger.kernel.org \
--cc=rctay89@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).