From: Jonathan Nieder <jrnieder@gmail.com>
To: Christian Couder <christian.couder@gmail.com>
Cc: "Jonathan Tan" <jonathantanmy@google.com>,
git <git@vger.kernel.org>, "Junio C Hamano" <gitster@pobox.com>,
"Jeff King" <peff@peff.net>,
"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: Re: [WIP 0/7] CDN offloading of fetch response
Date: Thu, 28 Feb 2019 15:21:08 -0800 [thread overview]
Message-ID: <20190228232108.GA163714@google.com> (raw)
In-Reply-To: <CAP8UFD1tKNFO9wU8CbgNnSnQyvHYPsZMk1Bit2y1jxH4vk62qQ@mail.gmail.com>
Hi,
Sorry for the slow followup. Thanks for probing into the design ---
this should be useful for getting the docs to be clear.
Christian Couder wrote:
> So it's likely that users will want a way to host on such sites
> incomplete repos using CDN offloading to a CDN on another site. And
> then if the CDN is not accessible for some reason, things will
> completely break when users will clone.
I think this would be a broken setup --- we can make it clear in the
protocol and server docs that you should only point to a CDN for which
you control the contents, to avoid breaking clients.
That doesn't prevent adding additional features in the future e.g. for
"server suggested alternates" --- it's just that I consider that a
separate feature.
Using CDN offloading requires cooperation of the hosting provider.
It's a way to optimize how fetches work, not a way to have a partial
repository on the server side.
[...]
> On Tue, Feb 26, 2019 at 12:45 AM Jonathan Nieder <jrnieder@gmail.com> wrote:
>> This doesn't stop a hosting provider from using e.g. server options to
>> allow the client more control over how their response is served, just
>> like can be done for other features of how the transfer works (how
>> often to send progress updates, whether to prioritize latency or
>> throughput, etc).
>
> Could you give a more concrete example of what could be done?
What I mean is passing server options using "git fetch --server-option".
For example:
git fetch -o priority=BATCH origin master
or
git fetch -o avoid-cdn=badcdn.example.com origin master
The interpretation of server options is up to the server.
>> What the client *can* do is turn off support for packfile URLs in a
>> request completely. This is required for backward compatibility and
>> allows working around a host that has configured the feature
>> incorrectly.
>
> If the full content of a repo is really large, the size of a full pack
> file sent by an initial clone could be really big and many client
> machines could not have enough memory to deal with that. And this
> suppose that repo hosting providers would be ok to host very large
> repos in the first place.
Do we require the packfile to fit in memory? If so, we should fix
that (to use streaming instead).
Thanks,
Jonathan
next prev parent reply other threads:[~2019-02-28 23:21 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-23 23:38 [WIP 0/7] CDN offloading of fetch response Jonathan Tan
2019-02-23 23:38 ` [WIP 1/7] http: use --stdin and --keep when downloading pack Jonathan Tan
2019-02-23 23:38 ` [WIP 2/7] http: improve documentation of http_pack_request Jonathan Tan
2019-02-23 23:38 ` [WIP 3/7] http-fetch: support fetching packfiles by URL Jonathan Tan
2019-02-23 23:38 ` [WIP 4/7] Documentation: order protocol v2 sections Jonathan Tan
2019-02-23 23:38 ` [WIP 5/7] Documentation: add Packfile URIs design doc Jonathan Tan
2019-02-23 23:39 ` [WIP 6/7] upload-pack: refactor reading of pack-objects out Jonathan Tan
2019-02-23 23:39 ` [WIP 7/7] upload-pack: send part of packfile response as uri Jonathan Tan
2019-02-24 15:54 ` Junio C Hamano
2019-02-25 21:04 ` Christian Couder
2019-02-26 1:53 ` Jonathan Nieder
2019-02-26 7:08 ` Christian Couder
2019-03-01 0:09 ` Josh Steadmon
2019-03-01 0:17 ` Jonathan Tan
2019-02-25 21:30 ` [WIP 0/7] CDN offloading of fetch response Christian Couder
2019-02-25 23:45 ` Jonathan Nieder
2019-02-26 8:30 ` Christian Couder
2019-02-26 9:12 ` Ævar Arnfjörð Bjarmason
2019-03-04 8:24 ` Christian Couder
2019-02-28 23:21 ` Jonathan Nieder [this message]
2019-03-04 8:54 ` Christian Couder
2019-03-08 21:55 ` [PATCH v2 0/8] " Jonathan Tan
2019-03-08 21:55 ` [PATCH v2 1/8] http: use --stdin when getting dumb HTTP pack Jonathan Tan
2019-03-08 21:55 ` [PATCH v2 2/8] http: improve documentation of http_pack_request Jonathan Tan
2019-03-08 21:55 ` [PATCH v2 3/8] http-fetch: support fetching packfiles by URL Jonathan Tan
2019-03-08 21:55 ` [PATCH v2 4/8] Documentation: order protocol v2 sections Jonathan Tan
2019-03-08 21:55 ` [PATCH v2 5/8] Documentation: add Packfile URIs design doc Jonathan Tan
2019-04-23 5:31 ` Jeff King
2019-04-23 20:38 ` Jonathan Tan
2019-04-23 22:18 ` Ævar Arnfjörð Bjarmason
2019-04-23 22:22 ` Jonathan Nieder
2019-04-23 22:30 ` Ævar Arnfjörð Bjarmason
2019-04-23 22:51 ` Jonathan Nieder
2019-04-23 22:11 ` Jonathan Nieder
2019-04-23 22:25 ` Ævar Arnfjörð Bjarmason
2019-04-23 22:48 ` Jonathan Nieder
2019-04-24 7:48 ` Ævar Arnfjörð Bjarmason
2019-04-24 3:01 ` Junio C Hamano
2019-03-08 21:55 ` [PATCH v2 6/8] upload-pack: refactor reading of pack-objects out Jonathan Tan
2019-03-08 21:55 ` [PATCH v2 7/8] fetch-pack: support more than one pack lockfile Jonathan Tan
2019-03-08 21:55 ` [PATCH v2 8/8] upload-pack: send part of packfile response as uri Jonathan Tan
2019-03-19 20:48 ` [PATCH v2 0/8] CDN offloading of fetch response Josh Steadmon
2019-04-23 5:21 ` Jeff King
2019-04-23 19:23 ` Jonathan Tan
2019-04-24 9:09 ` Ævar Arnfjörð Bjarmason
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190228232108.GA163714@google.com \
--to=jrnieder@gmail.com \
--cc=avarab@gmail.com \
--cc=christian.couder@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=jonathantanmy@google.com \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).