Re: jt/fetch-cdn-offload (was What's cooking in git.git (Apr 2019, #04; Mon, 22))

[Jonathan Tan <jonathantanmy@google.com>]

> There has been no answer to my comments in:
> 
> https://public-inbox.org/git/CAP8UFD3SWNu=btPxV2vV=neYrofbgKPzz_WLvsJbv6bKjRoCpQ@mail.gmail.com/
> 
> especially the part related to the "-o avoid-cdn=badcdn.example.com"
> example that Jonathan Nieder gave.

It seems to me that if you use a server option to blacklist, you can
also use a server option to whitelist. But I don't think this is the
main issue, as you state below.

> > If this version is good with everyone, then this is the final version.
> 
> It is not good for me as I think the "-o
> avoid-cdn=badcdn.example.com", or even "-o usecdn=goodcdn.example.com"
> options, (that has been the only thing suggested to work around
> problems with CDNs that people cannot use or don't want to use,) will
> likely end up to be some other kind of promisor remote but not quite a
> real promisor remote.

I think that the resemblance between promisor remotes and the targets of
"packfile-uri" (CDNs) is not that deep. Even if we have a sufficiently
smart remote helper that does not require Git-specific logic in the
service hosting the remote, promisor remotes operate on the blob level
(maybe tree at most) and does not require much coordination with the
server that you originally fetched from; for example, a server can just
as easily serve "blob:limit=43" as it does "blob:limit=42" without
coordinating with the promisor remote.

But this is not the same with the CDNs. CDNs operate on the packfile
level - we do not want to send individual objects. And the coordination
between the CDN and the server that you originally fetched from has to
be greater - due to the numbers of objects in packfiles, the server will
need to know what is served by the CDN on a higher level (e.g. things
like "everything in refs/heads/* after <date>"), and if we want to even
slightly shift what the CDN serves and what the server serves, both need
to know.

(Also, I'm not convinced that a sufficiently smart remote helper can be
built that can turn a generic CDN into a performant promisor remote that
we want - in particular, if partial clones with "tree:0" filter become
more popular, servers might need to be able to send trees with various
filters: the tree alone, the tree with all its subtrees recursively
(maybe with a maximum depth), and the tree with all its subtrees and
blobs recursively.)

> In a more general way I don't understand why I was repeatedly asked
> (especially by Jonathen Nieder, you and Junio) to dump ODB remotes in
> favor of promisor remotes because promisor remotes would be more
> unified, and now you develop something that is not unified with
> promisor remote, though it could very well be.

I don't remember saying this (although perhaps I did), but ODB remotes
are similar to promisor remotes in that they both operate on the object
level. As for CDNs, they operate on the packfile level, so unification
with promisor remote is not so easy. (And there is the sufficiently
smart remote helper issue.)