git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Jonathan Tan <jonathantanmy@google.com>
To: jrnieder@gmail.com
Cc: christian.couder@gmail.com, sbeller@google.com,
	git@vger.kernel.org, gitster@pobox.com, peff@peff.net,
	Ben.Peart@microsoft.com, jonathantanmy@google.com,
	pclouds@gmail.com, mh@glandium.org, larsxschneider@gmail.com,
	e@80x24.org, chriscool@tuxfamily.org, jeffhost@microsoft.com,
	sunshine@sunshineco.com, dev+git@drbeat.li
Subject: Re: [PATCH v4 9/9] Documentation/config: add odb.<name>.promisorRemote
Date: Tue, 16 Oct 2018 15:22:43 -0700	[thread overview]
Message-ID: <20181016222243.58620-1-jonathantanmy@google.com> (raw)
In-Reply-To: <20181016174304.GA221682@aiede.svl.corp.google.com>

>  1. Teaching partial clone to attempt to fetch missing objects from
>     multiple remotes instead of only one.  This is useful because you
>     can have a server that is nearby and cheaper to serve from (some
>     kind of local cache server) that you make requests to first before
>     falling back to the canonical source of objects.

Quoting the above definition of (1) for reference. I think Jonathan
Nieder has covered the relevant points well - I'll just expand on (1).

> So much for the current setup.  For (1), I believe you are proposing to
> still have only one effective <promisor-id>, so it doesn't necessarily
> require modifying the extensions.* configuration.  Instead, the idea is
> that when trying to access an object, we would follow one of a list of
> steps:
> 
>  1. First, check the local object store. If it's there, we're done.
>  2. Second, try alternates --- maybe the object is in one of those!
>  3. Now, try promisor remotes, one at a time, in user-configured order.
> 
> In other words, I think that for (1) all we would need is a new
> configuration
> 
> 	[object]
> 		missingObjectRemote = local-cache-remote
> 		missingObjectRemote = origin
> 
> The semantics would be that when trying to access a promised object,
> we attempt to fetch from these remotes one at a time, in the order
> specified.  We could require that the remote named in
> extensions.partialClone be one of the listed remotes, without having
> to care where it shows up in the list.

Or allow extensions.partialClone=<R> wherein <R> is not in the
missingObjectRemote, in which case <R> is tried first, so that we don't
have to reject some configurations.

> That way, we get the benefit (1) without having to change the
> semantics of extensions.partialClone and without having to care about
> the order of sections in the config.  What do you think?

Let's define the promisor remotes of a repository as those in
missingObjectRemote or extensions.partialClone (currently, we talk about
"the promisor remote" (singular), defined in extensions.partialClone).

Overall, this seems like a reasonable idea to me, if we keep the
restriction that we can only fetch with filter from a promisor remote.
This allows us to extend the definition of a promisor object in a
manner consistent to the current definition - to say "a promisor object
is one promised by at least one promisor remote" (currently, "a promisor
object is promised by the promisor remote").

In the presence of missingObjectRemote, old versions of Git, when lazily
fetching, would only know to try the extensions.partialClone remote. But
this is safe because existing data wouldn't be clobbered (since we're
not using ideas like adding meaning to the contents of the .promisor
file). Also, other things like fsck and gc still work.

  reply	other threads:[~2018-10-16 22:22 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-08-02  6:14 [PATCH v4 0/9] Introducing remote ODBs Christian Couder
2018-08-02  6:14 ` [PATCH v4 1/9] fetch-object: make functions return an error code Christian Couder
2018-08-02  6:14 ` [PATCH v4 2/9] Add initial remote odb support Christian Couder
2018-08-02  6:14 ` [PATCH v4 3/9] remote-odb: implement remote_odb_get_direct() Christian Couder
2018-08-02  6:15 ` [PATCH v4 4/9] remote-odb: implement remote_odb_get_many_direct() Christian Couder
2018-08-02  6:15 ` [PATCH v4 5/9] remote-odb: add remote_odb_reinit() Christian Couder
2018-08-02  6:15 ` [PATCH v4 6/9] Use remote_odb_get_direct() and has_remote_odb() Christian Couder
2018-08-02  6:15 ` [PATCH v4 7/9] Use odb.origin.partialclonefilter instead of core.partialclonefilter Christian Couder
2018-08-02  6:15 ` [PATCH v4 8/9] t0410: test fetching from many promisor remotes Christian Couder
2018-08-02  6:15 ` [PATCH v4 9/9] Documentation/config: add odb.<name>.promisorRemote Christian Couder
2018-08-02 22:55   ` Stefan Beller
2018-09-25  8:07     ` Christian Couder
2018-09-25 22:31       ` Junio C Hamano
2018-09-26  4:12         ` Jeff King
2018-09-26 13:44           ` Taylor Blau
2018-09-26 18:11           ` Junio C Hamano
2018-10-16 17:43       ` Jonathan Nieder
2018-10-16 22:22         ` Jonathan Tan [this message]
2018-10-19  0:01           ` Junio C Hamano
2018-10-19  0:33             ` Jonathan Nieder
2018-10-19  2:55               ` Junio C Hamano
2018-10-31  6:28         ` Christian Couder
2018-11-01 21:16           ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20181016222243.58620-1-jonathantanmy@google.com \
    --to=jonathantanmy@google.com \
    --cc=Ben.Peart@microsoft.com \
    --cc=chriscool@tuxfamily.org \
    --cc=christian.couder@gmail.com \
    --cc=dev+git@drbeat.li \
    --cc=e@80x24.org \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jeffhost@microsoft.com \
    --cc=jrnieder@gmail.com \
    --cc=larsxschneider@gmail.com \
    --cc=mh@glandium.org \
    --cc=pclouds@gmail.com \
    --cc=peff@peff.net \
    --cc=sbeller@google.com \
    --cc=sunshine@sunshineco.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).