git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Christian Couder <christian.couder@gmail.com>
To: Stefan Beller <sbeller@google.com>
Cc: git <git@vger.kernel.org>, Junio C Hamano <gitster@pobox.com>,
	Jeff King <peff@peff.net>, Ben Peart <Ben.Peart@microsoft.com>,
	Jonathan Tan <jonathantanmy@google.com>,
	Duy Nguyen <pclouds@gmail.com>, Mike Hommey <mh@glandium.org>,
	Lars Schneider <larsxschneider@gmail.com>,
	Eric Wong <e@80x24.org>,
	Christian Couder <chriscool@tuxfamily.org>,
	Jeff Hostetler <jeffhost@microsoft.com>,
	Eric Sunshine <sunshine@sunshineco.com>,
	Beat Bolli <dev+git@drbeat.li>
Subject: Re: [PATCH v4 9/9] Documentation/config: add odb.<name>.promisorRemote
Date: Tue, 25 Sep 2018 10:07:18 +0200	[thread overview]
Message-ID: <CAP8UFD1ia1xWk9pjfTUQ3zD7=dP=8UjKzf=G0ptsz=qRH8_X+Q@mail.gmail.com> (raw)
In-Reply-To: <CAGZ79kYPik87zvLUPPKFzcATFUdBFcXrAkTYxpxvAQy6w1CcUw@mail.gmail.com>

On Fri, Aug 3, 2018 at 12:55 AM, Stefan Beller <sbeller@google.com> wrote:
> On Wed, Aug 1, 2018 at 11:16 PM Christian Couder
> <christian.couder@gmail.com> wrote:
>>
>> From: Christian Couder <christian.couder@gmail.com>
>>
>> Signed-off-by: Junio C Hamano <gitster@pobox.com>
>> ---
>>  Documentation/config.txt | 5 +++++
>>  1 file changed, 5 insertions(+)
>>
>> diff --git a/Documentation/config.txt b/Documentation/config.txt
>> index 43b2de7b5f..2d048d47f2 100644
>> --- a/Documentation/config.txt
>> +++ b/Documentation/config.txt
>> @@ -2513,6 +2513,11 @@ This setting can be overridden with the `GIT_NOTES_REWRITE_REF`
>>  environment variable, which must be a colon separated list of refs or
>>  globs.
>>
>> +odb.<name>.promisorRemote::
>> +       The name of a promisor remote. For now promisor remotes are
>> +       the only kind of remote object database (odb) that is
>> +       supported.
>> +
>
> Can you explain the end goal for this? (I did not find it in the cover letter,
> nor do I make sense of this documentation)

First thank you for (re)opening this discussion, as I think it may
help resolve the issues related to my work.

In the cover letter there is a "Discussion" section which is about
this, but I agree that it might not be very clear.

The main issue that this patch series tries to solve is that
extensions.partialclone config option limits the partial clone and
promisor features to only one remote. One related issue is that it
also prevents to have other kind of promisor/partial clone/odb
remotes. By other kind I mean remotes that would not necessarily be
git repos, but that could store objects (that's where ODB, for Object
DataBase, comes from) and could provide those objects to Git through a
helper (or driver) script or program.

For reference I tried to raise these issues (especially the first one)
at least twice before extensions.partialclone was merged:

https://public-inbox.org/git/CAP8UFD3Jt+0Lq9Yx_7x3sJD+jG+A25bAgDg7zp+dZV43+1-vow@mail.gmail.com/
https://public-inbox.org/git/CAP8UFD0P7kVo2NP4Wq7OaSV4H1+sqHapuzW5AQef+enNS0S5hw@mail.gmail.com/

but it was still merged as is.

(So of course now it's not surprising that my work on this patch
series keeps conflicting with work that is still going on promisors
and partial clone, and unfortunately the result is that my work keeps
being ejected from pu when it can reach it.)

> So from what I understand, this series relates to partialClone, which
> has the remote name of the "promisor" in extensions.partialclone.
> That is the remote to contact for any needs w.r.t. partial clone and
> fetching on demand.

Yes.

> This key "odb.<name1>.promisorRemote = <name2>" introduces
> 2 new names, where do each of these two names hook in?
> name2 is a remote, such as "origin" from what I can tell, but
> which naming scheme does name1 follow here?

There is just one new name. Instead of:

  extensions.partialclone = <remote name>

there is:

  odb.<remote odb name>.promisorRemote = <remote name>

So it is now like:

  remote.<remote name>.url = <remote url>

which we use for remote repositories.

And it enables us to:

  - have more than one promisor remote
  - specify different parameters for each promisor remote
  - make it possible later to have other kind of promisor/odb remotes

It also restores the distributed nature of Git which was kind of
broken for promisor remotes.

> What makes the odb key different, in that the partial clone
> feature only handles objects as well?

I am not sure I understand this question. Anyway if we want more than
one promisor remote, we need to be able to specify different
parameters for each promisor remote. For example now
core.partialclonefilter is used to specify some filters for the
promisor remote, but how can we nicely specify different partial clone
filters if we have more than one promisor remote?

With the changes in this patch series core.partialclonefilter is
replaced with odb.<remote odb name>.partialclonefilter, so that
parameters for a remote odb are properly grouped together in the
section where the remote odb is defined.

One alternative scheme could be for example to have:

  remote.<remote name>.promisor = (true|false)

or maybe:

  remote.<remote name>.partialclone = (true|false)

instead of:

  extensions.partialclone = <remote name>

And then we could also have:

  remote.<remote name>.partialclonefilter = <partial clone filter>

The issue with this scheme is that it kind of overloads the
"remote.<remote name>.*" namespace for something that can be seen as
different especially if, as I want to do it later, we are going to
have other kind of promisor/odb remotes.

I plan to send a V5 of this patch series really soon now, where I will
try to explain better the end goal.

  reply	other threads:[~2018-09-25  8:07 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-08-02  6:14 [PATCH v4 0/9] Introducing remote ODBs Christian Couder
2018-08-02  6:14 ` [PATCH v4 1/9] fetch-object: make functions return an error code Christian Couder
2018-08-02  6:14 ` [PATCH v4 2/9] Add initial remote odb support Christian Couder
2018-08-02  6:14 ` [PATCH v4 3/9] remote-odb: implement remote_odb_get_direct() Christian Couder
2018-08-02  6:15 ` [PATCH v4 4/9] remote-odb: implement remote_odb_get_many_direct() Christian Couder
2018-08-02  6:15 ` [PATCH v4 5/9] remote-odb: add remote_odb_reinit() Christian Couder
2018-08-02  6:15 ` [PATCH v4 6/9] Use remote_odb_get_direct() and has_remote_odb() Christian Couder
2018-08-02  6:15 ` [PATCH v4 7/9] Use odb.origin.partialclonefilter instead of core.partialclonefilter Christian Couder
2018-08-02  6:15 ` [PATCH v4 8/9] t0410: test fetching from many promisor remotes Christian Couder
2018-08-02  6:15 ` [PATCH v4 9/9] Documentation/config: add odb.<name>.promisorRemote Christian Couder
2018-08-02 22:55   ` Stefan Beller
2018-09-25  8:07     ` Christian Couder [this message]
2018-09-25 22:31       ` Junio C Hamano
2018-09-26  4:12         ` Jeff King
2018-09-26 13:44           ` Taylor Blau
2018-09-26 18:11           ` Junio C Hamano
2018-10-16 17:43       ` Jonathan Nieder
2018-10-16 22:22         ` Jonathan Tan
2018-10-19  0:01           ` Junio C Hamano
2018-10-19  0:33             ` Jonathan Nieder
2018-10-19  2:55               ` Junio C Hamano
2018-10-31  6:28         ` Christian Couder
2018-11-01 21:16           ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAP8UFD1ia1xWk9pjfTUQ3zD7=dP=8UjKzf=G0ptsz=qRH8_X+Q@mail.gmail.com' \
    --to=christian.couder@gmail.com \
    --cc=Ben.Peart@microsoft.com \
    --cc=chriscool@tuxfamily.org \
    --cc=dev+git@drbeat.li \
    --cc=e@80x24.org \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jeffhost@microsoft.com \
    --cc=jonathantanmy@google.com \
    --cc=larsxschneider@gmail.com \
    --cc=mh@glandium.org \
    --cc=pclouds@gmail.com \
    --cc=peff@peff.net \
    --cc=sbeller@google.com \
    --cc=sunshine@sunshineco.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).