avoiding fetching specific refs from a remote

avoiding fetching specific refs from a remote

[Jacob Keller]

Hi,

I've recently had a few repositories switch their default branch from
master to main. Currently, for backwards compatibility these
repositories have kept master around as a synonym for main. It's
unclear when this will be removed.

This results in the annoying completion behavior where:

$git checkout ma<TAB>
main
master

Essentially, because both main and master have similar names, tab
completion requires remembering to type one additional character than
I am used to.

I'd like to be able to customize my pull so that I either (a) exclude
the reference when performing a fetch, or (b) exclude the reference
when tab completing.

I don't know if there is any mechanism already in place to do this, so
I'm trying to figure out which one would be most preferable on the
list?

something like adding a remote.<name>.excludeRefs which would allow
specifying a set of references to exclude from matching the refspec..

I realize I could customize the refspec to list the set of things I
want, but then I would miss any potential future branches until I
updated my remote config again.

I would like to be able to remove this annoyance for myself and colleagues.

It seems to me the easiest solution would be the sort of exclude
option, as this enables generally skipping out on specific refs, and
could be useful beyond just this initial impetus.

Thanks,
Jake

Re: avoiding fetching specific refs from a remote

[Jeff King]

On Tue, Aug 04, 2020 at 02:53:30PM -0700, Jacob Keller wrote:

> This results in the annoying completion behavior where:
> 
> $git checkout ma<TAB>
> main
> master
> 
> Essentially, because both main and master have similar names, tab
> completion requires remembering to type one additional character than
> I am used to.

Heh, I didn't think anybody considered that implication so far; the fact
that they were similar names has generally been considered a positive. I
agree it would be annoying.

> I'd like to be able to customize my pull so that I either (a) exclude
> the reference when performing a fetch, or (b) exclude the reference
> when tab completing.

Not quite (b), but if you:

  - delete any local "master" branch, if you haven't already

  - set $GIT_COMPLETION_CHECKOUT_NO_GUESS to "1"

then it shouldn't complete "master". The second is necessary because
we'd try to complete the magic "git checkout master" that auto-creates
the branch from the remote version.

> something like adding a remote.<name>.excludeRefs which would allow
> specifying a set of references to exclude from matching the refspec..
> 
> I realize I could customize the refspec to list the set of things I
> want, but then I would miss any potential future branches until I
> updated my remote config again.

This is definitely a reasonable thing to want, and it has come up off
and on over the years. One search term for the list archive is "negative
refspecs", though it turns up a lot of useless hits when the two words
are not directly adjacent.

This old thread might be worth reading:

  https://lore.kernel.org/git/20140124090104.GA396@x4/

and there is even a patch in there:

  https://lore.kernel.org/git/20140125013433.GA22336@sigill.intra.peff.net/

but I didn't go over it carefully enough to know whether it is utter
trash, or something that could be used as a starting point.

-Peff

Re: avoiding fetching specific refs from a remote

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

>> Essentially, because both main and master have similar names, tab
>> completion requires remembering to type one additional character than
>> I am used to.
>
> Heh, I didn't think anybody considered that implication so far; the fact
> that they were similar names has generally been considered a positive. I
> agree it would be annoying.

FWIW, I've been saying that 'maint' will become harder to type all
along ever since I heard 'main'.

https://lore.kernel.org/git/xmqqtuz9tq30.fsf@gitster.c.googlers.com/

> This is definitely a reasonable thing to want, and it has come up off
> and on over the years. One search term for the list archive is "negative
> refspecs", though it turns up a lot of useless hits when the two words
> are not directly adjacent.
>
> This old thread might be worth reading:
>
>   https://lore.kernel.org/git/20140124090104.GA396@x4/
>
> and there is even a patch in there:
>
>   https://lore.kernel.org/git/20140125013433.GA22336@sigill.intra.peff.net/
>
> but I didn't go over it carefully enough to know whether it is utter
> trash, or something that could be used as a starting point.

I think the idea is interesting.  I sometimes find negative
pathspecs quite useful, and I suspect negative refspecs would also
be.

Re: avoiding fetching specific refs from a remote

[Jacob Keller]

On Wed, Aug 5, 2020 at 9:22 AM Junio C Hamano <gitster@pobox.com> wrote:
>
> Jeff King <peff@peff.net> writes:
>
> >> Essentially, because both main and master have similar names, tab
> >> completion requires remembering to type one additional character than
> >> I am used to.
> >
> > Heh, I didn't think anybody considered that implication so far; the fact
> > that they were similar names has generally been considered a positive. I
> > agree it would be annoying.
>
> FWIW, I've been saying that 'maint' will become harder to type all
> along ever since I heard 'main'.
>
> https://lore.kernel.org/git/xmqqtuz9tq30.fsf@gitster.c.googlers.com/
>

Right. I'm generally for the name change, and I like having the remote
honor the old name as a compatibility at least for some duration.

> > This is definitely a reasonable thing to want, and it has come up off
> > and on over the years. One search term for the list archive is "negative
> > refspecs", though it turns up a lot of useless hits when the two words
> > are not directly adjacent.
> >
> > This old thread might be worth reading:
> >
> >   https://lore.kernel.org/git/20140124090104.GA396@x4/
> >
> > and there is even a patch in there:
> >
> >   https://lore.kernel.org/git/20140125013433.GA22336@sigill.intra.peff.net/
> >
> > but I didn't go over it carefully enough to know whether it is utter
> > trash, or something that could be used as a starting point.
>
> I think the idea is interesting.  I sometimes find negative
> pathspecs quite useful, and I suspect negative refspecs would also
> be.

I've found negative pathspecs incredibly useful in some contexts,
(with my only issue being to remember that if you specify a negative
pathspec, you must also specify a positive one i.e. "." or something)

I think I'll try to revive that one patch Peff pointed out and then
see if it makes sense or needs to be redone.

Re: avoiding fetching specific refs from a remote

[Jeff King]

On Wed, Aug 05, 2020 at 09:22:47AM -0700, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> >> Essentially, because both main and master have similar names, tab
> >> completion requires remembering to type one additional character than
> >> I am used to.
> >
> > Heh, I didn't think anybody considered that implication so far; the fact
> > that they were similar names has generally been considered a positive. I
> > agree it would be annoying.
> 
> FWIW, I've been saying that 'maint' will become harder to type all
> along ever since I heard 'main'.
> 
> https://lore.kernel.org/git/xmqqtuz9tq30.fsf@gitster.c.googlers.com/

True, but I thought that was just a reason against "main" in git.git
itself. Most repos don't have "maint" (and we already conflicted on "ma"
with the existing name :) though I agree that having one as a prefix of
the other makes completion doubly painful). The interesting bit here is
that the old name probably _isn't_ going away immediately, so this would
affect everybody.

-Peff

Re: avoiding fetching specific refs from a remote

[Jacob Keller]

On Tue, Aug 4, 2020 at 11:37 PM Jeff King <peff@peff.net> wrote:
>
> On Tue, Aug 04, 2020 at 02:53:30PM -0700, Jacob Keller wrote:
>
> > This results in the annoying completion behavior where:
> >
> > $git checkout ma<TAB>
> > main
> > master
> >
> > Essentially, because both main and master have similar names, tab
> > completion requires remembering to type one additional character than
> > I am used to.
>
> Heh, I didn't think anybody considered that implication so far; the fact
> that they were similar names has generally been considered a positive. I
> agree it would be annoying.
>
> > I'd like to be able to customize my pull so that I either (a) exclude
> > the reference when performing a fetch, or (b) exclude the reference
> > when tab completing.
>
> Not quite (b), but if you:
>
>   - delete any local "master" branch, if you haven't already
>
>   - set $GIT_COMPLETION_CHECKOUT_NO_GUESS to "1"
>
> then it shouldn't complete "master". The second is necessary because
> we'd try to complete the magic "git checkout master" that auto-creates
> the branch from the remote version.
>

I don't like this approach because I like having the guessing options
available in most repositories.

> > something like adding a remote.<name>.excludeRefs which would allow
> > specifying a set of references to exclude from matching the refspec..
> >
> > I realize I could customize the refspec to list the set of things I
> > want, but then I would miss any potential future branches until I
> > updated my remote config again.
>
> This is definitely a reasonable thing to want, and it has come up off
> and on over the years. One search term for the list archive is "negative
> refspecs", though it turns up a lot of useless hits when the two words
> are not directly adjacent.
>
> This old thread might be worth reading:
>
>   https://lore.kernel.org/git/20140124090104.GA396@x4/
>
> and there is even a patch in there:
>
>   https://lore.kernel.org/git/20140125013433.GA22336@sigill.intra.peff.net/
>
> but I didn't go over it carefully enough to know whether it is utter
> trash, or something that could be used as a starting point.
>

Thanks, it looks like this is at least sensible.

> -Peff

Re: avoiding fetching specific refs from a remote

[Philippe Blain]

Hi Peff,

> Le 5 août 2020 à 02:37, Jeff King <peff@peff.net> a écrit :
> 
> On Tue, Aug 04, 2020 at 02:53:30PM -0700, Jacob Keller wrote:
> 
>> something like adding a remote.<name>.excludeRefs which would allow
>> specifying a set of references to exclude from matching the refspec..
>> 
>> I realize I could customize the refspec to list the set of things I
>> want, but then I would miss any potential future branches until I
>> updated my remote config again.
> 
> This is definitely a reasonable thing to want, and it has come up off
> and on over the years. One search term for the list archive is "negative
> refspecs", though it turns up a lot of useless hits when the two words
> are not directly adjacent.

I tried searching "negative refspecs" (*with* the double quotes) and I get 20 results so it 
seems to work. Although interestingly it doesn't find this message you wrote I'm responding to 
because "negative" and "refspec" are on separate lines... I'm CC-ing Eric in case he knows more
about this limitation.

Philippe.

Re: avoiding fetching specific refs from a remote

[Eric Wong]

Philippe Blain <levraiphilippeblain@gmail.com> wrote:
> > Le 5 août 2020 à 02:37, Jeff King <peff@peff.net> a écrit :
> > This is definitely a reasonable thing to want, and it has come up off
> > and on over the years. One search term for the list archive is "negative
> > refspecs", though it turns up a lot of useless hits when the two words
> > are not directly adjacent.
> 
> I tried searching "negative refspecs" (*with* the double quotes) and I get 20 results so it 
> seems to work. Although interestingly it doesn't find this message you wrote I'm responding to 
> because "negative" and "refspec" are on separate lines... I'm CC-ing Eric in case he knows more
> about this limitation.

Initially I thought it was phrase searching being disabled
on lore (Xapian 1.2, "chert" backend).  However, I just
tried on lore and it shows Jeff's message from
2020-08-05 6:37 UTC:

 https://lore.kernel.org/git/?q=%22negative+refspecs%22

So maybe it was a delay due to Xapian's MVCC mechanism;
but that shouldn't be HOURS off (maybe minutes at most),
assuming you tried to search shortly before you sent
the message I'm replying to at 22:12 UTC...

My v1 instance at public-inbox.org (which uses Xapian 1.4
and the "glass" backend) shows it, too:

 https://public-inbox.org/git/?q=%22negative+refspecs%22

You could also try NEAR and ADJ (I don't have much experience
with those): https://xapian.org/docs/queryparser.html

Re: avoiding fetching specific refs from a remote

[Jacob Keller]

On Tue, Aug 4, 2020 at 11:37 PM Jeff King <peff@peff.net> wrote:
>
> On Tue, Aug 04, 2020 at 02:53:30PM -0700, Jacob Keller wrote:
>
> > This results in the annoying completion behavior where:
> >
> > $git checkout ma<TAB>
> > main
> > master
> >
> > Essentially, because both main and master have similar names, tab
> > completion requires remembering to type one additional character than
> > I am used to.
>
> Heh, I didn't think anybody considered that implication so far; the fact
> that they were similar names has generally been considered a positive. I
> agree it would be annoying.
>
> > I'd like to be able to customize my pull so that I either (a) exclude
> > the reference when performing a fetch, or (b) exclude the reference
> > when tab completing.
>
> Not quite (b), but if you:
>
>   - delete any local "master" branch, if you haven't already
>
>   - set $GIT_COMPLETION_CHECKOUT_NO_GUESS to "1"
>
> then it shouldn't complete "master". The second is necessary because
> we'd try to complete the magic "git checkout master" that auto-creates
> the branch from the remote version.
>
> > something like adding a remote.<name>.excludeRefs which would allow
> > specifying a set of references to exclude from matching the refspec..
> >
> > I realize I could customize the refspec to list the set of things I
> > want, but then I would miss any potential future branches until I
> > updated my remote config again.
>
> This is definitely a reasonable thing to want, and it has come up off
> and on over the years. One search term for the list archive is "negative
> refspecs", though it turns up a lot of useless hits when the two words
> are not directly adjacent.
>
> This old thread might be worth reading:
>
>   https://lore.kernel.org/git/20140124090104.GA396@x4/
>
> and there is even a patch in there:
>
>   https://lore.kernel.org/git/20140125013433.GA22336@sigill.intra.peff.net/
>
> but I didn't go over it carefully enough to know whether it is utter
> trash, or something that could be used as a starting point.
>
> -Peff

I dug up the patch and have applied it to master. One concern I have
is how do we tell what side the negative refspec applies to? It seems
like we'd need to be able to distinguish whether it applies to the
refname on the remote, or the refname on the local branch. The patch
as-is assumes that negative refspecs only ever have "src".. I guess
for fetch that means the name of the ref on the remote, and for push
that means the name of the ref on the local end?

I was trying to modify the patch to add support to the push code flows
so that it was more complete and could be submitted. I'll be sending
what I have as an RFC soon.

Thanks,
Jake

Re: avoiding fetching specific refs from a remote

[Jeff King]

On Wed, Aug 12, 2020 at 03:04:09PM -0700, Jacob Keller wrote:

> I dug up the patch and have applied it to master. One concern I have
> is how do we tell what side the negative refspec applies to? It seems
> like we'd need to be able to distinguish whether it applies to the
> refname on the remote, or the refname on the local branch. The patch
> as-is assumes that negative refspecs only ever have "src".. I guess
> for fetch that means the name of the ref on the remote, and for push
> that means the name of the ref on the local end?

Yeah, unlike a normal refspec that can have both a src and dst, there
really is only one "side" to a negative refspec, because it is not going
anywhere. So selecting by src makes the most sense to me. We could also
allow selecting by dst, like:

  git fetch origin refs/heads/*:refs/remotes/origin/* \
    ^refs/heads/foo           \ (1) omit remote foo branch; implies src
    ^refs/heads/bar:          \ (2) omit remote bar branch by explicit src
    ^:refs/remotes/origin/baz   (3) omit remote baz branch by explicit dst

I probably wouldn't bother with (2) or (3) unless somebody really wants
them. We can do (1) now and then extend later without loss of
compatibility.

Another thing to think about is how to handle overlap. E.g., in:

  ^refs/heads/foo refs/heads/*:refs/remotes/origin/*

should:

  1. The first take precedence over the second because we apply
     positive, then negative?

  2. Or should it be first over second because the first is specific and
     the second is a wildcard?

  3. Or should it be second over first because later refspecs override
     earlier?

I don't have a real preference, and I think there are many
self-consistent schemes you could come up with. But it probably makes
sense to think it through so that we don't get stuck supporting a
half-baked behavior later.

> I was trying to modify the patch to add support to the push code flows
> so that it was more complete and could be submitted. I'll be sending
> what I have as an RFC soon.

Sounds good. Thanks for picking this up!

-Peff