`git fetch` not updating 'origin/HEAD' after branch rename

`git fetch` not updating 'origin/HEAD' after branch rename

[Sam Bostock]

Hi!

Hopefully I followed the instructions on https://git-scm.com/community
correctly to report this bug.

Long story short, it seems to me that `git fetch` should update
"refs/remotes/origin/HEAD" when the upstream HEAD changes, but it
doesn't. See my filled out bug report below.

Thanks,

Sam

----

Thank you for filling out a Git bug report!
Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)

1. Clone a repo with some branch name (say "master") from some remote
origin (say GitHub).
2. Replace/rename the main branch on the remote (say "main").
3. Run `git fetch` locally to sync with remote.
4. Run `git rev-parse --abbrev-ref --verify origin/HEAD`.

What did you expect to happen? (Expected behavior)

The `git fetch` should update the refs accordingly so "origin/HEAD"
now points to
the new branch, which should be output from the command above:

    $ git rev-parse --abbrev-ref --verify origin/HEAD
    origin/main

What happened instead? (Actual behavior)

The "origin/HEAD" ref seems to not point to anything anymore (because the
branch it was pointing to was "deleted"). `git fetch` fails to update
it to reflect the change.

    $ git rev-parse --abbrev-ref --verify origin/HEAD
    fatal: Needed a single revision

What's different between what you expected and what actually happened?

The HEAD on origin has changed, but this change is not reflected in
the local refs by `git fetch`.

Anything else you want to add:

Please review the rest of the bug report below.
You can delete any lines you don't wish to share.


[System Info]
git version:
git version 2.30.2
cpu: x86_64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8
shell-path: /nix/store/74shlfgb18717ixjlpivpxd7iqcyhyn5-bash-4.4-p23/bin/bash
uname: Darwin 19.6.0 Darwin Kernel Version 19.6.0: Tue Jan 12 22:13:05
PST 2021; root:xnu-6153.141.16~1/RELEASE_X86_64 x86_64
compiler info: clang: 7.1.0 (tags/RELEASE_710/final)
libc info: no libc information available
$SHELL (typically, interactive shell): /usr/local/bin/bash


[Enabled Hooks]

Re: `git fetch` not updating 'origin/HEAD' after branch rename

[Junio C Hamano]

Sam Bostock <sam.bostock@shopify.com> writes:

> Hopefully I followed the instructions on https://git-scm.com/community
> correctly to report this bug.
>
> Long story short, it seems to me that `git fetch` should update
> "refs/remotes/origin/HEAD" when the upstream HEAD changes, but it
> doesn't. See my filled out bug report below.

That is working as intended.

Since its inception, refs/remotes/origin/HEAD has been designed to
mean "the branch, which the owner of the repository considers the
most interesting for his/her purpose to keep track in the 'origin'
remote repository".  It is perfectly OK to repoint origin/HEAD to
point at another branch you are interested in after you make a
clone.

The origin/HEAD happens to be primed by noticing what the remote
repository sets HEAD to point to their branch upon cloning, but that
is purely for convenience (i.e. HEAD branch of a repository offered
for cloning points at a branch the owner of the repository being
cloned considers the primary branch in his/her repository, and that
often coincides with the earlier definition of origin/HEAD for those
who clone such a repository).  It would be a bug to update it to
something other than you set (either passively by keeping what
"clone" gave you initially, or actively by updating it to point at
another branch) upon "git fetch" from the remote.

There was an idea floated to use the same mechanism used by "git
clone" to prime origin/HEAD upon "git fetch" when origin/HEAD is
missing, and I do not think there were many objections against the
idea, so that may happen sometime in the future.  It may also be
possible to add "git fetch --reset-remote-tracking-HEAD" option to
make "fetch" overwrite existing origin/HEAD but somebody has to
propose such a change, argue for its benefit and get it accepted by
the community.

Thanks.

Re: `git fetch` not updating 'origin/HEAD' after branch rename

[Andreas Schwab]

On Apr 13 2021, Junio C Hamano wrote:

> It may also be
> possible to add "git fetch --reset-remote-tracking-HEAD" option to
> make "fetch" overwrite existing origin/HEAD but somebody has to
> propose such a change, argue for its benefit and get it accepted by
> the community.

Isn't that what git remote set-head origin -a does?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: `git fetch` not updating 'origin/HEAD' after branch rename

[Junio C Hamano]

Andreas Schwab <schwab@linux-m68k.org> writes:

> On Apr 13 2021, Junio C Hamano wrote:
>
>> It may also be
>> possible to add "git fetch --reset-remote-tracking-HEAD" option to
>> make "fetch" overwrite existing origin/HEAD but somebody has to
>> propose such a change, argue for its benefit and get it accepted by
>> the community.
>
> Isn't that what git remote set-head origin -a does?

Yes, but the point is to do so when fetch is run (i.e. it talks with
the other side not just to learn which branch their HEAD points at,
but it actually does the fetching of new history on branches).  And
it might open the door to add a configuration to tell "fetch" to do
so unconditionally, if it turns out to be useful enough.

Re: `git fetch` not updating 'origin/HEAD' after branch rename

[Chris Torek]

On Tue, Apr 13, 2021 at 5:18 PM Sam Bostock <sam.bostock@shopify.com> wrote:
> Long story short, it seems to me that `git fetch` should update
> "refs/remotes/origin/HEAD" when the upstream HEAD changes, but it
> doesn't.

It's never been the *intent* to have `git fetch` update
the corresponding remote-tracking `HEAD` ref.  To make
that happen, you must run `git remote`:

    git remote set-head origin -a

for instance.

I have, however, often thought that this is the wrong
*default* way for things to work, and that at least by default,
`git fetch origin` should update `origin/HEAD` if the
fetch result indicates that it should.  See also Junio's
reply.  I think a configuration knob (similar to `fetch.prune`)
would be reasonable here.  Users could then be encouraged
to set `fetch.prune` to `true`, and `fetch.update-remote-HEAD`
(or whatever) to `true` as well.

Chris

Re: `git fetch` not updating 'origin/HEAD' after branch rename

[Jeff King]

On Tue, Apr 13, 2021 at 05:26:02PM -0700, Chris Torek wrote:

> On Tue, Apr 13, 2021 at 5:18 PM Sam Bostock <sam.bostock@shopify.com> wrote:
> > Long story short, it seems to me that `git fetch` should update
> > "refs/remotes/origin/HEAD" when the upstream HEAD changes, but it
> > doesn't.
> 
> It's never been the *intent* to have `git fetch` update
> the corresponding remote-tracking `HEAD` ref.  To make
> that happen, you must run `git remote`:
> 
>     git remote set-head origin -a
> 
> for instance.
> 
> I have, however, often thought that this is the wrong
> *default* way for things to work, and that at least by default,
> `git fetch origin` should update `origin/HEAD` if the
> fetch result indicates that it should.  See also Junio's
> reply.  I think a configuration knob (similar to `fetch.prune`)
> would be reasonable here.  Users could then be encouraged
> to set `fetch.prune` to `true`, and `fetch.update-remote-HEAD`
> (or whatever) to `true` as well.

I think so, too. The previous discussion Junio referred to it is here:

  https://lore.kernel.org/git/20201118091219.3341585-1-felipe.contreras@gmail.com/

If you do read it, note that the parts of the conversation about unborn
heads are out of date. We do pick this up via clone these days, courtesy
of the commits in 69571dfe21 (Merge branch 'jt/clone-unborn-head',
2021-02-17).

-Peff

Re: `git fetch` not updating 'origin/HEAD' after branch rename

[Ævar Arnfjörð Bjarmason]

On Wed, Apr 14 2021, Chris Torek wrote:

> On Tue, Apr 13, 2021 at 5:18 PM Sam Bostock <sam.bostock@shopify.com> wrote:
>> Long story short, it seems to me that `git fetch` should update
>> "refs/remotes/origin/HEAD" when the upstream HEAD changes, but it
>> doesn't.
>
> It's never been the *intent* to have `git fetch` update
> the corresponding remote-tracking `HEAD` ref.  To make
> that happen, you must run `git remote`:
>
>     git remote set-head origin -a
>
> for instance.
>
> I have, however, often thought that this is the wrong
> *default* way for things to work, and that at least by default,
> `git fetch origin` should update `origin/HEAD` if the
> fetch result indicates that it should.  See also Junio's
> reply.  I think a configuration knob (similar to `fetch.prune`)
> would be reasonable here.  Users could then be encouraged
> to set `fetch.prune` to `true`, and `fetch.update-remote-HEAD`
> (or whatever) to `true` as well.

As you'll see from the previous linked thread I happen to agree with
you, but let's not step past that agreement and (no doubt subtly and
unintentionally, in this case) misrepresent the other side, which also
has a legitimate argument.

Which is that this notion that "[a client should update its view of a
remote to what a] fetch result indicates that it should" isn't something
that exists, and that we ever had it is only an emergent implementation
detail.

So first of all, a "fetch result" doesn't look anything like
that. Rather the server presents refs it has, and the client decides
what it wants.

Which is the start of the disconnect in mental models around this. What
you see with "git ls-remote origin" != what you should expect to have at
a local refs/remotes/origin/* after a fetch.

Nothing in git itself actually needs this remote HEAD past clone time,
and with 4f37d457065 (clone: respect remote unborn HEAD, 2021-02-05)
there'll be even less reason to pay attention to it.

We don't even always get it on "clone", we don't do it with "--bare",
since the reason we do it is to one-time setup the default branch, which
we then hardcode in the config in the non-bare case:

    [remote "origin"]
            url = https://github.com/git/git.git
            fetch = +refs/heads/*:refs/remotes/origin/*
    [branch "master"]
            remote = origin
            merge = refs/heads/master

So arguably the bug is the other way around, that we should never save
this information at all in the ref store at "clone" time, which would
make "init && config && fetch" and "clone" consistent.

I haven't looked, but wouldn't be surprised to discover that this was
originally needed back when we needed to ferry information between
shellscripts, and was just diligently retained when the relevant parts
were rewritten in C.

But I digress.

Now, what people *do actually* legitimately use this information is a
convenient local cache of "what's the main upstream branch?".

I myself have written local scripts that used that, and been bitten by
this information being stale / not there (in the case of
init/config/fetch).

But why does anyone need that? Well, I don't think they actually
do. What they actually do want is to push or fetch the "main"
branch. The "what was HEAD that one time I talked to this remote" is
just a roundabout way of getting that.

So isn't this whole thing just wart that we should fix in the protocol?
Wouldn't the use-case for this be satisfied with something like:

    [branch "master"]
            remote = origin
            merge = HEAD ; does not work as you might expect

Right now that will just push to refs/heads/HEAD, but what if we had a
protocol extension to intercept it (or rather, some merge = <a name
incompatible with a current push, maybe "$HEAD">, as an aside setting it
to ":HEAD" has some very funny results) wouldn't that satisfy the
use-case?

After all, who's really interested in what the remote's idea of their
HEAD when they last fetched is?

Don't those users actually want the *current* idea of what HEAD is for
the purposes of fetching or pushing?

If we supported "I want the tip of your HEAD" (which we do) or "I have
this ref update for you, on top of your idea of HEAD" (we don't) you
could "git push" and it would do the right thing whether the primary
branch was renamed from "master" to "main" during your "git push" or
not.

Re: `git fetch` not updating 'origin/HEAD' after branch rename

[Andreas Schwab]

On Apr 14 2021, Ævar Arnfjörð Bjarmason wrote:

> Nothing in git itself actually needs this remote HEAD past clone time,
> and with 4f37d457065 (clone: respect remote unborn HEAD, 2021-02-05)
> there'll be even less reason to pay attention to it.

Isn't that what git rev-parse origin resolves to?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: `git fetch` not updating 'origin/HEAD' after branch rename

[Junio C Hamano]

Andreas Schwab <schwab@linux-m68k.org> writes:

> On Apr 14 2021, Ævar Arnfjörð Bjarmason wrote:
>
>> Nothing in git itself actually needs this remote HEAD past clone time,
>> and with 4f37d457065 (clone: respect remote unborn HEAD, 2021-02-05)
>> there'll be even less reason to pay attention to it.
>
> Isn't that what git rev-parse origin resolves to?

Correct.

And that is why refs/remotes/origin/HEAD is defined to point at the
branch that the owner of LOCAL repository considers is of interest,
which may not be the same as what the owner of the REMOTE repository
thinks is the primary branch in their worldview.

Re: `git fetch` not updating 'origin/HEAD' after branch rename

[Jeff King]

On Wed, Apr 14, 2021 at 12:56:27PM +0200, Ævar Arnfjörð Bjarmason wrote:

> Now, what people *do actually* legitimately use this information is a
> convenient local cache of "what's the main upstream branch?".
> 
> I myself have written local scripts that used that, and been bitten by
> this information being stale / not there (in the case of
> init/config/fetch).
> 
> But why does anyone need that? Well, I don't think they actually
> do. What they actually do want is to push or fetch the "main"
> branch. The "what was HEAD that one time I talked to this remote" is
> just a roundabout way of getting that.
> 
> So isn't this whole thing just wart that we should fix in the protocol?
> Wouldn't the use-case for this be satisfied with something like:
> 
>     [branch "master"]
>             remote = origin
>             merge = HEAD ; does not work as you might expect
> 
> Right now that will just push to refs/heads/HEAD, but what if we had a
> protocol extension to intercept it (or rather, some merge = <a name
> incompatible with a current push, maybe "$HEAD">, as an aside setting it
> to ":HEAD" has some very funny results) wouldn't that satisfy the
> use-case?

Keep in mind that if we do not have a cache, then finding out the remote
HEAD involves a network trip. But we use branch.*.merge in lots of
places that aren't fetch/push. E.g., how would "git rev-parse
@{upstream}" work with the config you gave above?

> After all, who's really interested in what the remote's idea of their
> HEAD when they last fetched is?
> 
> Don't those users actually want the *current* idea of what HEAD is for
> the purposes of fetching or pushing?

IMHO the cache of the remote HEAD in refs/remotes/origin/HEAD is the
same as the cache of the remote refs in refs/remotes/origin/*. We only
talk to the network during fetch/push operations, but it is convenient
to have a local cache that names the things we saw there.

As others mentioned, "git rev-parse origin" is another local command
that works without touching the network.

So I find the local cache of the remote HEAD quite useful. I think the
only issue with it is that it is not kept up to date like the rest of
the branches in refs/remotes/origin/*. That was an intentional decision,
because you may want to define your own view of what is considered the
default for the remote.

But that is a much rarer case than people who do want to auto-update it
on fetch (and for many years nobody cared too much, because people
rarely updated HEAD anyway; but these days there's quite a lot of
renaming going on). So it makes sense to at least provide a configurable
option to allow updating it on each fetch.

-Peff