Re: Bug in git submodule update --remote

[Philippe Blain <levraiphilippeblain@gmail.com>]

Hi Ben,

Le 2021-05-14 à 14:11, Ben Avison a écrit :
> A recent-ish change in git 2.27.1, introduced in commit f0a96e8d, has also got me wondering about whether some related functionality is correct. I'm not sure what the best way to fix things is, so can I invite opinions?
> 
> The scenario: I have a repository with a submodule. The submodule's upstream repository uses a fork workflow, so the submodule has two remotes, one for pulling in other people's changes, and one for uploading my own pull requests.
> 
> $ mkdir myproject
> $ cd myproject
> $ git init
> $ git submodule add https://github.com/acme/library
> $ cd library
> $ git remote add -f myfork https://github.com/user/library
> $ cd ..
> $ git add .
> $ git commit -m "Initial commit"
> $ git remote add origin https://github.com/user/myproject
> $ git push -u origin master
> 
> Furthermore, assume I forked https://github.com/acme/example some time ago, so that the master branches between it and my fork have diverged.
> 
> Time passes. People hack away on both projects. I want to fix a bug or implement a new feature in the library, so I start by ensuring both are up-to-date:
> 
> $ git pull
> $ git submodule update --remote
> $ cd library
> $ git checkout -b new-feature
> $ # hack away
> $ git add .
> $ git commit -m "New feature"
> $ git push -u myfork new-feature

Starting on a slight tangent, I know the 'git push -u $remote $branch' command
is often mentioned in tutorials, but since you are using the forking workflow,
I would suggest looking into the 'remote.pushDefault' / branch.$.pushRemote configs [1]. This allows you
to track upstream/master on your feature branches (i.e. the branch into which your feature
will eventually be integrated), and to push directly to your fork with a simple 'git push' :)

> $ cd ..
> 
> Some more time passes, and I want to work on it again. Again, I start by ensuring I'm up-to-date. Before git 2.27.1, I could do:
> 
> $ git submodule update --remote
> 
> Now, I get:
> 
> fatal: Needed a single revision
> Unable to find current myfork/HEAD revision in submodule path 'library'

That's a shame. Dscho's f0a96e8d4c (submodule: fall back
to remote's HEAD for missing remote.<name>.branch, 2020-06-24) did not touch
git-sumodule.sh, so when I took a look at it in [3] I did not think of going through
the code path...

> 
> What's going on is that within "git submodule update --remote", the sha1 used is formed by looking up the submodule's ref "<remote>/<branch>". The change in git 2.27.1 is that if no remote tracking branch is stated in the superproject's .gitmodules file, <branch> defaults to "HEAD" rather than "master" as previously. That's fine if <remote> is "origin", but in practice, it depends on how the submodule is currently checked out:
> 
> * if it's in detached HEAD state, <remote> is set to "origin"
> * if its branch is not a tracking branch, "origin" is also used
> * but if it's a tracking branch (as I used in my workflow above - not uncommon in pull request workflows because you might grant other people access to the branch during the review process) then it looks up the name of the remote which is being tracked
> 

Yes, that's indeed the behaviour of submodule--helper.c::get_default_remote,
which is called by submodule--helper.c::print_default_remote, i.e.
'git submodule--helper print-default-remote', which is called by
git-submodule.sh::cmd_update (line 580 as of v2.32.0-rc1).

> First observation: a ref called "myfork/HEAD" presumably *could* have been created by the "git remote add" command, reflecting that remote's default branch. Should it?

I think that's what happens with the '-m' switch, i.e. 'git remote add -m master myfork $url'.
The symref $remote/HEAD can also be created after adding the remote
with 'git remote set-head myfork --auto' [2], which will
query the remote's real default branch (i.e. the remote HEAD) and set the local symref accordingly.
The fact that this is not done automatically upon
'git remote add' (or git fetch) is a historical mistake in my opinion but some others might disagree.

> 
> In practice, that's not actually what I'd want, though. If I do "git submodule update --remote", I personally normally do so as a shortcut compared to cloning everything again. I don't expect the behaviour to change depending on whether I happen to have left one of the submodules checked out on a tracking branch or not: myfork/master is potentially quite out of date compared to origin/master.

I completely agree.

> 
> It also makes very little sense to me that issuing "git submodule update --remote" twice in quick succession would leave us in a different state, because the first call puts all the submodules into detached HEAD state, meaning that the second call always looks up the remote tracking branches from the submodule's "origin" remote.

I also agree.

> 
> One way this could be fixed is that if <branch> turns out to be "HEAD", we could force <remote> to be "origin". However, this doesn't address the equivalent problem that arises if the remote tracking branch *is* named in .gitmodules.
> 
> I'd therefore like to propose that "git submodule update --remote" *always* uses the remote named "origin" to form the ref used to check out the submodule. However, I'm not sure whether everyone would agree that this constitutes a bugfix, or whether I'd need to introduce a new switch to enforce this behaviour.
>

I prefer introducing a new config variable, as you suggest in your patch
downthread. I'll comment there with more details, but I do not like the
approach of hard-coding 'origin' with no escape hatch. Personnally I use
'origin' for my fork and 'upstream' for the autoritative repository in all
my clones, and I can't do this in submodules because of the fact that 'origin'
is hard-coded at few places in the submodule code, so let't not hard code it yet
one more time.

Cheers,

Philippe.


[1] https://github.blog/2015-07-29-git-2-5-including-multiple-worktrees-and-triangular-workflows/#improved-support-for-triangular-workflows
[2] http://git-scm.com/docs/git-remote#Documentation/git-remote.txt-emset-headem
[3] https://lore.kernel.org/git/D2ED942B-9397-472A-B017-190016531547@gmail.com/