Re: fatal: cannot rebase with locally recorded submodule modifications

[Philippe Blain <levraiphilippeblain@gmail.com>]

Hi Shourya,

> Le 7 févr. 2020 à 17:04, Shourya Shukla <shouryashukla.oo@gmail.com> a écrit :
> 
> Hello Brice,
> 
> The error you are facing here is actually a bug in the
> 'git pull' command[1]. When one tries to use the
> '--recurse-submodules' tag with the command, it tends to fail.

Let's not discourage our users ;) It is untrue that it tends to fail.

>> Why is failing here?
> 
> Going into a bit detail of 'git pull', the command actually
> is a combination of 'git fetch' and 'git merge'(unless any
> other tag is specified i.e. 'rebase' here which would mean
> a fetch followed by a rebase). Quoting from the documentation:
> 
>> Using --recurse-submodules can only fetch new commits in already
> checked out submodules right now
> 
> Using the '--recurse-submodules' tag will try merging/rebasing any
> changes(updations, additions, deletions, etc.) made to the fetched
> submodule(s) from upstream, i.e. the _un-checked out_ submodules into
> our local repository which will cause the error to be thrown as it is
> not supported right now.

That part of the doc talks about the fact that when using '--recurse-submodules', only commits
in already checked out submodules ("populated submodules" is the wording used in the docs 
for that option) will be *fetched*. If you take a look at, for example,

    git blame -L 240,+10 Documentation/git-pull.txt

you'll see that this section was added in 794a3592ad (fetch/pull: Describe 
--recurse-submodule restrictions in the BUGS section, 2011-03-06), when 'git fetch'
learned the '--recurse-submodules' flag. At the time 'git pull --recurse-submodules' 
did *not* update the working tree of the submodules, it just passed '--recurse-submodules' 
to the underlying 'git fetch'. 'git pull' learned to update the working tree of the submodules
(and also rebase the submodules themselves under very specific circumstances) in 
a6d7eb2c7a (pull: optionally rebase submodules (remote submodule changes only), 
2017-06-23). 

So this "bug" you mention is not what is happening here. Since Brice clone with 
'--recurse-submodules', all submodules will be cloned, initialized and populated as 
part of the superproject clone, so they are all already checked out when 'git pull' is invoked.

Also, the "Bugs" section in the docs that you refer to does not talk about the command
erroring; it just describes a limitation of 'git fetch --recurse-submodules'. And as a side note,
there is no test in the test suite that checks this behaviour (looking at t5526-fetch-submodules.sh
and t5572-pull-submodule.sh). It would be good to fix that by adding a couple 'test_expect_failure' 
tests, or even to fix the behaviour itself. 

Note also that when this "Bugs" section was added, the submodule repositories 
were cloned directly in the superproject working tree instead of being cloned to 
.git/modules/<submodule-name>. This explains why this limitation existed at the time:
you can't fetch the submodules if there is no filesystem location to put their Git repository!
A few months later, 501770e1bb (Move git-dir for submodules, 2011-08-15), implemented 
the "new-style" submodule setup, i.e. cloning the submodules repositories inside the Git repository
of the superproject, as is standard these days. So there is no good reason today to not fix this 
bug, in my opinion.

Going back to Brice's bug:

> The "workaround" is to do a 'git fetch' and then merge again after the
> 'git pull <tag> --recurse-submodules'.

I don't think there is a workaround until my fixes get merged, apart from
not using '--recurse-submodules' with 'git pull --rebase'. Of course then you 
have to remember to *always* do 'git submodule update --recursive' after 
each pull if you want your submodule checkouts to stay in sync with the commits
recorded in the superproject.

> The presence/absence of 'git reset --hard' before the pull  won't create
> any difference to this error in my opinion.

On the contrary: as I wrote in my answer to Brice, it is exactly because 'git reset --hard' is used 
here that the code wrongly errors.

Cheers, and thank you for all the work you put in the rewrite of the submodule code
from shell to C this summer.

Philippe.