fatal: could not reset submodule index

fatal: could not reset submodule index

[Antoine W. Campagna]

Hi,

I would like to add submodules to existing projects.
Some branches would have the submodules and some branches would not.
Since we often switch from one branch to another, I would like the submodules to update automatically so I activate the option submodule.recurse.
But I am experiencing a problem if I do the following:
  1. Clone repo, with master containing no submodules
  2. Checkout a branch that contains submodules. It results in this error:
	fatal: not a git repository: ../.git/modules/submodule
	fatal: could not reset submodule index
I think git is trying to update the submodule but the submodule has not yet been initialized (runs "git submodule update" without "--init").
Is there a way to ask git to initialize the submodule automatically ?


Here is the full reproduction instructions:

# Create a repository
mkdir main
cd main
git init
touch main.txt
git add main.txt
git commit -a -m "Initial commit"
cd ..

# Create a second repository
mkdir sub
cd sub
git init
touch sub.txt
git add sub.txt
git commit -a -m "Initial commit of repo sub"
cd ..

# Add the second repository as submodule, on a separate branch
cd main
git branch with-submodule
git checkout with-submodule
git submodule add ../sub sub
git commit -a -m "Add submodule"

# Set main repo back to master branch (without the submodule)
git checkout master
cd ..

# Make a clone and checkout the branch
git clone main clone1
cd clone1
git checkout with-submodule
# Submodule is not automatically updated (sub folder is empty)
git submodule update --init --recursive
# Now the submodule content is there
# But I want to automatically update submodules when checking out a branch
cd ..

# Trying again, adding --recursive during clone
git clone --recursive main clone2
cd clone2
git checkout with-submodule
# Submodule is still not automatically updated (sub folder is empty)
git submodule update --init --recursive
cd ..

# Trying again, adding --recurse-submodules during checkout
git clone --recursive main clone3
cd clone3
git checkout --recurse-submodules with-submodule
# Fails with these error messages :
#	fatal: not a git repository: ../.git/modules/sub
#	fatal: could not reset submodule index
# It seems like Git tries to update the submodule but without having initialized the submodule
cd ..

# Trying again with submodule.recurse
git config --global submodule.recurse true
git clone main clone4
cd clone4
git checkout with-submodule
# Submodule is still not automatically updated (sub folder is empty)
# It seems like submodule.recurse does not affect git clone

# Trying again with both submodule.recurse and --recursive
git config --global submodule.recurse true
git clone --recursive main clone5
cd clone5
git checkout with-submodule
# Fails with these error messages :
#	fatal: not a git repository: ../.git/modules/sub
#	fatal: could not reset submodule index
# Same issue as with "git checkout --recurse-submodules"

# I tested this in git-bash on Windows 10
$ git --version
	git version 2.17.1.windows.2
# And in Ubuntu in WSL
$ git --version
	git version 2.17.1

RE: fatal: could not reset submodule index

[Antoine W. Campagna]

On Wed, Jun 13, 2018 at 18:19, Antoine W. Campagna wrote:
> Here is the full reproduction instructions:

Newlines got mangled, making my message hard to read. Sorry.

Here is the corrected reproduction instructions:

# Create a repository
mkdir main
cd main
git init
touch main.txt
git add main.txt
git commit -a -m "Initial commit"
cd ..

# Create a second repository
mkdir sub
cd sub
git init
touch sub.txt
git add sub.txt
git commit -a -m "Initial commit of repo sub"
cd ..

# Add the second repository as submodule, on a separate branch
cd main
git branch with-submodule
git checkout with-submodule
git submodule add ../sub sub
git commit -a -m "Add submodule"

# Set main repo back to master branch (without the submodule)
git checkout master
cd ..

# Make a clone and checkout the branch
git clone main clone1
cd clone1
git checkout with-submodule
# Submodule is not automatically updated (sub folder is empty)
git submodule update --init --recursive
# Now the submodule content is there
# But I want to automatically update submodules when checking out a branch
cd ..

# Trying again, adding --recursive during clone
git clone --recursive main clone2
cd clone2
git checkout with-submodule
# Submodule is still not automatically updated (sub folder is empty)
git submodule update --init --recursive
cd ..

# Trying again, adding --recurse-submodules during checkout
git clone --recursive main clone3
cd clone3
git checkout --recurse-submodules with-submodule
# Fails with these error messages :
#	fatal: not a git repository: ../.git/modules/sub
#	fatal: could not reset submodule index
# It seems like Git tries to update the submodule but without having initialized the submodule cd ..

# Trying again with submodule.recurse
git config --global submodule.recurse true
git clone main clone4 cd clone4
git checkout with-submodule
# Submodule is still not automatically updated (sub folder is empty)
# It seems like submodule.recurse does not affect git clone

# Trying again with both submodule.recurse and --recursive
git config --global submodule.recurse true
git clone --recursive main clone5 cd clone5
git checkout with-submodule
# Fails with these error messages :
#	fatal: not a git repository: ../.git/modules/sub
#	fatal: could not reset submodule index
# Same issue as with "git checkout --recurse-submodules"

Re: fatal: could not reset submodule index

[Stefan Beller]

> # Trying again with submodule.recurse
> git config --global submodule.recurse true
> git clone main clone4 cd clone4
> git checkout with-submodule
> # Submodule is still not automatically updated (sub folder is empty)
> # It seems like submodule.recurse does not affect git clone

That is true; submodule.recurse is not affecting git clone.
This was a design decision once it was introduced, as the git clone
might be too large. Maybe we need to revisit that decision and
just clone the submodules if submodule.recurse is set.

The problem that this bug report highlights, is the following:
Each branch has its own .gitmodules file and they can be
all different, however at clone time we only clone submodules
from the currently active branch.

So you could get your test case to pass with
    git clone --branch with-submodule main clone5

RE: fatal: could not reset submodule index

[Antoine W. Campagna]

> That is true; submodule.recurse is not affecting git clone.
> This was a design decision once it was introduced, as the git clone might be too large. Maybe we need to revisit that decision and just clone the submodules if submodule.recurse is set.
>
> The problem that this bug report highlights, is the following:
> Each branch has its own .gitmodules file and they can be all different, however at clone time we only clone submodules from the currently active branch.
>
> So you could get your test case to pass with
>     git clone --branch with-submodule main clone5
> 

Thank you for your input Stefan.

But here is our situation : 
7 different projects in separate repositories
Big parts of the code is similar between the projects
We want to extract these similar portions in separate repositories and add them as submodules (maybe about 10 of them).
We would do this progressively, adding one submodule at a time.
We need to support older versions so we have to keep branches that will not have submodules.
Since we need to checkout these branches regularly, we would want to have submodule.recurse enabled.
With this setup, every time a submodule is added in one of the projects, each developer would get the "fatal: could not reset submodule index" error and would be unable to checkout master branch.