Issue in switching branches with different submodules

Issue in switching branches with different submodules

[Christian.Zitzmann]

Hello,
I've found an issue in git when working with submodules.
My Project is set up using hundreds of components by submodules (and 
nested submodules), that are independent created in central development 
groups.
Now it occurs that the structure of the submodules is changed over time.

E.g.
Project1(OldBranch)
  - ComponentX/SubComp1 -> ssh://Server/ComponentX/SubComp1
  - ComponentX/SubComp2 -> ssh://Server/ComponentX/SubComp2
  - ComponentX/SubComp2 -> ssh://Server/ComponentX/SubComp2

Project1(Masster)
  - ComponentX-> ssh://Server/ComponentX

There is both a repository for the old subcomponents, and for the new 
Component on the server.

When trying to switch between the branches all git commands are failing.
It seems like most git commands are not creating the SubComponent 
submodules because the .git file from the Component is not deleted

A 'git submodule sync' fails with:
fatal: Not a git repository: 
D:/Project1/ComponentX/../.git/modules/ComponentX

Looking into the folders I see:
D:/Project1/.git/modules/ComponentX/SubComp1
D:/Project1/.git/modules/ComponentX/SubComp2
D:/Project1/.git/modules/ComponentX/SubComp3
D:/Project1/ComponentX/.git (file)

A 'git submodule update --init also fails with this folders
Neither a forced checkout or a hard reset is working.

Similar errors can occur when switching between branches with a different 
number of components used as submodules vs. project specific content.
As a result it happens that people are working with an incosistend state 
of the working directory.

My expectation would be that, when switching between branches/commits with 
a git checkout --recurse-submodules the branche including all submodules 
is checked out fully to exactly the state of the desired branch/commit
I.e the following usecases are an issue
- Deleted submodule
- Added submodule as replacement of local directory
- Changed remote location of submodule (One component is available from 
different providers having individual repositories)
- Restructured Component (see example)

Similar issues are existing when creating the new structure to commit it, 
but here the error is more obvious and people are manually deleting as 
much as required to get the new submodules correctly in.

Best regards

Christian Zitzmann


P HEV E SW SF PMT

Continental Automotive GmbH
Siemensstrasse 12, 93055 Regensburg, Germany

Telefon/Phone: +49 941 790-7265
E-Mail: Christian.Zitzmann@continental-corporation.com

Re: Issue in switching branches with different submodules

[Stefan Beller]

On Thu, Feb 22, 2018 at 4:10 AM,
<Christian.Zitzmann@continental-corporation.com> wrote:
> Hello,
> I've found an issue in git when working with submodules.
> My Project is set up using hundreds of components by submodules (and
> nested submodules), that are independent created in central development
> groups.
> Now it occurs that the structure of the submodules is changed over time.
>
> E.g.
> Project1(OldBranch)
>   - ComponentX/SubComp1 -> ssh://Server/ComponentX/SubComp1
>   - ComponentX/SubComp2 -> ssh://Server/ComponentX/SubComp2
>   - ComponentX/SubComp2 -> ssh://Server/ComponentX/SubComp2
>
> Project1(Masster)
>   - ComponentX-> ssh://Server/ComponentX
>
> There is both a repository for the old subcomponents, and for the new
> Component on the server.

ok, so you're saying this is all a client side problem?

> When trying to switch between the branches all git commands are failing.
> It seems like most git commands are not creating the SubComponent
> submodules because the .git file from the Component is not deleted
>
> A 'git submodule sync' fails with:
> fatal: Not a git repository:
> D:/Project1/ComponentX/../.git/modules/ComponentX
>
> Looking into the folders I see:
> D:/Project1/.git/modules/ComponentX/SubComp1
> D:/Project1/.git/modules/ComponentX/SubComp2
> D:/Project1/.git/modules/ComponentX/SubComp3
> D:/Project1/ComponentX/.git (file)

As a quick workaround to repair your current corrupted repo,
you can just delete the .git file, which presumably contains

  gitdir: ../.git/modules/ComponentX

I think this reveals (yet another) problem that we have with
with submodule names. Submodules used to have its git directory
inside its own working tree, but now they are encouraged to have
it in the superproject in .git/modules/<name>.

Considering your submodules
    "ComponentX/..."
    "ComponentX"
you may have a directory conflict, because there is already a
directory named "ComponentX" so we cannot store the later
submodules git directory.

> A 'git submodule update --init also fails with this folders
> Neither a forced checkout or a hard reset is working.

For this problem you could manually repair the superproject by
giving better names to submodules, such that you have unique
names, such that one is not the prefix directory of another.

> Similar errors can occur when switching between branches with a different
> number of components used as submodules vs. project specific content.
> As a result it happens that people are working with an incosistend state
> of the working directory.
>
> My expectation would be that, when switching between branches/commits with
> a git checkout --recurse-submodules the branche including all submodules
> is checked out fully to exactly the state of the desired branch/commit
> I.e the following usecases are an issue
> - Deleted submodule
> - Added submodule as replacement of local directory
> - Changed remote location of submodule (One component is available from
> different providers having individual repositories)
> - Restructured Component (see example)

I agree with this expectation and we'd need to discuss how to make this happen
as the submodule naming scheme doesn't allow for this to work flawlessly.

> Similar issues are existing when creating the new structure to commit it,
> but here the error is more obvious and people are manually deleting as
> much as required to get the new submodules correctly in.
>
> Best regards

Thanks for this bug report; I needed some time to ingest it and come up
with speculation on what might be the cause. (Not sure if it is the correct
root cause though)

Thanks,
Stefan