[BUG] `git reset --hard` fails with `update = none` submodules

[BUG] `git reset --hard` fails with `update = none` submodules

[Rose Kunkel]

# What did you do before the bug happened? (Steps to reproduce your issue)
1. Clone a git repository that sets `update = none` in .gitmodules:
$ git clone --recurse-submodules https://github.com/ubolonton/tree-sitter-langs

2. Perform a hard reset:
$ cd tree-sitter-langs
$ git reset --hard

# What did you expect to happen? (Expected behavior)
The reset should succeed and do nothing.

# What happened instead? (Actual behavior)
The reset command fails with
```
fatal: not a git repository: ../../.git/modules/repos/agda
fatal: could not reset submodule index
```

[System Info]
git version:
git version 2.32.0
cpu: x86_64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
uname: Linux 5.12.10-hardened1-1-hardened #1 SMP PREEMPT Thu, 10 Jun
2021 21:12:42 +0000 x86_64
compiler info: gnuc: 11.1
libc info: glibc: 2.33
$SHELL (typically, interactive shell): /usr/bin/zsh

Re: [BUG] `git reset --hard` fails with `update = none` submodules

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1436 bytes --]

On 2021-06-16 at 00:16:06, Rose Kunkel wrote:
> # What did you do before the bug happened? (Steps to reproduce your issue)
> 1. Clone a git repository that sets `update = none` in .gitmodules:
> $ git clone --recurse-submodules https://github.com/ubolonton/tree-sitter-langs
> 
> 2. Perform a hard reset:
> $ cd tree-sitter-langs
> $ git reset --hard
> 
> # What did you expect to happen? (Expected behavior)
> The reset should succeed and do nothing.

I think we're in agreement on this.  This should be a fresh clone and so
a hard reset should change nothing.

> # What happened instead? (Actual behavior)
> The reset command fails with
> ```
> fatal: not a git repository: ../../.git/modules/repos/agda
> fatal: could not reset submodule index
> ```

Hmmm, I can't reproduce this behavior.  What I see is this:

  $ git reset --hard
  HEAD is now at 5d362ce Release 0.10.0

I'm running git version 2.32.0.272.g935e593368 on Debian sid (with the
experimental packages).

Can you try the clone and run a "git status" command in the repository
to see if anything is modified after your clone?  Are the submodules
checked out when you perform the clone?  In my case, I see lines like
this:

  Skipping submodule 'repos/agda'

If you're seeing something different, then that might contribute to the
different behavior we're seeing.
-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [BUG] `git reset --hard` fails with `update = none` submodules

[Rose Kunkel]

Running `git status` in the resulting repository gives
```
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
```

This is the output from the clone command:
```
Cloning into 'tree-sitter-langs'...
remote: Enumerating objects: 609, done.
remote: Counting objects: 100% (83/83), done.
remote: Compressing objects: 100% (52/52), done.
remote: Total 609 (delta 40), reused 58 (delta 24), pack-reused 526
Receiving objects: 100% (609/609), 117.17 KiB | 1.05 MiB/s, done.
Resolving deltas: 100% (322/322), done.
Submodule 'repos/agda'
(https://github.com/tree-sitter/tree-sitter-agda) registered for path
'repos/agda'
Submodule 'repos/bash'
(https://github.com/tree-sitter/tree-sitter-bash) registered for path
'repos/bash'
Submodule 'repos/c' (https://github.com/tree-sitter/tree-sitter-c)
registered for path 'repos/c'
Submodule 'repos/c-sharp'
(https://github.com/tree-sitter/tree-sitter-c-sharp) registered for
path 'repos/c-sharp'
Submodule 'repos/cpp' (https://github.com/tree-sitter/tree-sitter-cpp)
registered for path 'repos/cpp'
Submodule 'repos/css' (https://github.com/tree-sitter/tree-sitter-css)
registered for path 'repos/css'
Submodule 'repos/elm' (https://github.com/razzeee/tree-sitter-elm)
registered for path 'repos/elm'
Submodule 'repos/fluent'
(https://github.com/tree-sitter/tree-sitter-fluent) registered for
path 'repos/fluent'
Submodule 'repos/go' (https://github.com/tree-sitter/tree-sitter-go)
registered for path 'repos/go'
Submodule 'repos/html'
(https://github.com/tree-sitter/tree-sitter-html) registered for path
'repos/html'
Submodule 'repos/janet-simple'
(https://codeberg.org/sogaiu/tree-sitter-janet-simple) registered for
path 'repos/janet-simple'
Submodule 'repos/java'
(https://github.com/tree-sitter/tree-sitter-java) registered for path
'repos/java'
Submodule 'repos/javascript'
(https://github.com/tree-sitter/tree-sitter-javascript) registered for
path 'repos/javascript'
Submodule 'repos/jsdoc'
(https://github.com/tree-sitter/tree-sitter-jsdoc) registered for path
'repos/jsdoc'
Submodule 'repos/json'
(https://github.com/tree-sitter/tree-sitter-json) registered for path
'repos/json'
Submodule 'repos/julia'
(https://github.com/tree-sitter/tree-sitter-julia) registered for path
'repos/julia'
Submodule 'repos/ocaml'
(https://github.com/tree-sitter/tree-sitter-ocaml) registered for path
'repos/ocaml'
Submodule 'repos/php' (https://github.com/tree-sitter/tree-sitter-php)
registered for path 'repos/php'
Submodule 'repos/python'
(https://github.com/tree-sitter/tree-sitter-python) registered for
path 'repos/python'
Submodule 'repos/ruby'
(https://github.com/tree-sitter/tree-sitter-ruby) registered for path
'repos/ruby'
Submodule 'repos/rust'
(https://github.com/tree-sitter/tree-sitter-rust) registered for path
'repos/rust'
Submodule 'repos/scala'
(https://github.com/tree-sitter/tree-sitter-scala) registered for path
'repos/scala'
Submodule 'repos/swift'
(https://github.com/tree-sitter/tree-sitter-swift) registered for path
'repos/swift'
Submodule 'repos/typescript'
(https://github.com/tree-sitter/tree-sitter-typescript) registered for
path 'repos/typescript'
Skipping submodule 'repos/agda'
Skipping submodule 'repos/bash'
Skipping submodule 'repos/c'
Skipping submodule 'repos/c-sharp'
Skipping submodule 'repos/cpp'
Skipping submodule 'repos/css'
Skipping submodule 'repos/elm'
Skipping submodule 'repos/fluent'
Skipping submodule 'repos/go'
Skipping submodule 'repos/html'
Skipping submodule 'repos/janet-simple'
Skipping submodule 'repos/java'
Skipping submodule 'repos/javascript'
Skipping submodule 'repos/jsdoc'
Skipping submodule 'repos/json'
Skipping submodule 'repos/julia'
Skipping submodule 'repos/ocaml'
Skipping submodule 'repos/php'
Skipping submodule 'repos/python'
Skipping submodule 'repos/ruby'
Skipping submodule 'repos/rust'
Skipping submodule 'repos/scala'
Skipping submodule 'repos/swift'
Skipping submodule 'repos/typescript'
```

On Tue, Jun 15, 2021 at 5:51 PM brian m. carlson
<sandals@crustytoothpaste.net> wrote:
>
> On 2021-06-16 at 00:16:06, Rose Kunkel wrote:
> > # What did you do before the bug happened? (Steps to reproduce your issue)
> > 1. Clone a git repository that sets `update = none` in .gitmodules:
> > $ git clone --recurse-submodules https://github.com/ubolonton/tree-sitter-langs
> >
> > 2. Perform a hard reset:
> > $ cd tree-sitter-langs
> > $ git reset --hard
> >
> > # What did you expect to happen? (Expected behavior)
> > The reset should succeed and do nothing.
>
> I think we're in agreement on this.  This should be a fresh clone and so
> a hard reset should change nothing.
>
> > # What happened instead? (Actual behavior)
> > The reset command fails with
> > ```
> > fatal: not a git repository: ../../.git/modules/repos/agda
> > fatal: could not reset submodule index
> > ```
>
> Hmmm, I can't reproduce this behavior.  What I see is this:
>
>   $ git reset --hard
>   HEAD is now at 5d362ce Release 0.10.0
>
> I'm running git version 2.32.0.272.g935e593368 on Debian sid (with the
> experimental packages).
>
> Can you try the clone and run a "git status" command in the repository
> to see if anything is modified after your clone?  Are the submodules
> checked out when you perform the clone?  In my case, I see lines like
> this:
>
>   Skipping submodule 'repos/agda'
>
> If you're seeing something different, then that might contribute to the
> different behavior we're seeing.
> --
> brian m. carlson (he/him or they/them)
> Toronto, Ontario, CA

Re: [BUG] `git reset --hard` fails with `update = none` submodules

[Rose Kunkel]

Potentially relevant: `git config --global --list` shows
```
status.showstash=true
status.submodulesummary=true
submodule.recurse=true
user.name=Rose Kunkel
user.email=rose@rosekunkel.me
pull.rebase=false
init.defaultbranch=main
```

On Tue, Jun 15, 2021 at 5:57 PM Rose Kunkel <rose@rosekunkel.me> wrote:
>
> Running `git status` in the resulting repository gives
> ```
> On branch master
> Your branch is up to date with 'origin/master'.
>
> nothing to commit, working tree clean
> ```
>
> This is the output from the clone command:
> ```
> Cloning into 'tree-sitter-langs'...
> remote: Enumerating objects: 609, done.
> remote: Counting objects: 100% (83/83), done.
> remote: Compressing objects: 100% (52/52), done.
> remote: Total 609 (delta 40), reused 58 (delta 24), pack-reused 526
> Receiving objects: 100% (609/609), 117.17 KiB | 1.05 MiB/s, done.
> Resolving deltas: 100% (322/322), done.
> Submodule 'repos/agda'
> (https://github.com/tree-sitter/tree-sitter-agda) registered for path
> 'repos/agda'
> Submodule 'repos/bash'
> (https://github.com/tree-sitter/tree-sitter-bash) registered for path
> 'repos/bash'
> Submodule 'repos/c' (https://github.com/tree-sitter/tree-sitter-c)
> registered for path 'repos/c'
> Submodule 'repos/c-sharp'
> (https://github.com/tree-sitter/tree-sitter-c-sharp) registered for
> path 'repos/c-sharp'
> Submodule 'repos/cpp' (https://github.com/tree-sitter/tree-sitter-cpp)
> registered for path 'repos/cpp'
> Submodule 'repos/css' (https://github.com/tree-sitter/tree-sitter-css)
> registered for path 'repos/css'
> Submodule 'repos/elm' (https://github.com/razzeee/tree-sitter-elm)
> registered for path 'repos/elm'
> Submodule 'repos/fluent'
> (https://github.com/tree-sitter/tree-sitter-fluent) registered for
> path 'repos/fluent'
> Submodule 'repos/go' (https://github.com/tree-sitter/tree-sitter-go)
> registered for path 'repos/go'
> Submodule 'repos/html'
> (https://github.com/tree-sitter/tree-sitter-html) registered for path
> 'repos/html'
> Submodule 'repos/janet-simple'
> (https://codeberg.org/sogaiu/tree-sitter-janet-simple) registered for
> path 'repos/janet-simple'
> Submodule 'repos/java'
> (https://github.com/tree-sitter/tree-sitter-java) registered for path
> 'repos/java'
> Submodule 'repos/javascript'
> (https://github.com/tree-sitter/tree-sitter-javascript) registered for
> path 'repos/javascript'
> Submodule 'repos/jsdoc'
> (https://github.com/tree-sitter/tree-sitter-jsdoc) registered for path
> 'repos/jsdoc'
> Submodule 'repos/json'
> (https://github.com/tree-sitter/tree-sitter-json) registered for path
> 'repos/json'
> Submodule 'repos/julia'
> (https://github.com/tree-sitter/tree-sitter-julia) registered for path
> 'repos/julia'
> Submodule 'repos/ocaml'
> (https://github.com/tree-sitter/tree-sitter-ocaml) registered for path
> 'repos/ocaml'
> Submodule 'repos/php' (https://github.com/tree-sitter/tree-sitter-php)
> registered for path 'repos/php'
> Submodule 'repos/python'
> (https://github.com/tree-sitter/tree-sitter-python) registered for
> path 'repos/python'
> Submodule 'repos/ruby'
> (https://github.com/tree-sitter/tree-sitter-ruby) registered for path
> 'repos/ruby'
> Submodule 'repos/rust'
> (https://github.com/tree-sitter/tree-sitter-rust) registered for path
> 'repos/rust'
> Submodule 'repos/scala'
> (https://github.com/tree-sitter/tree-sitter-scala) registered for path
> 'repos/scala'
> Submodule 'repos/swift'
> (https://github.com/tree-sitter/tree-sitter-swift) registered for path
> 'repos/swift'
> Submodule 'repos/typescript'
> (https://github.com/tree-sitter/tree-sitter-typescript) registered for
> path 'repos/typescript'
> Skipping submodule 'repos/agda'
> Skipping submodule 'repos/bash'
> Skipping submodule 'repos/c'
> Skipping submodule 'repos/c-sharp'
> Skipping submodule 'repos/cpp'
> Skipping submodule 'repos/css'
> Skipping submodule 'repos/elm'
> Skipping submodule 'repos/fluent'
> Skipping submodule 'repos/go'
> Skipping submodule 'repos/html'
> Skipping submodule 'repos/janet-simple'
> Skipping submodule 'repos/java'
> Skipping submodule 'repos/javascript'
> Skipping submodule 'repos/jsdoc'
> Skipping submodule 'repos/json'
> Skipping submodule 'repos/julia'
> Skipping submodule 'repos/ocaml'
> Skipping submodule 'repos/php'
> Skipping submodule 'repos/python'
> Skipping submodule 'repos/ruby'
> Skipping submodule 'repos/rust'
> Skipping submodule 'repos/scala'
> Skipping submodule 'repos/swift'
> Skipping submodule 'repos/typescript'
> ```
>
> On Tue, Jun 15, 2021 at 5:51 PM brian m. carlson
> <sandals@crustytoothpaste.net> wrote:
> >
> > On 2021-06-16 at 00:16:06, Rose Kunkel wrote:
> > > # What did you do before the bug happened? (Steps to reproduce your issue)
> > > 1. Clone a git repository that sets `update = none` in .gitmodules:
> > > $ git clone --recurse-submodules https://github.com/ubolonton/tree-sitter-langs
> > >
> > > 2. Perform a hard reset:
> > > $ cd tree-sitter-langs
> > > $ git reset --hard
> > >
> > > # What did you expect to happen? (Expected behavior)
> > > The reset should succeed and do nothing.
> >
> > I think we're in agreement on this.  This should be a fresh clone and so
> > a hard reset should change nothing.
> >
> > > # What happened instead? (Actual behavior)
> > > The reset command fails with
> > > ```
> > > fatal: not a git repository: ../../.git/modules/repos/agda
> > > fatal: could not reset submodule index
> > > ```
> >
> > Hmmm, I can't reproduce this behavior.  What I see is this:
> >
> >   $ git reset --hard
> >   HEAD is now at 5d362ce Release 0.10.0
> >
> > I'm running git version 2.32.0.272.g935e593368 on Debian sid (with the
> > experimental packages).
> >
> > Can you try the clone and run a "git status" command in the repository
> > to see if anything is modified after your clone?  Are the submodules
> > checked out when you perform the clone?  In my case, I see lines like
> > this:
> >
> >   Skipping submodule 'repos/agda'
> >
> > If you're seeing something different, then that might contribute to the
> > different behavior we're seeing.
> > --
> > brian m. carlson (he/him or they/them)
> > Toronto, Ontario, CA

Re: [BUG] `git reset --hard` fails with `update = none` submodules

[Rose Kunkel]

If I set `submodule.recurse = false` in .git/config, I get the
behavior you're seeing.

On Tue, Jun 15, 2021 at 6:03 PM Rose Kunkel <rose@rosekunkel.me> wrote:
>
> Potentially relevant: `git config --global --list` shows
> ```
> status.showstash=true
> status.submodulesummary=true
> submodule.recurse=true
> user.name=Rose Kunkel
> user.email=rose@rosekunkel.me
> pull.rebase=false
> init.defaultbranch=main
> ```
>
> On Tue, Jun 15, 2021 at 5:57 PM Rose Kunkel <rose@rosekunkel.me> wrote:
> >
> > Running `git status` in the resulting repository gives
> > ```
> > On branch master
> > Your branch is up to date with 'origin/master'.
> >
> > nothing to commit, working tree clean
> > ```
> >
> > This is the output from the clone command:
> > ```
> > Cloning into 'tree-sitter-langs'...
> > remote: Enumerating objects: 609, done.
> > remote: Counting objects: 100% (83/83), done.
> > remote: Compressing objects: 100% (52/52), done.
> > remote: Total 609 (delta 40), reused 58 (delta 24), pack-reused 526
> > Receiving objects: 100% (609/609), 117.17 KiB | 1.05 MiB/s, done.
> > Resolving deltas: 100% (322/322), done.
> > Submodule 'repos/agda'
> > (https://github.com/tree-sitter/tree-sitter-agda) registered for path
> > 'repos/agda'
> > Submodule 'repos/bash'
> > (https://github.com/tree-sitter/tree-sitter-bash) registered for path
> > 'repos/bash'
> > Submodule 'repos/c' (https://github.com/tree-sitter/tree-sitter-c)
> > registered for path 'repos/c'
> > Submodule 'repos/c-sharp'
> > (https://github.com/tree-sitter/tree-sitter-c-sharp) registered for
> > path 'repos/c-sharp'
> > Submodule 'repos/cpp' (https://github.com/tree-sitter/tree-sitter-cpp)
> > registered for path 'repos/cpp'
> > Submodule 'repos/css' (https://github.com/tree-sitter/tree-sitter-css)
> > registered for path 'repos/css'
> > Submodule 'repos/elm' (https://github.com/razzeee/tree-sitter-elm)
> > registered for path 'repos/elm'
> > Submodule 'repos/fluent'
> > (https://github.com/tree-sitter/tree-sitter-fluent) registered for
> > path 'repos/fluent'
> > Submodule 'repos/go' (https://github.com/tree-sitter/tree-sitter-go)
> > registered for path 'repos/go'
> > Submodule 'repos/html'
> > (https://github.com/tree-sitter/tree-sitter-html) registered for path
> > 'repos/html'
> > Submodule 'repos/janet-simple'
> > (https://codeberg.org/sogaiu/tree-sitter-janet-simple) registered for
> > path 'repos/janet-simple'
> > Submodule 'repos/java'
> > (https://github.com/tree-sitter/tree-sitter-java) registered for path
> > 'repos/java'
> > Submodule 'repos/javascript'
> > (https://github.com/tree-sitter/tree-sitter-javascript) registered for
> > path 'repos/javascript'
> > Submodule 'repos/jsdoc'
> > (https://github.com/tree-sitter/tree-sitter-jsdoc) registered for path
> > 'repos/jsdoc'
> > Submodule 'repos/json'
> > (https://github.com/tree-sitter/tree-sitter-json) registered for path
> > 'repos/json'
> > Submodule 'repos/julia'
> > (https://github.com/tree-sitter/tree-sitter-julia) registered for path
> > 'repos/julia'
> > Submodule 'repos/ocaml'
> > (https://github.com/tree-sitter/tree-sitter-ocaml) registered for path
> > 'repos/ocaml'
> > Submodule 'repos/php' (https://github.com/tree-sitter/tree-sitter-php)
> > registered for path 'repos/php'
> > Submodule 'repos/python'
> > (https://github.com/tree-sitter/tree-sitter-python) registered for
> > path 'repos/python'
> > Submodule 'repos/ruby'
> > (https://github.com/tree-sitter/tree-sitter-ruby) registered for path
> > 'repos/ruby'
> > Submodule 'repos/rust'
> > (https://github.com/tree-sitter/tree-sitter-rust) registered for path
> > 'repos/rust'
> > Submodule 'repos/scala'
> > (https://github.com/tree-sitter/tree-sitter-scala) registered for path
> > 'repos/scala'
> > Submodule 'repos/swift'
> > (https://github.com/tree-sitter/tree-sitter-swift) registered for path
> > 'repos/swift'
> > Submodule 'repos/typescript'
> > (https://github.com/tree-sitter/tree-sitter-typescript) registered for
> > path 'repos/typescript'
> > Skipping submodule 'repos/agda'
> > Skipping submodule 'repos/bash'
> > Skipping submodule 'repos/c'
> > Skipping submodule 'repos/c-sharp'
> > Skipping submodule 'repos/cpp'
> > Skipping submodule 'repos/css'
> > Skipping submodule 'repos/elm'
> > Skipping submodule 'repos/fluent'
> > Skipping submodule 'repos/go'
> > Skipping submodule 'repos/html'
> > Skipping submodule 'repos/janet-simple'
> > Skipping submodule 'repos/java'
> > Skipping submodule 'repos/javascript'
> > Skipping submodule 'repos/jsdoc'
> > Skipping submodule 'repos/json'
> > Skipping submodule 'repos/julia'
> > Skipping submodule 'repos/ocaml'
> > Skipping submodule 'repos/php'
> > Skipping submodule 'repos/python'
> > Skipping submodule 'repos/ruby'
> > Skipping submodule 'repos/rust'
> > Skipping submodule 'repos/scala'
> > Skipping submodule 'repos/swift'
> > Skipping submodule 'repos/typescript'
> > ```
> >
> > On Tue, Jun 15, 2021 at 5:51 PM brian m. carlson
> > <sandals@crustytoothpaste.net> wrote:
> > >
> > > On 2021-06-16 at 00:16:06, Rose Kunkel wrote:
> > > > # What did you do before the bug happened? (Steps to reproduce your issue)
> > > > 1. Clone a git repository that sets `update = none` in .gitmodules:
> > > > $ git clone --recurse-submodules https://github.com/ubolonton/tree-sitter-langs
> > > >
> > > > 2. Perform a hard reset:
> > > > $ cd tree-sitter-langs
> > > > $ git reset --hard
> > > >
> > > > # What did you expect to happen? (Expected behavior)
> > > > The reset should succeed and do nothing.
> > >
> > > I think we're in agreement on this.  This should be a fresh clone and so
> > > a hard reset should change nothing.
> > >
> > > > # What happened instead? (Actual behavior)
> > > > The reset command fails with
> > > > ```
> > > > fatal: not a git repository: ../../.git/modules/repos/agda
> > > > fatal: could not reset submodule index
> > > > ```
> > >
> > > Hmmm, I can't reproduce this behavior.  What I see is this:
> > >
> > >   $ git reset --hard
> > >   HEAD is now at 5d362ce Release 0.10.0
> > >
> > > I'm running git version 2.32.0.272.g935e593368 on Debian sid (with the
> > > experimental packages).
> > >
> > > Can you try the clone and run a "git status" command in the repository
> > > to see if anything is modified after your clone?  Are the submodules
> > > checked out when you perform the clone?  In my case, I see lines like
> > > this:
> > >
> > >   Skipping submodule 'repos/agda'
> > >
> > > If you're seeing something different, then that might contribute to the
> > > different behavior we're seeing.
> > > --
> > > brian m. carlson (he/him or they/them)
> > > Toronto, Ontario, CA

Re: [BUG] `git reset --hard` fails with `update = none` submodules

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1620 bytes --]

On 2021-06-16 at 01:03:40, Rose Kunkel wrote:
> Potentially relevant: `git config --global --list` shows
> ```
> status.showstash=true
> status.submodulesummary=true
> submodule.recurse=true

Thanks for this additional information.  This line is the critical
piece.  Now I get this:

  $ git reset --hard
  fatal: not a git repository: ../../.git/modules/repos/agda
  fatal: could not reset submodule index

Predictably, "git -c submodules.recurse=true reset --hard" also results
in the same thing.

The --recurse-submodules option for git reset says this (emphasis mine):

  When the working tree is updated, using --recurse-submodules will also
  recursively reset the working tree of all *active* submodules
  according to the commit recorded in the superproject, also setting the
  submodules' HEAD to be detached at that commit.

On my system, .git/config has this:

  [submodule]
          active = .

So these submodules are active, but they probably should not be, since
we haven't checked anything out (or, for that matter, cloned any data)
and it wouldn't make sense to try to operate on them automatically with
submodules.recurse or --recurse-submodules.

My gut tells me that we should probably mark submodules with update=none
set on a clone as inactive.  Of course, this is a tricky area that I'm
not super familiar with, so opinions or thoughts are welcome.

If folks think this is a good way forward, I'll look into writing a
patch, probably tomorrow evening since it's starting to get late here.
-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [BUG] `git reset --hard` fails with `update = none` submodules

[Rose Kunkel]

That sounds reasonable to me.

I do think it's pretty unintuitive that `update = none` means that
submodules never get initialized, even with an explicit `git submodule
init` command. If this is intended behavior, it should be better
documented. If not, fixing that would also fix this bug.

On Tue, Jun 15, 2021 at 6:25 PM brian m. carlson
<sandals@crustytoothpaste.net> wrote:
>
> On 2021-06-16 at 01:03:40, Rose Kunkel wrote:
> > Potentially relevant: `git config --global --list` shows
> > ```
> > status.showstash=true
> > status.submodulesummary=true
> > submodule.recurse=true
>
> Thanks for this additional information.  This line is the critical
> piece.  Now I get this:
>
>   $ git reset --hard
>   fatal: not a git repository: ../../.git/modules/repos/agda
>   fatal: could not reset submodule index
>
> Predictably, "git -c submodules.recurse=true reset --hard" also results
> in the same thing.
>
> The --recurse-submodules option for git reset says this (emphasis mine):
>
>   When the working tree is updated, using --recurse-submodules will also
>   recursively reset the working tree of all *active* submodules
>   according to the commit recorded in the superproject, also setting the
>   submodules' HEAD to be detached at that commit.
>
> On my system, .git/config has this:
>
>   [submodule]
>           active = .
>
> So these submodules are active, but they probably should not be, since
> we haven't checked anything out (or, for that matter, cloned any data)
> and it wouldn't make sense to try to operate on them automatically with
> submodules.recurse or --recurse-submodules.
>
> My gut tells me that we should probably mark submodules with update=none
> set on a clone as inactive.  Of course, this is a tricky area that I'm
> not super familiar with, so opinions or thoughts are welcome.
>
> If folks think this is a good way forward, I'll look into writing a
> patch, probably tomorrow evening since it's starting to get late here.
> --
> brian m. carlson (he/him or they/them)
> Toronto, Ontario, CA

Re: [BUG] `git reset --hard` fails with `update = none` submodules

[Rose Kunkel]

Actually, nevermind that last comment. I misunderstood what `git
submodule init` does. I thought it cloned missing submodules, but
apparently that's done by `git submodule update`, so the behavior does
make sense.

On Tue, Jun 15, 2021 at 6:39 PM Rose Kunkel <rose@rosekunkel.me> wrote:
>
> That sounds reasonable to me.
>
> I do think it's pretty unintuitive that `update = none` means that
> submodules never get initialized, even with an explicit `git submodule
> init` command. If this is intended behavior, it should be better
> documented. If not, fixing that would also fix this bug.
>
> On Tue, Jun 15, 2021 at 6:25 PM brian m. carlson
> <sandals@crustytoothpaste.net> wrote:
> >
> > On 2021-06-16 at 01:03:40, Rose Kunkel wrote:
> > > Potentially relevant: `git config --global --list` shows
> > > ```
> > > status.showstash=true
> > > status.submodulesummary=true
> > > submodule.recurse=true
> >
> > Thanks for this additional information.  This line is the critical
> > piece.  Now I get this:
> >
> >   $ git reset --hard
> >   fatal: not a git repository: ../../.git/modules/repos/agda
> >   fatal: could not reset submodule index
> >
> > Predictably, "git -c submodules.recurse=true reset --hard" also results
> > in the same thing.
> >
> > The --recurse-submodules option for git reset says this (emphasis mine):
> >
> >   When the working tree is updated, using --recurse-submodules will also
> >   recursively reset the working tree of all *active* submodules
> >   according to the commit recorded in the superproject, also setting the
> >   submodules' HEAD to be detached at that commit.
> >
> > On my system, .git/config has this:
> >
> >   [submodule]
> >           active = .
> >
> > So these submodules are active, but they probably should not be, since
> > we haven't checked anything out (or, for that matter, cloned any data)
> > and it wouldn't make sense to try to operate on them automatically with
> > submodules.recurse or --recurse-submodules.
> >
> > My gut tells me that we should probably mark submodules with update=none
> > set on a clone as inactive.  Of course, this is a tricky area that I'm
> > not super familiar with, so opinions or thoughts are welcome.
> >
> > If folks think this is a good way forward, I'll look into writing a
> > patch, probably tomorrow evening since it's starting to get late here.
> > --
> > brian m. carlson (he/him or they/them)
> > Toronto, Ontario, CA

Re: [BUG] `git reset --hard` fails with `update = none` submodules

[Junio C Hamano]

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

> On 2021-06-16 at 01:03:40, Rose Kunkel wrote:
>> Potentially relevant: `git config --global --list` shows
>> ```
>> status.showstash=true
>> status.submodulesummary=true
>> submodule.recurse=true
>
> Thanks for this additional information.  This line is the critical
> piece.  Now I get this:
>
>   $ git reset --hard
>   fatal: not a git repository: ../../.git/modules/repos/agda
>   fatal: could not reset submodule index
>
> Predictably, "git -c submodules.recurse=true reset --hard" also results
> in the same thing.
>
> The --recurse-submodules option for git reset says this (emphasis mine):
>
>   When the working tree is updated, using --recurse-submodules will also
>   recursively reset the working tree of all *active* submodules
>   according to the commit recorded in the superproject, also setting the
>   submodules' HEAD to be detached at that commit.
>
> On my system, .git/config has this:
>
>   [submodule]
>           active = .
>
> So these submodules are active, but they probably should not be, since
> we haven't checked anything out (or, for that matter, cloned any data)
> and it wouldn't make sense to try to operate on them automatically with
> submodules.recurse or --recurse-submodules.
>
> My gut tells me that we should probably mark submodules with update=none
> set on a clone as inactive.  Of course, this is a tricky area that I'm
> not super familiar with, so opinions or thoughts are welcome.
>
> If folks think this is a good way forward, I'll look into writing a
> patch, probably tomorrow evening since it's starting to get late here.

Cc'ing some folks who recently mumbled the word "submodule" in their
recent topics for input.

Thanks.

Re: [BUG] `git reset --hard` fails with `update = none` submodules

[Philippe Blain]

Hi all,

Le 2021-06-15 à 23:10, Junio C Hamano a écrit :
> "brian m. carlson" <sandals@crustytoothpaste.net> writes:
> 
>> On 2021-06-16 at 01:03:40, Rose Kunkel wrote:
>>> Potentially relevant: `git config --global --list` shows
>>> ```
>>> status.showstash=true
>>> status.submodulesummary=true
>>> submodule.recurse=true
>>
>> Thanks for this additional information.  This line is the critical
>> piece.  Now I get this:
>>
>>    $ git reset --hard
>>    fatal: not a git repository: ../../.git/modules/repos/agda
>>    fatal: could not reset submodule index
>>
>> Predictably, "git -c submodules.recurse=true reset --hard" also results
>> in the same thing.
>>
>> The --recurse-submodules option for git reset says this (emphasis mine):
>>
>>    When the working tree is updated, using --recurse-submodules will also
>>    recursively reset the working tree of all *active* submodules
>>    according to the commit recorded in the superproject, also setting the
>>    submodules' HEAD to be detached at that commit.
>>
>> On my system, .git/config has this:
>>
>>    [submodule]
>>            active = .
>>
>> So these submodules are active, but they probably should not be, since
>> we haven't checked anything out (or, for that matter, cloned any data)
>> and it wouldn't make sense to try to operate on them automatically with
>> submodules.recurse or --recurse-submodules.
>>
>> My gut tells me that we should probably mark submodules with update=none
>> set on a clone as inactive.  Of course, this is a tricky area that I'm
>> not super familiar with, so opinions or thoughts are welcome.
>>
>> If folks think this is a good way forward, I'll look into writing a
>> patch, probably tomorrow evening since it's starting to get late here.

This is probably a good fix. 'git clone --recurse-submodules' is really
too eager to write the 'submodule.active=.' config but it should be more careful;
the above is an example and [1] is another one.

I think it is a good way forward that having 'submodule.$name.update=none' in
'.gitmodules' means that 'git clone' would write 'submodule.$name.active=false'
to the local config. This way it would still be the case that 'submodule.$name.update'
itself only ever applies to 'git submodule update', which is what is documented [2].

A longer term plan could be to completely deprecate 'submodule.$name.update=none' and allow
'submodule.$name.active' in '.gitmodules'. Maybe anecdotal, but at least one user [3]
tried to do that and found out it does not work. Having 'active' allowed in '.gitmodules'
would (I think) cover all uses cases for 'update=none', which could then be deprecated,
but that would need a transition period. But this change is not necessarily needed; it
would just make the whole system more consistent IMO.

Cheers,

Philippe.
P.S. The current bug was reported last year [4] but I could not reproduce it at when I
replied. Adding '--force' to the 'git checkout other-branch' command in my reproducer
script does trigger the bug, though.


[1] https://lore.kernel.org/git/20200501005432.h62dnpkx7feb7rto@glandium.org/T/#u
[2] https://git-scm.com/docs/git-config#Documentation/git-config.txt-submoduleltnamegtupdate
[3] https://lore.kernel.org/git/HE1PR04MB2987A255BC327320D1DDFE6692F70@HE1PR04MB2987.eurprd04.prod.outlook.com/
[4] https://lore.kernel.org/git/CABVQXt5E-R22G62W-tQieA7XiZKZiOA8Hp2xewYhwwOS8wFh0Q@mail.gmail.com/

Re: [BUG] `git reset --hard` fails with `update = none` submodules

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1685 bytes --]

On 2021-06-16 at 13:20:44, Philippe Blain wrote:
> Hi all,
> 
> Le 2021-06-15 à 23:10, Junio C Hamano a écrit :
> > "brian m. carlson" <sandals@crustytoothpaste.net> writes:
> > 
> > > My gut tells me that we should probably mark submodules with update=none
> > > set on a clone as inactive.  Of course, this is a tricky area that I'm
> > > not super familiar with, so opinions or thoughts are welcome.
> > > 
> > > If folks think this is a good way forward, I'll look into writing a
> > > patch, probably tomorrow evening since it's starting to get late here.
> 
> This is probably a good fix. 'git clone --recurse-submodules' is really
> too eager to write the 'submodule.active=.' config but it should be more careful;
> the above is an example and [1] is another one.
> 
> I think it is a good way forward that having 'submodule.$name.update=none' in
> '.gitmodules' means that 'git clone' would write 'submodule.$name.active=false'
> to the local config. This way it would still be the case that 'submodule.$name.update'
> itself only ever applies to 'git submodule update', which is what is documented [2].

I have a patch along this line and am looking into some test failures,
so it's not that I've forgotten about it, but that it is, as usual, not
as trivial as I'd hoped.  It may take me until the weekend to come up
with something nice.

Our problem seems to be that once we mark it as inactive, we never end
up initializing the repository even if something like --checkout is
passed (at least in the tests), so I'm trying to come up with an elegant
way to deal with that.
-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

[PATCH] submodule: mark submodules with update=none as inactive

[brian m. carlson]

When the user recursively clones a repository with submodules and one or
more of those submodules is marked with the submodule.<name>.update=none
configuration, the submodule will end up being active.  This is a
problem because we will have skipped cloning or checking out the
submodule, and as a result, other commands, such as git reset or git
checkout, will fail if they are invoked with --recurse-submodules (or
when submodule.recurse is true).

This is obviously not the behavior the user wanted, so let's fix this by
specifically setting the submodule as inactive in this case when we're
cloning the repository.  That will make us properly ignore the submodule
when performing recursive operations.

Note that we only do this when the --require-init option is passed,
which is only passed during clone.  That's because the update-clone
submodule helper is also invoked during a user-invoked git submodule
update, where we explicitly indicate that we override the configuration
setting, so we don't want to set such a configuration option then.

Reported-by: Rose Kunkel <rose@rosekunkel.me>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 builtin/submodule--helper.c |  5 +++++
 t/t5601-clone.sh            | 24 ++++++++++++++++++++++++
 2 files changed, 29 insertions(+)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index ae6174ab05..2e14cc7a26 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2102,6 +2102,11 @@ static int prepare_to_clone_next_submodule(const struct cache_entry *ce,
 	if (suc->update.type == SM_UPDATE_NONE
 	    || (suc->update.type == SM_UPDATE_UNSPECIFIED
 		&& update_type == SM_UPDATE_NONE)) {
+		if (suc->require_init) {
+			key = xstrfmt("submodule.%s.active", sub->name);
+			git_config_set(key, "false");
+			free(key);
+		}
 		strbuf_addf(out, _("Skipping submodule '%s'"), displaypath);
 		strbuf_addch(out, '\n');
 		goto cleanup;
diff --git a/t/t5601-clone.sh b/t/t5601-clone.sh
index c0688467e7..efe6b13be0 100755
--- a/t/t5601-clone.sh
+++ b/t/t5601-clone.sh
@@ -752,6 +752,30 @@ test_expect_success 'batch missing blob request does not inadvertently try to fe
 	git clone --filter=blob:limit=0 "file://$(pwd)/server" client
 '
 
+test_expect_success 'clone with submodule with update=none is not active' '
+	rm -rf server client &&
+
+	test_create_repo server &&
+	echo a >server/a &&
+	echo b >server/b &&
+	git -C server add a b &&
+	git -C server commit -m x &&
+
+	echo aa >server/a &&
+	echo bb >server/b &&
+	git -C server submodule add --name c "$(pwd)/repo_for_submodule" c &&
+	git -C server config -f .gitmodules submodule.c.update none &&
+	git -C server add a b c .gitmodules &&
+	git -C server commit -m x &&
+
+	git clone --recurse-submodules server client &&
+	git -C client config submodule.c.active >actual &&
+	echo false >expected &&
+	test_cmp actual expected &&
+	# This would fail if the submodule were active, since it is not checked out.
+	git -C client reset --recurse-submodules --hard
+'
+
 . "$TEST_DIRECTORY"/lib-httpd.sh
 start_httpd
 

Re: [PATCH] submodule: mark submodules with update=none as inactive

[Philippe Blain]

Hi brian,

Le 2021-06-19 à 17:44, brian m. carlson a écrit :
> When the user recursively clones a repository with submodules and one or
> more of those submodules is marked with the submodule.<name>.update=none
> configuration, the submodule will end up being active.  This is a
> problem because we will have skipped cloning or checking out the
> submodule, and as a result, other commands, such as git reset or git
> checkout, will fail if they are invoked with --recurse-submodules (or
> when submodule.recurse is true).
> 
> This is obviously not the behavior the user wanted, so let's fix this by
> specifically setting the submodule as inactive in this case when we're
> cloning the repository. 

I'm not sure we want to fix it only at (recursive) clone time. The original bug report
was with 'git clone --recurse-submodules', but a non-recursive clone
followed by the usual 'git submodule init && git submodule update' (or
in one step 'git submodule update --init') would also lead to the same
bad experience (I'm not sure I want to call it a bug per se... it's certainly
a UX bug :)

> That will make us properly ignore the submodule
> when performing recursive operations.
> 
> Note that we only do this when the --require-init option is passed,
> which is only passed during clone.  That's because the update-clone
> submodule helper is also invoked during a user-invoked git submodule
> update, where we explicitly indicate that we override the configuration
> setting, so we don't want to set such a configuration option then.

I'm not sure what you mean here by 'where we explicitely indicate that we
override the configuration setting'. For me, as I wrote above,
'git clone --recurse-submodules' and 'git clone' followed by
'git submodule update --init' should lead to mostly [*] the same end result.

If you mean 'git submodule update --checkout', that indeed seems to sometimes override the 'update=none'
configuration (a fact which is absent from the documentation), then it's true that we
would not want to write 'active=false' at that invocation. As an aside, in my limited testing
I could not always get 'git submodule update --checkout' to clone and checkout 'update=none' submdules;
it would fail with "fatal: could not get a repository handle for submodule 'sub1'" because
'git checkout/reset --recurse-submodules' leaves a bad .git/modules/sub1/config file
with the core.worktree setting when the command fails (this should not happen)...

In any case, that leads me to think that maybe the right place to write the 'active' setting
would be during 'git submodule init', thus builtin/submodule--helper.c::init_submodule ?
This way it would lead to the same behaviour if the clone was recursive or not,
and it would not interfere with 'git submodule update --checkout'.

[*] I say 'mostly' because in the first case you end up with a 'submodule.active=.'
configuration, and no 'submodule.$name.active' configuration for each submodule,
whereas in the second case, there is no 'submodule.active' setting and
'submodule.$name.active' is true for each submodule.

> 
> Reported-by: Rose Kunkel <rose@rosekunkel.me>
> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
> ---
>   builtin/submodule--helper.c |  5 +++++
>   t/t5601-clone.sh            | 24 ++++++++++++++++++++++++
>   2 files changed, 29 insertions(+)

I think that such a change would warrant a mention in the doc, in
gitsubmodules [1], git-config [2], and probably git-submodule [3] if we agree that
'git submodule init' is the right place to set the 'active=false' flag.

Thanks for proposing a patch!

Philippe.

[1] https://git-scm.com/docs/gitmodules#Documentation/gitmodules.txt-submoduleltnamegtupdate
[2] https://git-scm.com/docs/git-config#Documentation/git-config.txt-submoduleltnamegtupdate
[3] https://git-scm.com/docs/git-submodule#Documentation/git-submodule.txt-init--ltpathgt82308203

Re: [PATCH] submodule: mark submodules with update=none as inactive

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 2179 bytes --]

On 2021-06-22 at 03:45:45, Philippe Blain wrote:
> > That will make us properly ignore the submodule
> > when performing recursive operations.
> > 
> > Note that we only do this when the --require-init option is passed,
> > which is only passed during clone.  That's because the update-clone
> > submodule helper is also invoked during a user-invoked git submodule
> > update, where we explicitly indicate that we override the configuration
> > setting, so we don't want to set such a configuration option then.
> 
> I'm not sure what you mean here by 'where we explicitely indicate that we
> override the configuration setting'. For me, as I wrote above,
> 'git clone --recurse-submodules' and 'git clone' followed by
> 'git submodule update --init' should lead to mostly [*] the same end result.
> 
> If you mean 'git submodule update --checkout', that indeed seems to sometimes override the 'update=none'
> configuration (a fact which is absent from the documentation), then it's true that we
> would not want to write 'active=false' at that invocation. As an aside, in my limited testing
> I could not always get 'git submodule update --checkout' to clone and checkout 'update=none' submdules;
> it would fail with "fatal: could not get a repository handle for submodule 'sub1'" because
> 'git checkout/reset --recurse-submodules' leaves a bad .git/modules/sub1/config file
> with the core.worktree setting when the command fails (this should not happen)...

Yes, that's what I meant.

> In any case, that leads me to think that maybe the right place to write the 'active' setting
> would be during 'git submodule init', thus builtin/submodule--helper.c::init_submodule ?
> This way it would lead to the same behaviour if the clone was recursive or not,
> and it would not interfere with 'git submodule update --checkout'.

Let me take a look at some other approaches and see if I can come up
with something a little bit better.

My apologies for the delay in response; I'm in the process of moving at
the moment and my attention has been directed elsewhere than the list.
-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [PATCH] submodule: mark submodules with update=none as inactive

[Philippe Blain]

Hi brian,

Le 2021-06-25 à 19:02, brian m. carlson a écrit :
> On 2021-06-22 at 03:45:45, Philippe Blain wrote:
>>> That will make us properly ignore the submodule
>>> when performing recursive operations.
>>>
>>> Note that we only do this when the --require-init option is passed,
>>> which is only passed during clone.  That's because the update-clone
>>> submodule helper is also invoked during a user-invoked git submodule
>>> update, where we explicitly indicate that we override the configuration
>>> setting, so we don't want to set such a configuration option then.
>>
>> I'm not sure what you mean here by 'where we explicitely indicate that we
>> override the configuration setting'. For me, as I wrote above,
>> 'git clone --recurse-submodules' and 'git clone' followed by
>> 'git submodule update --init' should lead to mostly [*] the same end result.
>>
>> If you mean 'git submodule update --checkout', that indeed seems to sometimes override the 'update=none'
>> configuration (a fact which is absent from the documentation), 

Note that I'm taking back that statemnt about the doc; the description of 'git submodule update'
states:

"The "updating" can be done in several ways depending on command line options
and the value of submodule.<name>.update configuration variable. The command line
option takes precedence over the configuration variable."

I was somehow under the impression that 'none' was a special case, but it's not.

then it's true that we
>> would not want to write 'active=false' at that invocation. As an aside, in my limited testing
>> I could not always get 'git submodule update --checkout' to clone and checkout 'update=none' submdules;
>> it would fail with "fatal: could not get a repository handle for submodule 'sub1'" because
>> 'git checkout/reset --recurse-submodules' leaves a bad .git/modules/sub1/config file
>> with the core.worktree setting when the command fails (this should not happen)...
> 
> Yes, that's what I meant.
> 
>> In any case, that leads me to think that maybe the right place to write the 'active' setting
>> would be during 'git submodule init', thus builtin/submodule--helper.c::init_submodule ?
>> This way it would lead to the same behaviour if the clone was recursive or not,
>> and it would not interfere with 'git submodule update --checkout'.
> 
> Let me take a look at some other approaches and see if I can come up
> with something a little bit better.

I tested the following and it fixes the bug (for both recursive clone and
normal clone followed by 'git submodule update --init') and
t7406-submodule-update.sh still passes:

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index d55f6262e9..a4cd86c72f 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -686,6 +686,13 @@ static void init_submodule(const char *path, const char *prefix,
  
  		if (git_config_set_gently(sb.buf, upd))
  			die(_("Failed to register update mode for submodule path '%s'"), displaypath);
+
+		/* Mark update=none submodules as inactive */
+		if (sub->update_strategy.type == SM_UPDATE_NONE) {
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "submodule.%s.active", sub->name);
+			git_config_set_gently(sb.buf, "false");
+		}
  	}
  	strbuf_release(&sb);
  	free(displaypath);

> 
> My apologies for the delay in response; I'm in the process of moving at
> the moment and my attention has been directed elsewhere than the list.
> 

No problem of course!

Philippe.


[1] https://git-scm.com/docs/git-submodule#Documentation/git-submodule.txt-update--init--remote-N--no-fetch--no-recommend-shallow-f--force--checkout--rebase--merge--referenceltrepositorygt--depthltdepthgt--recursive--jobsltngt--no-single-branch--ltpathgt82308203

[PATCH v2] submodule: mark submodules with update=none as inactive

[brian m. carlson]

When the user recursively clones a repository with submodules and one or
more of those submodules is marked with the submodule.<name>.update=none
configuration, the submodule will end up being active.  This is a
problem because we will have skipped cloning or checking out the
submodule, and as a result, other commands, such as git reset or git
checkout, will fail if they are invoked with --recurse-submodules (or
when submodule.recurse is true).

This is obviously not the behavior the user wanted, so let's fix this by
specifically setting the submodule as inactive in this case when we're
initializing the repository.  That will make us properly ignore the
submodule when performing recursive operations.

We only do this when initializing a submodule, since git submodule
update can update the submodule with various options despite the setting
of "none" and we want those options to override it as they currently do.

Reported-by: Rose Kunkel <rose@rosekunkel.me>
Helped-by: Philippe Blain <levraiphilippeblain@gmail.com>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 builtin/submodule--helper.c |  6 ++++++
 t/t5601-clone.sh            | 24 ++++++++++++++++++++++++
 2 files changed, 30 insertions(+)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index ae6174ab05..a3f8c45d97 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -686,6 +686,12 @@ static void init_submodule(const char *path, const char *prefix,
 
 		if (git_config_set_gently(sb.buf, upd))
 			die(_("Failed to register update mode for submodule path '%s'"), displaypath);
+
+		if (sub->update_strategy.type == SM_UPDATE_NONE) {
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "submodule.%s.active", sub->name);
+			git_config_set_gently(sb.buf, "false");
+		}
 	}
 	strbuf_release(&sb);
 	free(displaypath);
diff --git a/t/t5601-clone.sh b/t/t5601-clone.sh
index c0688467e7..efe6b13be0 100755
--- a/t/t5601-clone.sh
+++ b/t/t5601-clone.sh
@@ -752,6 +752,30 @@ test_expect_success 'batch missing blob request does not inadvertently try to fe
 	git clone --filter=blob:limit=0 "file://$(pwd)/server" client
 '
 
+test_expect_success 'clone with submodule with update=none is not active' '
+	rm -rf server client &&
+
+	test_create_repo server &&
+	echo a >server/a &&
+	echo b >server/b &&
+	git -C server add a b &&
+	git -C server commit -m x &&
+
+	echo aa >server/a &&
+	echo bb >server/b &&
+	git -C server submodule add --name c "$(pwd)/repo_for_submodule" c &&
+	git -C server config -f .gitmodules submodule.c.update none &&
+	git -C server add a b c .gitmodules &&
+	git -C server commit -m x &&
+
+	git clone --recurse-submodules server client &&
+	git -C client config submodule.c.active >actual &&
+	echo false >expected &&
+	test_cmp actual expected &&
+	# This would fail if the submodule were active, since it is not checked out.
+	git -C client reset --recurse-submodules --hard
+'
+
 . "$TEST_DIRECTORY"/lib-httpd.sh
 start_httpd
 

Re: [PATCH v2] submodule: mark submodules with update=none as inactive

[Philippe Blain]

Hi brian,

[re-cc'ing Emily and Jonathan who Junio cc'ed in <xmqqeed2sdwc.fsf@gitster.g>
but seemed to have been dropped when you sent v1 and v2 of the patch]

Le 2021-07-01 à 18:51, brian m. carlson a écrit :
> When the user recursively clones a repository with submodules 

Here I would add:

", or runs 'git submodule update --init' after a
non-recursive clone of such a repository, "

> and one or
> more of those submodules is marked with the submodule.<name>.update=none
> configuration, the submodule 

"those submodules" would be clearer, I think.

> will end up being active.  This is a
> problem because we will have skipped cloning or checking out the
> submodule, and as a result, other commands, such as git reset or git
> checkout, will fail if they are invoked with --recurse-submodules (or
> when submodule.recurse is true).
> 
> This is obviously not the behavior the user wanted, so let's fix this by
> specifically setting the submodule as inactive in this case when we're
> initializing the repository.  That will make us properly ignore the
> submodule when performing recursive operations.
> 
> We only do this when initializing a submodule, 

Here for even more clarity I would add:

i.e. 'git submodule init' or 'git submodule update --init',

> since git submodule
> update can update the submodule with various options despite the setting
> of "none" and we want those options to override it as they currently do.
> 
> Reported-by: Rose Kunkel <rose@rosekunkel.me>
> Helped-by: Philippe Blain <levraiphilippeblain@gmail.com>
> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
> ---
>   builtin/submodule--helper.c |  6 ++++++
>   t/t5601-clone.sh            | 24 ++++++++++++++++++++++++
>   2 files changed, 30 insertions(+)

As I said in my review of v1, I think this would warrant a mention in the doc.

In general, I think 'git-submodule(1)' could be more precise about which submodules
are touched by which subcommands. Since the topic that introduced the 'active' concept
was merged in a93dcb0a56 (Merge branch 'bw/submodule-is-active', 2017-03-30), these subcommand
recurse only in active submodules:

- init (with a big caveat, see below)
- sync
- update

The doc makes no mention of that for sync and update. sync says it synchronizes 'all'
submodules, and update says it updates 'registered' submodules ('registered' in not
defined formally anywhere either). And 'active' is mentioned in the description of
'init', but not defined. It would be good to explicitely say "see the 'Active submodules'
section in gitsubmodules(7) for a definition of 'active'", or something like that.

I'm not saying we need to fix that necessarily in this patch, I'm just noting
what my reading of the code and of the doc reveals.

> 
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index ae6174ab05..a3f8c45d97 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -686,6 +686,12 @@ static void init_submodule(const char *path, const char *prefix,
>   
>   		if (git_config_set_gently(sb.buf, upd))
>   			die(_("Failed to register update mode for submodule path '%s'"), displaypath);
> +
> +		if (sub->update_strategy.type == SM_UPDATE_NONE) {
> +			strbuf_reset(&sb);
> +			strbuf_addf(&sb, "submodule.%s.active", sub->name);
> +			git_config_set_gently(sb.buf, "false");
> +		}
>   	}
>   	strbuf_release(&sb);
>   	free(displaypath);

I did more testing with this patch applied and I fear it is not
completely sufficient. There are 2 main problems, I think.
The first is that the following still triggers the bug:

     git clone server client
     git -C client submodule update --init
     git -C client submodule init       # should be no-op, but isn't
     git -C client reset --hard --recurse-submodules

That's because:

1) 'git submodule init' operates on *all* submodules if 'submodule.active' is unset
     and not <path> is given.
     (see submodule--helper.c::module_init), or the doc [1].
2) 'git submodule init' sets 'submodule.$name.active' to true for the submodules
     on which it operates, unless already covered by 'submodule.active'
     (see submodule--helper.c::init_submodule)
3) the code we're adding to set 'active' to false if 'update=none' is only executed
    if 'submodule.c.update' is not yet in the config, so it gets skipped if we
    repeat 'git submodule init'. (I think this behaviour is sound).

So that's unfortunate, and is also kind of contradictory to what the doc says
for 'git submodule init':
"This command does not alter existing information in .git/config.".
And just to be clear, the behaviour I describe above is already existing, the current
patch just makes it more obvious.

I think we could manage to change that behaviour a bit
in order to have 'submodule init' not modify the config for submodules which are already marked inactive,
*unless* they are explitely matched by the pathspec on the command line.
So we would have:

     git clone server client; cd client
     git submodule init      # initial call sets 'submodule.c.active=false'
     git submodule init      # does not touch c, it's already marked inactive
     git submodule init c    # OK, we really want to mark it as active

To do that, we could use the same trick that we do in update_clone, i.e.

     if (pathspec.nr)
         info.explicit = 1

where 'explicit' (tentative name) is a new field in 'struct init_cb', so that 'init_submodule'
knows if the current submodule was explicitely listed on the command line.


Then there is a second thing. As stated in the commit message,
'git submodule update --checkout' should override the 'update=none'
setting and clone and checkout the submodule. But this behaviour
is broken by the code we're adding, because 'submodule update' only recurse into
active submodules! (see the call to 'is_submodule_active' in
submodule--helper.c::prepare_to_clone_next_submodule).

So this does not clone the submodule:

     git clone --recurse server client   # recursive clone
     git -C client submodule --checkout  # should clone c, doesn't

Neither does this:
    
     git clone server client                   # non-recursive clone
     git -C client submodule update --init
     git -C client submodule update --checkout # should clone c, doesn't

But because of the first problem above, this works(!):

     git clone server client
     git -C client submodule update --init
     git -C client submodule update --init --checkout

Because in the third call, c is set to 'active' by init_submodule,
then is *not* skipped by prepare_to_clone_next_submodule.


So it's all a little bit complicated! But I think that with my suggestion above,
i.e. that 'git submodule init', in the absence of 'submodule.active', would
only switch inactive submodules to active if they are explicitely listed, then
we could get a saner behaviour, at the expense of having to explicitely init
'update=none' submodules on the command line if we really want to '--checkout' :
     
     git clone server client
     git -C client submodule update --init        # first call: set c to inactive
     git -C client submodule update --init        # no-op
     git -C client submodule update --checkout    # does not clone c (currently quiet)
     git -C client submodule update --checkout c  # does not clone c, but warns (current behaviour)
     git -C client submodule init c               # sets c to active
     git -C client submodule update --checkout    # clones c

where the last two command could be a single
'git submodule update --init --checkout c' and ideally the
4th command should also warn the user that they now have to explicitely 'init'
c if they want to check it out, which could simply mean tweaking the already
existing message in next_submodule_warn_missing to also check if
the current submodule has 'update=none' and then display the warning
(instead of just showing it if the submodule was listed on the command
line, which is the current behaviour). Additionnaly, the warning should
say "Maybe you want to use 'update --init %s'?", i.e. specify the path.


What do you think of my suggestions ? I can help push this forward
by contributing patches if we agree that we should go forward with
this slight behaviour change in 'git submodule init' ...


> diff --git a/t/t5601-clone.sh b/t/t5601-clone.sh
> index c0688467e7..efe6b13be0 100755
> --- a/t/t5601-clone.sh
> +++ b/t/t5601-clone.sh
> @@ -752,6 +752,30 @@ test_expect_success 'batch missing blob request does not inadvertently try to fe
>   	git clone --filter=blob:limit=0 "file://$(pwd)/server" client
>   '
>   
> +test_expect_success 'clone with submodule with update=none is not active' '
> +	rm -rf server client &&
> +
> +	test_create_repo server &&
> +	echo a >server/a &&
> +	echo b >server/b &&
> +	git -C server add a b &&
> +	git -C server commit -m x &&
> +
> +	echo aa >server/a &&
> +	echo bb >server/b &&
> +	git -C server submodule add --name c "$(pwd)/repo_for_submodule" c &&
> +	git -C server config -f .gitmodules submodule.c.update none &&
> +	git -C server add a b c .gitmodules &&
> +	git -C server commit -m x &&
> +
> +	git clone --recurse-submodules server client &&
> +	git -C client config submodule.c.active >actual &&
> +	echo false >expected &&
> +	test_cmp actual expected &&
> +	# This would fail if the submodule were active, since it is not checked out.
> +	git -C client reset --recurse-submodules --hard
> +'

I think we might want to also test the non-recursive clone case as well,
i.e. 'git clone' and then 'git submodule update --init', as well as
subsequent calls to 'git submodule init' in light of my analysis above.

Also, the only place in the test suite that I could find where
'update=none' is tested is in t7406.35-38 in t7406-submodule-update.sh
so maybe it would make more sense to put the test(s) there ?

Thanks,

Philippe.

[1] https://git-scm.com/docs/git-submodule#Documentation/git-submodule.txt-init--ltpathgt82308203

Re: [PATCH v2] submodule: mark submodules with update=none as inactive

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 8938 bytes --]

On 2021-07-09 at 20:26:35, Philippe Blain wrote:
> Hi brian,
> 
> [re-cc'ing Emily and Jonathan who Junio cc'ed in <xmqqeed2sdwc.fsf@gitster.g>
> but seemed to have been dropped when you sent v1 and v2 of the patch]
> 
> Le 2021-07-01 à 18:51, brian m. carlson a écrit :
> > When the user recursively clones a repository with submodules
> 
> Here I would add:
> 
> ", or runs 'git submodule update --init' after a
> non-recursive clone of such a repository, "
> 
> > and one or
> > more of those submodules is marked with the submodule.<name>.update=none
> > configuration, the submodule
> 
> "those submodules" would be clearer, I think.

Sure, I can make that change.

> > will end up being active.  This is a
> > problem because we will have skipped cloning or checking out the
> > submodule, and as a result, other commands, such as git reset or git
> > checkout, will fail if they are invoked with --recurse-submodules (or
> > when submodule.recurse is true).
> > 
> > This is obviously not the behavior the user wanted, so let's fix this by
> > specifically setting the submodule as inactive in this case when we're
> > initializing the repository.  That will make us properly ignore the
> > submodule when performing recursive operations.
> > 
> > We only do this when initializing a submodule,
> 
> Here for even more clarity I would add:
> 
> i.e. 'git submodule init' or 'git submodule update --init',

Okay.

> > since git submodule
> > update can update the submodule with various options despite the setting
> > of "none" and we want those options to override it as they currently do.
> > 
> > Reported-by: Rose Kunkel <rose@rosekunkel.me>
> > Helped-by: Philippe Blain <levraiphilippeblain@gmail.com>
> > Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
> > ---
> >   builtin/submodule--helper.c |  6 ++++++
> >   t/t5601-clone.sh            | 24 ++++++++++++++++++++++++
> >   2 files changed, 30 insertions(+)
> 
> As I said in my review of v1, I think this would warrant a mention in the doc.
> 
> In general, I think 'git-submodule(1)' could be more precise about which submodules
> are touched by which subcommands. Since the topic that introduced the 'active' concept
> was merged in a93dcb0a56 (Merge branch 'bw/submodule-is-active', 2017-03-30), these subcommand
> recurse only in active submodules:
> 
> - init (with a big caveat, see below)
> - sync
> - update
> 
> The doc makes no mention of that for sync and update. sync says it synchronizes 'all'
> submodules, and update says it updates 'registered' submodules ('registered' in not
> defined formally anywhere either). And 'active' is mentioned in the description of
> 'init', but not defined. It would be good to explicitely say "see the 'Active submodules'
> section in gitsubmodules(7) for a definition of 'active'", or something like that.
> 
> I'm not saying we need to fix that necessarily in this patch, I'm just noting
> what my reading of the code and of the doc reveals.



> I did more testing with this patch applied and I fear it is not
> completely sufficient. There are 2 main problems, I think.
> The first is that the following still triggers the bug:
> 
>     git clone server client
>     git -C client submodule update --init
>     git -C client submodule init       # should be no-op, but isn't
>     git -C client reset --hard --recurse-submodules
> 
> That's because:
> 
> 1) 'git submodule init' operates on *all* submodules if 'submodule.active' is unset
>     and not <path> is given.
>     (see submodule--helper.c::module_init), or the doc [1].
> 2) 'git submodule init' sets 'submodule.$name.active' to true for the submodules
>     on which it operates, unless already covered by 'submodule.active'
>     (see submodule--helper.c::init_submodule)
> 3) the code we're adding to set 'active' to false if 'update=none' is only executed
>    if 'submodule.c.update' is not yet in the config, so it gets skipped if we
>    repeat 'git submodule init'. (I think this behaviour is sound).
> 
> So that's unfortunate, and is also kind of contradictory to what the doc says
> for 'git submodule init':
> "This command does not alter existing information in .git/config.".
> And just to be clear, the behaviour I describe above is already existing, the current
> patch just makes it more obvious.

Right, I noticed that.

> I think we could manage to change that behaviour a bit
> in order to have 'submodule init' not modify the config for submodules which are already marked inactive,
> *unless* they are explitely matched by the pathspec on the command line.
> So we would have:
> 
>     git clone server client; cd client
>     git submodule init      # initial call sets 'submodule.c.active=false'
>     git submodule init      # does not touch c, it's already marked inactive
>     git submodule init c    # OK, we really want to mark it as active

We could also add an option, --all, to make it work on all submodules.
That's because previously "git submodule init" did work on all the
submodules in the repository in question, but it doesn't now because of
our change.

> So it's all a little bit complicated! But I think that with my suggestion above,
> i.e. that 'git submodule init', in the absence of 'submodule.active', would
> only switch inactive submodules to active if they are explicitely listed, then
> we could get a saner behaviour, at the expense of having to explicitely init
> 'update=none' submodules on the command line if we really want to '--checkout' :
>     git clone server client
>     git -C client submodule update --init        # first call: set c to inactive
>     git -C client submodule update --init        # no-op
>     git -C client submodule update --checkout    # does not clone c (currently quiet)
>     git -C client submodule update --checkout c  # does not clone c, but warns (current behaviour)
>     git -C client submodule init c               # sets c to active
>     git -C client submodule update --checkout    # clones c
> 
> where the last two command could be a single
> 'git submodule update --init --checkout c' and ideally the
> 4th command should also warn the user that they now have to explicitely 'init'
> c if they want to check it out, which could simply mean tweaking the already
> existing message in next_submodule_warn_missing to also check if
> the current submodule has 'update=none' and then display the warning
> (instead of just showing it if the submodule was listed on the command
> line, which is the current behaviour). Additionnaly, the warning should
> say "Maybe you want to use 'update --init %s'?", i.e. specify the path.
> 
> 
> What do you think of my suggestions ? I can help push this forward
> by contributing patches if we agree that we should go forward with
> this slight behaviour change in 'git submodule init' ...

With the modification of adding --all to init so users can get a
behavior a little more similar to the previous, yes, that sounds good.
It would be great if you'd be willing to send a few patches.

> > diff --git a/t/t5601-clone.sh b/t/t5601-clone.sh
> > index c0688467e7..efe6b13be0 100755
> > --- a/t/t5601-clone.sh
> > +++ b/t/t5601-clone.sh
> > @@ -752,6 +752,30 @@ test_expect_success 'batch missing blob request does not inadvertently try to fe
> >   	git clone --filter=blob:limit=0 "file://$(pwd)/server" client
> >   '
> > +test_expect_success 'clone with submodule with update=none is not active' '
> > +	rm -rf server client &&
> > +
> > +	test_create_repo server &&
> > +	echo a >server/a &&
> > +	echo b >server/b &&
> > +	git -C server add a b &&
> > +	git -C server commit -m x &&
> > +
> > +	echo aa >server/a &&
> > +	echo bb >server/b &&
> > +	git -C server submodule add --name c "$(pwd)/repo_for_submodule" c &&
> > +	git -C server config -f .gitmodules submodule.c.update none &&
> > +	git -C server add a b c .gitmodules &&
> > +	git -C server commit -m x &&
> > +
> > +	git clone --recurse-submodules server client &&
> > +	git -C client config submodule.c.active >actual &&
> > +	echo false >expected &&
> > +	test_cmp actual expected &&
> > +	# This would fail if the submodule were active, since it is not checked out.
> > +	git -C client reset --recurse-submodules --hard
> > +'
> 
> I think we might want to also test the non-recursive clone case as well,
> i.e. 'git clone' and then 'git submodule update --init', as well as
> subsequent calls to 'git submodule init' in light of my analysis above.
> 
> Also, the only place in the test suite that I could find where
> 'update=none' is tested is in t7406.35-38 in t7406-submodule-update.sh
> so maybe it would make more sense to put the test(s) there ?

Sure, I can do that.
-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]