[PATCH 0/4] t: replace incorrect test_must_fail usage (part 5)

[PATCH 0/4] t: replace incorrect test_must_fail usage (part 5)

[Denton Liu]

The overall scope of these patches is to replace inappropriate uses of
test_must_fail. IOW, we should only allow test_must_fail to run on `git`
and `test-tool`. Ultimately, we will conclude by making test_must_fail
error out on non-git commands. An advance view of the final series can
be found here[1].

This is the fifth part. It focuses on lib-submodule-update.sh and tests
that make use of it.

The first part can be found here[2]. The second part can be found
here[3]. The third part can be found here[4]. The fourth part can be
found here[5].

[1]: (may be rebased at any time) https://github.com/Denton-L/git/tree/ready/cleanup-test-must-fail2
[2]: https://lore.kernel.org/git/cover.1576583819.git.liu.denton@gmail.com/
[3]: https://lore.kernel.org/git/cover.1577454401.git.liu.denton@gmail.com/
[4]: https://lore.kernel.org/git/cover.1585209554.git.liu.denton@gmail.com/
[5]: https://lore.kernel.org/git/cover.1587372771.git.liu.denton@gmail.com/

Denton Liu (4):
  lib-submodule-update: add space after function name
  lib-submodule-update: consolidate --recurse-submodules
  lib-submodule-update: prepend "git" to $command
  lib-submodule-update: pass OVERWRITING_FAIL

 t/lib-submodule-update.sh        | 47 ++++++++++++++++++++------------
 t/t1013-read-tree-submodule.sh   |  4 +--
 t/t2013-checkout-submodule.sh    |  4 +--
 t/t3426-rebase-submodule.sh      |  8 +++---
 t/t3512-cherry-pick-submodule.sh |  2 +-
 t/t3513-revert-submodule.sh      | 22 +++++++++------
 t/t3906-stash-submodule.sh       | 17 +++++++-----
 t/t4137-apply-submodule.sh       | 10 ++++---
 t/t4255-am-submodule.sh          | 10 ++++---
 t/t5572-pull-submodule.sh        | 16 +++++------
 t/t6041-bisect-submodule.sh      | 35 +++++++++++++-----------
 t/t7112-reset-submodule.sh       |  6 ++--
 t/t7613-merge-submodule.sh       |  8 +++---
 13 files changed, 108 insertions(+), 81 deletions(-)

-- 
2.26.2.548.gbb00c8a0a9

[PATCH 1/4] lib-submodule-update: add space after function name

[Denton Liu]

In the shell scripts in this codebase, the usual style is to include a
space between the function name and the (). Add these missing spaces to
conform to the usual style of the code.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 64fc6487dd..a3732d3f6c 100644
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -183,7 +183,7 @@ test_git_directory_is_unchanged () {
 	)
 }
 
-test_git_directory_exists() {
+test_git_directory_exists () {
 	test -e ".git/modules/$1" &&
 	if test -f sub1/.git
 	then
@@ -309,7 +309,7 @@ test_submodule_content () {
 
 # Internal function; use test_submodule_switch() or
 # test_submodule_forced_switch() instead.
-test_submodule_switch_common() {
+test_submodule_switch_common () {
 	command="$1"
 	######################### Appearing submodule #########################
 	# Switching to a commit letting a submodule appear creates empty dir ...
@@ -631,7 +631,7 @@ test_submodule_forced_switch () {
 
 # Internal function; use test_submodule_switch_recursing_with_args() or
 # test_submodule_forced_switch_recursing_with_args() instead.
-test_submodule_recursing_with_args_common() {
+test_submodule_recursing_with_args_common () {
 	command="$1"
 
 	######################### Appearing submodule #########################
-- 
2.26.2.548.gbb00c8a0a9

[PATCH 2/4] lib-submodule-update: consolidate --recurse-submodules

[Denton Liu]

Both test_submodule_switch_recursing_with_args() and
test_submodule_forced_switch_recursing_with_args() call the internal
function test_submodule_recursing_with_args_common() with the final
argument of `--recurse-submodules`. Consolidate this duplication by
appending the argument in test_submodule_recursing_with_args_common().

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index a3732d3f6c..81457b4c31 100644
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -632,7 +632,7 @@ test_submodule_forced_switch () {
 # Internal function; use test_submodule_switch_recursing_with_args() or
 # test_submodule_forced_switch_recursing_with_args() instead.
 test_submodule_recursing_with_args_common () {
-	command="$1"
+	command="$1 --recurse-submodules"
 
 	######################### Appearing submodule #########################
 	# Switching to a commit letting a submodule appear checks it out ...
@@ -840,7 +840,7 @@ test_submodule_recursing_with_args_common () {
 # test_submodule_switch_recursing_with_args "$GIT_COMMAND"
 test_submodule_switch_recursing_with_args () {
 	cmd_args="$1"
-	command="git $cmd_args --recurse-submodules"
+	command="git $cmd_args"
 	test_submodule_recursing_with_args_common "$command"
 
 	RESULTDS=success
@@ -957,7 +957,7 @@ test_submodule_switch_recursing_with_args () {
 # away local changes in the superproject is allowed.
 test_submodule_forced_switch_recursing_with_args () {
 	cmd_args="$1"
-	command="git $cmd_args --recurse-submodules"
+	command="git $cmd_args"
 	test_submodule_recursing_with_args_common "$command"
 
 	RESULT=success
-- 
2.26.2.548.gbb00c8a0a9

Re: [PATCH 2/4] lib-submodule-update: consolidate --recurse-submodules

[Junio C Hamano]

Denton Liu <liu.denton@gmail.com> writes:

> Both test_submodule_switch_recursing_with_args() and
> test_submodule_forced_switch_recursing_with_args() call the internal
> function test_submodule_recursing_with_args_common() with the final
> argument of `--recurse-submodules`. Consolidate this duplication by
> appending the argument in test_submodule_recursing_with_args_common().
>

OK, I just made sure that these two are the only callers and they
both pass the "--recurse-submodules", so the patch looks quite
sensible.

Thanks.

> Signed-off-by: Denton Liu <liu.denton@gmail.com>
> ---
>  t/lib-submodule-update.sh | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
> index a3732d3f6c..81457b4c31 100644
> --- a/t/lib-submodule-update.sh
> +++ b/t/lib-submodule-update.sh
> @@ -632,7 +632,7 @@ test_submodule_forced_switch () {
>  # Internal function; use test_submodule_switch_recursing_with_args() or
>  # test_submodule_forced_switch_recursing_with_args() instead.
>  test_submodule_recursing_with_args_common () {
> -	command="$1"
> +	command="$1 --recurse-submodules"
>  
>  	######################### Appearing submodule #########################
>  	# Switching to a commit letting a submodule appear checks it out ...
> @@ -840,7 +840,7 @@ test_submodule_recursing_with_args_common () {
>  # test_submodule_switch_recursing_with_args "$GIT_COMMAND"
>  test_submodule_switch_recursing_with_args () {
>  	cmd_args="$1"
> -	command="git $cmd_args --recurse-submodules"
> +	command="git $cmd_args"
>  	test_submodule_recursing_with_args_common "$command"
>  
>  	RESULTDS=success
> @@ -957,7 +957,7 @@ test_submodule_switch_recursing_with_args () {
>  # away local changes in the superproject is allowed.
>  test_submodule_forced_switch_recursing_with_args () {
>  	cmd_args="$1"
> -	command="git $cmd_args --recurse-submodules"
> +	command="git $cmd_args"
>  	test_submodule_recursing_with_args_common "$command"
>  
>  	RESULT=success

[PATCH 3/4] lib-submodule-update: prepend "git" to $command

[Denton Liu]

Since all invocations of test_submodule_forced_switch() are git
commands, automatically prepend "git" before invoking
test_submodule_switch_common().

Similarly, many invocations of test_submodule_switch() are also git
commands so automatically prepend "git" before invoking
test_submodule_switch_common() as well.

Finally, for invocations of test_submodule_switch() that invoke a custom
function, rename the old function to test_submodule_switch_func().

This is necessary because in a future commit, we will be adding some
logic that needs to distinguish between an invocation of a plain git
comamnd and an invocation of a test helper function.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh        | 14 +++++++++-----
 t/t1013-read-tree-submodule.sh   |  4 ++--
 t/t2013-checkout-submodule.sh    |  4 ++--
 t/t3426-rebase-submodule.sh      |  4 ++--
 t/t3512-cherry-pick-submodule.sh |  2 +-
 t/t3513-revert-submodule.sh      |  2 +-
 t/t3906-stash-submodule.sh       |  2 +-
 t/t4137-apply-submodule.sh       |  4 ++--
 t/t4255-am-submodule.sh          |  4 ++--
 t/t5572-pull-submodule.sh        |  8 ++++----
 t/t6041-bisect-submodule.sh      |  2 +-
 t/t7112-reset-submodule.sh       |  6 +++---
 t/t7613-merge-submodule.sh       |  8 ++++----
 13 files changed, 34 insertions(+), 30 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 81457b4c31..cd80d77707 100644
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -307,8 +307,8 @@ test_submodule_content () {
 # to protect the history!
 #
 
-# Internal function; use test_submodule_switch() or
-# test_submodule_forced_switch() instead.
+# Internal function; use test_submodule_switch_func(), test_submodule_switch_func(),
+# or test_submodule_forced_switch() instead.
 test_submodule_switch_common () {
 	command="$1"
 	######################### Appearing submodule #########################
@@ -566,8 +566,8 @@ test_submodule_switch_common () {
 #   # Do something here that updates the worktree and index to match target,
 #   # but not any submodule directories.
 # }
-# test_submodule_switch "my_func"
-test_submodule_switch () {
+# test_submodule_switch_func "my_func"
+test_submodule_switch_func () {
 	command="$1"
 	test_submodule_switch_common "$command"
 
@@ -587,12 +587,16 @@ test_submodule_switch () {
 	'
 }
 
+test_submodule_switch () {
+	test_submodule_switch_func "git $1"
+}
+
 # Same as test_submodule_switch(), except that throwing away local changes in
 # the superproject is allowed.
 test_submodule_forced_switch () {
 	command="$1"
 	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
-	test_submodule_switch_common "$command"
+	test_submodule_switch_common "git $command"
 
 	# When forced, a file in the superproject does not prevent creating a
 	# submodule of the same name.
diff --git a/t/t1013-read-tree-submodule.sh b/t/t1013-read-tree-submodule.sh
index 91a6fafcb4..b6df7444c0 100755
--- a/t/t1013-read-tree-submodule.sh
+++ b/t/t1013-read-tree-submodule.sh
@@ -12,8 +12,8 @@ test_submodule_switch_recursing_with_args "read-tree -u -m"
 
 test_submodule_forced_switch_recursing_with_args "read-tree -u --reset"
 
-test_submodule_switch "git read-tree -u -m"
+test_submodule_switch "read-tree -u -m"
 
-test_submodule_forced_switch "git read-tree -u --reset"
+test_submodule_forced_switch "read-tree -u --reset"
 
 test_done
diff --git a/t/t2013-checkout-submodule.sh b/t/t2013-checkout-submodule.sh
index 8f86b5f4b2..b2bdd1fcb4 100755
--- a/t/t2013-checkout-submodule.sh
+++ b/t/t2013-checkout-submodule.sh
@@ -68,8 +68,8 @@ test_submodule_switch_recursing_with_args "checkout"
 
 test_submodule_forced_switch_recursing_with_args "checkout -f"
 
-test_submodule_switch "git checkout"
+test_submodule_switch "checkout"
 
-test_submodule_forced_switch "git checkout -f"
+test_submodule_forced_switch "checkout -f"
 
 test_done
diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
index a2bba04ba9..788605ccc0 100755
--- a/t/t3426-rebase-submodule.sh
+++ b/t/t3426-rebase-submodule.sh
@@ -20,7 +20,7 @@ git_rebase () {
 	git rebase "$1"
 }
 
-test_submodule_switch "git_rebase"
+test_submodule_switch_func "git_rebase"
 
 git_rebase_interactive () {
 	git status -su >expect &&
@@ -38,7 +38,7 @@ git_rebase_interactive () {
 	git rebase -i "$1"
 }
 
-test_submodule_switch "git_rebase_interactive"
+test_submodule_switch_func "git_rebase_interactive"
 
 test_expect_success 'rebase interactive ignores modified submodules' '
 	test_when_finished "rm -rf super sub" &&
diff --git a/t/t3512-cherry-pick-submodule.sh b/t/t3512-cherry-pick-submodule.sh
index bd78287841..6ece1d8573 100755
--- a/t/t3512-cherry-pick-submodule.sh
+++ b/t/t3512-cherry-pick-submodule.sh
@@ -7,7 +7,7 @@ test_description='cherry-pick can handle submodules'
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "git cherry-pick"
+test_submodule_switch "cherry-pick"
 
 test_expect_success 'unrelated submodule/file conflict is ignored' '
 	test_create_repo sub &&
diff --git a/t/t3513-revert-submodule.sh b/t/t3513-revert-submodule.sh
index 5e39fcdb66..95a7f64471 100755
--- a/t/t3513-revert-submodule.sh
+++ b/t/t3513-revert-submodule.sh
@@ -26,6 +26,6 @@ git_revert () {
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
-test_submodule_switch "git_revert"
+test_submodule_switch_func "git_revert"
 
 test_done
diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
index b93d1d74da..6a7e801ca0 100755
--- a/t/t3906-stash-submodule.sh
+++ b/t/t3906-stash-submodule.sh
@@ -19,7 +19,7 @@ git_stash () {
 KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES=1
 KNOWN_FAILURE_CHERRY_PICK_SEES_EMPTY_COMMIT=1
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
-test_submodule_switch "git_stash"
+test_submodule_switch_func "git_stash"
 
 setup_basic () {
 	test_when_finished "rm -rf main sub" &&
diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
index a9bd40a6d0..b645e303a0 100755
--- a/t/t4137-apply-submodule.sh
+++ b/t/t4137-apply-submodule.sh
@@ -9,12 +9,12 @@ apply_index () {
 	git diff --ignore-submodules=dirty "..$1" | git apply --index -
 }
 
-test_submodule_switch "apply_index"
+test_submodule_switch_func "apply_index"
 
 apply_3way () {
 	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
 }
 
-test_submodule_switch "apply_3way"
+test_submodule_switch_func "apply_3way"
 
 test_done
diff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh
index 0ba8194403..1b179d5f45 100755
--- a/t/t4255-am-submodule.sh
+++ b/t/t4255-am-submodule.sh
@@ -9,14 +9,14 @@ am () {
 	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am -
 }
 
-test_submodule_switch "am"
+test_submodule_switch_func "am"
 
 am_3way () {
 	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am --3way -
 }
 
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "am_3way"
+test_submodule_switch_func "am_3way"
 
 test_expect_success 'setup diff.submodule' '
 	test_commit one &&
diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
index f916729a12..f911bf631e 100755
--- a/t/t5572-pull-submodule.sh
+++ b/t/t5572-pull-submodule.sh
@@ -17,21 +17,21 @@ git_pull () {
 }
 
 # pulls without conflicts
-test_submodule_switch "git_pull"
+test_submodule_switch_func "git_pull"
 
 git_pull_ff () {
 	reset_branch_to_HEAD "$1" &&
 	git pull --ff
 }
 
-test_submodule_switch "git_pull_ff"
+test_submodule_switch_func "git_pull_ff"
 
 git_pull_ff_only () {
 	reset_branch_to_HEAD "$1" &&
 	git pull --ff-only
 }
 
-test_submodule_switch "git_pull_ff_only"
+test_submodule_switch_func "git_pull_ff_only"
 
 git_pull_noff () {
 	reset_branch_to_HEAD "$1" &&
@@ -40,7 +40,7 @@ git_pull_noff () {
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "git_pull_noff"
+test_submodule_switch_func "git_pull_noff"
 
 test_expect_success 'pull --recurse-submodule setup' '
 	test_create_repo child &&
diff --git a/t/t6041-bisect-submodule.sh b/t/t6041-bisect-submodule.sh
index 62b8a2e7bb..0e0cdf638d 100755
--- a/t/t6041-bisect-submodule.sh
+++ b/t/t6041-bisect-submodule.sh
@@ -27,6 +27,6 @@ git_bisect () {
 	git bisect bad $BAD
 }
 
-test_submodule_switch "git_bisect"
+test_submodule_switch_func "git_bisect"
 
 test_done
diff --git a/t/t7112-reset-submodule.sh b/t/t7112-reset-submodule.sh
index 67346424a5..19830d9036 100755
--- a/t/t7112-reset-submodule.sh
+++ b/t/t7112-reset-submodule.sh
@@ -12,10 +12,10 @@ test_submodule_switch_recursing_with_args "reset --keep"
 
 test_submodule_forced_switch_recursing_with_args "reset --hard"
 
-test_submodule_switch "git reset --keep"
+test_submodule_switch "reset --keep"
 
-test_submodule_switch "git reset --merge"
+test_submodule_switch "reset --merge"
 
-test_submodule_forced_switch "git reset --hard"
+test_submodule_forced_switch "reset --hard"
 
 test_done
diff --git a/t/t7613-merge-submodule.sh b/t/t7613-merge-submodule.sh
index d1e9fcc781..04bf4be7d7 100755
--- a/t/t7613-merge-submodule.sh
+++ b/t/t7613-merge-submodule.sh
@@ -6,14 +6,14 @@ test_description='merge can handle submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 # merges without conflicts
-test_submodule_switch "git merge"
+test_submodule_switch "merge"
 
-test_submodule_switch "git merge --ff"
+test_submodule_switch "merge --ff"
 
-test_submodule_switch "git merge --ff-only"
+test_submodule_switch "merge --ff-only"
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "git merge --no-ff"
+test_submodule_switch "merge --no-ff"
 
 test_done
-- 
2.26.2.548.gbb00c8a0a9

[PATCH 4/4] lib-submodule-update: pass OVERWRITING_FAIL

[Denton Liu]

We are using `test_must_fail $command`. However, $command is not
necessarily a git command; it could be a test helper function.

In an effort to stop using test_must_fail with non-git commands, instead
of invoking `test_must_fail $command`, run
`OVERWRITING_FAIL=test_must_fail $command` instead. Increase the
granularity of the test helper functions by specifically choosing the
individual git invocation which is designed to fail.

While we're at it, some helper functions have git commands piping into
another git command. Break these pipes up into two separate invocations
with a file buffer so that the return code of the first command is not
lost.

This patch can be better viewed with `--ignore-all-space`.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh   | 25 +++++++++++++++++--------
 t/t3426-rebase-submodule.sh |  4 ++--
 t/t3513-revert-submodule.sh | 20 ++++++++++++--------
 t/t3906-stash-submodule.sh  | 15 +++++++++------
 t/t4137-apply-submodule.sh  |  6 ++++--
 t/t4255-am-submodule.sh     |  6 ++++--
 t/t5572-pull-submodule.sh   |  8 ++++----
 t/t6041-bisect-submodule.sh | 33 ++++++++++++++++++---------------
 8 files changed, 70 insertions(+), 47 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index cd80d77707..c8d1b6f528 100644
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -304,12 +304,15 @@ test_submodule_content () {
 # a removed submodule.
 #
 # Removing a submodule containing a .git directory must fail even when forced
-# to protect the history!
+# to protect the history! If we are testing this case,
+# OVERWRITING_FAIL=test_must_fail, otherwise OVERWRITING_FAIL will be the empty
+# string.
 #
 
 # Internal function; use test_submodule_switch_func(), test_submodule_switch_func(),
 # or test_submodule_forced_switch() instead.
 test_submodule_switch_common () {
+	OVERWRITING_FAIL=
 	command="$1"
 	######################### Appearing submodule #########################
 	# Switching to a commit letting a submodule appear creates empty dir ...
@@ -443,7 +446,7 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
-			test_must_fail $command replace_sub1_with_directory &&
+			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_directory &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -456,7 +459,7 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_directory &&
+			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_directory &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -470,7 +473,7 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
-			test_must_fail $command replace_sub1_with_file &&
+			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_file &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -484,7 +487,7 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_file &&
+			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_file &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -559,6 +562,11 @@ test_submodule_switch_common () {
 # conditions, set the appropriate KNOWN_FAILURE_* variable used in the tests
 # below to 1.
 #
+# Removing a submodule containing a .git directory must fail even when forced
+# to protect the history! If we are testing this case,
+# OVERWRITING_FAIL=test_must_fail, otherwise OVERWRITING_FAIL will be the empty
+# string.
+#
 # Use as follows:
 #
 # my_func () {
@@ -568,6 +576,7 @@ test_submodule_switch_common () {
 # }
 # test_submodule_switch_func "my_func"
 test_submodule_switch_func () {
+	OVERWRITING_FAIL=
 	command="$1"
 	test_submodule_switch_common "$command"
 
@@ -580,7 +589,7 @@ test_submodule_switch_func () {
 			cd submodule_update &&
 			git branch -t add_sub1 origin/add_sub1 &&
 			>sub1 &&
-			test_must_fail $command add_sub1 &&
+			OVERWRITING_FAIL=test_must_fail $command add_sub1 &&
 			test_superproject_content origin/no_submodule &&
 			test_must_be_empty sub1
 		)
@@ -588,7 +597,7 @@ test_submodule_switch_func () {
 }
 
 test_submodule_switch () {
-	test_submodule_switch_func "git $1"
+	test_submodule_switch_func "eval \$OVERWRITING_FAIL git $1"
 }
 
 # Same as test_submodule_switch(), except that throwing away local changes in
@@ -596,7 +605,7 @@ test_submodule_switch () {
 test_submodule_forced_switch () {
 	command="$1"
 	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
-	test_submodule_switch_common "git $command"
+	test_submodule_switch_common "eval \$OVERWRITING_FAIL git $command"
 
 	# When forced, a file in the superproject does not prevent creating a
 	# submodule of the same name.
diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
index 788605ccc0..c6a7f584ed 100755
--- a/t/t3426-rebase-submodule.sh
+++ b/t/t3426-rebase-submodule.sh
@@ -17,7 +17,7 @@ git_rebase () {
 	git status -su >actual &&
 	ls -1pR * >>actual &&
 	test_cmp expect actual &&
-	git rebase "$1"
+	$OVERWRITING_FAIL git rebase "$1"
 }
 
 test_submodule_switch_func "git_rebase"
@@ -35,7 +35,7 @@ git_rebase_interactive () {
 	test_cmp expect actual &&
 	set_fake_editor &&
 	echo "fake-editor.sh" >.git/info/exclude &&
-	git rebase -i "$1"
+	$OVERWRITING_FAIL git rebase -i "$1"
 }
 
 test_submodule_switch_func "git_rebase_interactive"
diff --git a/t/t3513-revert-submodule.sh b/t/t3513-revert-submodule.sh
index 95a7f64471..6c899db7e1 100755
--- a/t/t3513-revert-submodule.sh
+++ b/t/t3513-revert-submodule.sh
@@ -15,14 +15,18 @@ git_revert () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
-	git checkout "$1" &&
-	git revert HEAD &&
-	rm -rf * &&
-	tar xf "$TRASH_DIRECTORY/tmp.tar" &&
-	git status -su >actual &&
-	ls -1pR * >>actual &&
-	test_cmp expect actual &&
-	git revert HEAD
+	$OVERWRITING_FAIL git checkout "$1" &&
+	if test -z "$OVERWRITING_FAIL"
+	then
+		git checkout "$1" &&
+		git revert HEAD &&
+		rm -rf * &&
+		tar xf "$TRASH_DIRECTORY/tmp.tar" &&
+		git status -su >actual &&
+		ls -1pR * >>actual &&
+		test_cmp expect actual &&
+		git revert HEAD
+	fi
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
index 6a7e801ca0..860940072d 100755
--- a/t/t3906-stash-submodule.sh
+++ b/t/t3906-stash-submodule.sh
@@ -8,12 +8,15 @@ test_description='stash can handle submodules'
 git_stash () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
-	git read-tree -u -m "$1" &&
-	git stash &&
-	git status -su >actual &&
-	ls -1pR * >>actual &&
-	test_cmp expect actual &&
-	git stash apply
+	$OVERWRITING_FAIL git read-tree -u -m "$1" &&
+	if test -z "$OVERWRITING_FAIL"
+	then
+		git stash &&
+		git status -su >actual &&
+		ls -1pR * >>actual &&
+		test_cmp expect actual &&
+		git stash apply
+	fi
 }
 
 KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES=1
diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
index b645e303a0..dd08d9e1a4 100755
--- a/t/t4137-apply-submodule.sh
+++ b/t/t4137-apply-submodule.sh
@@ -6,13 +6,15 @@ test_description='git apply handling submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 apply_index () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --index -
+	git diff --ignore-submodules=dirty "..$1" >diff &&
+	$OVERWRITING_FAIL git apply --index - <diff
 }
 
 test_submodule_switch_func "apply_index"
 
 apply_3way () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
+	git diff --ignore-submodules=dirty "..$1" >diff &&
+	$OVERWRITING_FAIL git apply --3way - <diff
 }
 
 test_submodule_switch_func "apply_3way"
diff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh
index 1b179d5f45..b0fe78a956 100755
--- a/t/t4255-am-submodule.sh
+++ b/t/t4255-am-submodule.sh
@@ -6,13 +6,15 @@ test_description='git am handling submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 am () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am -
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
+	$OVERWRITING_FAIL git am - <patch
 }
 
 test_submodule_switch_func "am"
 
 am_3way () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am --3way -
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
+	$OVERWRITING_FAIL git am --3way - <patch
 }
 
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
index f911bf631e..4dd9913b6b 100755
--- a/t/t5572-pull-submodule.sh
+++ b/t/t5572-pull-submodule.sh
@@ -13,7 +13,7 @@ reset_branch_to_HEAD () {
 
 git_pull () {
 	reset_branch_to_HEAD "$1" &&
-	git pull
+	$OVERWRITING_FAIL git pull
 }
 
 # pulls without conflicts
@@ -21,21 +21,21 @@ test_submodule_switch_func "git_pull"
 
 git_pull_ff () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --ff
+	$OVERWRITING_FAIL git pull --ff
 }
 
 test_submodule_switch_func "git_pull_ff"
 
 git_pull_ff_only () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --ff-only
+	$OVERWRITING_FAIL git pull --ff-only
 }
 
 test_submodule_switch_func "git_pull_ff_only"
 
 git_pull_noff () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --no-ff
+	$OVERWRITING_FAIL git pull --no-ff
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
diff --git a/t/t6041-bisect-submodule.sh b/t/t6041-bisect-submodule.sh
index 0e0cdf638d..714d393899 100755
--- a/t/t6041-bisect-submodule.sh
+++ b/t/t6041-bisect-submodule.sh
@@ -10,21 +10,24 @@ git_bisect () {
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
 	GOOD=$(git rev-parse --verify HEAD) &&
-	git checkout "$1" &&
-	echo "foo" >bar &&
-	git add bar &&
-	git commit -m "bisect bad" &&
-	BAD=$(git rev-parse --verify HEAD) &&
-	git reset --hard HEAD^^ &&
-	git submodule update &&
-	git bisect start &&
-	git bisect good $GOOD &&
-	rm -rf * &&
-	tar xf "$TRASH_DIRECTORY/tmp.tar" &&
-	git status -su >actual &&
-	ls -1pR * >>actual &&
-	test_cmp expect actual &&
-	git bisect bad $BAD
+	$OVERWRITING_FAIL git checkout "$1" &&
+	if test -z "$OVERWRITING_FAIL"
+	then
+		echo "foo" >bar &&
+		git add bar &&
+		git commit -m "bisect bad" &&
+		BAD=$(git rev-parse --verify HEAD) &&
+		git reset --hard HEAD^^ &&
+		git submodule update &&
+		git bisect start &&
+		git bisect good $GOOD &&
+		rm -rf * &&
+		tar xf "$TRASH_DIRECTORY/tmp.tar" &&
+		git status -su >actual &&
+		ls -1pR * >>actual &&
+		test_cmp expect actual &&
+		git bisect bad $BAD
+	fi
 }
 
 test_submodule_switch_func "git_bisect"
-- 
2.26.2.548.gbb00c8a0a9

Re: [PATCH 4/4] lib-submodule-update: pass OVERWRITING_FAIL

[Junio C Hamano]

Denton Liu <liu.denton@gmail.com> writes:

> We are using `test_must_fail $command`. However, $command is not
> necessarily a git command; it could be a test helper function.
>
> In an effort to stop using test_must_fail with non-git commands, instead
> of invoking `test_must_fail $command`, run
> `OVERWRITING_FAIL=test_must_fail $command` instead. Increase the
> granularity of the test helper functions by specifically choosing the
> individual git invocation which is designed to fail.

Sorry, but I do not know why this is a good idea.

>  test_submodule_switch_common () {
> +	OVERWRITING_FAIL=
>  	command="$1"
>  	######################### Appearing submodule #########################
>  	# Switching to a commit letting a submodule appear creates empty dir ...
> @@ -443,7 +446,7 @@ test_submodule_switch_common () {
>  		(
>  			cd submodule_update &&
>  			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
> -			test_must_fail $command replace_sub1_with_directory &&
> +			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_directory &&

Here, $command may or may not be a git command and more importantly,
it could be a shell function, right?  Then we need to take it into
account that 

	VAR=VAL shell_function args...

will not work, no?

Some shells do not make this a single-shot environment variable
assignment that will not persist once the single function invocation
returns.

Re: [PATCH 4/4] lib-submodule-update: pass OVERWRITING_FAIL

[Denton Liu]

Hi Junio,

On Wed, Apr 29, 2020 at 12:24:48PM -0700, Junio C Hamano wrote:
> Denton Liu <liu.denton@gmail.com> writes:
> 
> > We are using `test_must_fail $command`. However, $command is not
> > necessarily a git command; it could be a test helper function.
> >
> > In an effort to stop using test_must_fail with non-git commands, instead
> > of invoking `test_must_fail $command`, run
> > `OVERWRITING_FAIL=test_must_fail $command` instead. Increase the
> > granularity of the test helper functions by specifically choosing the
> > individual git invocation which is designed to fail.
> 
> Sorry, but I do not know why this is a good idea.

This is useful because currently, when we run a test helper function, we
just mark the whole thing as `test_must_fail`. However, it's possible
that the helper function might fail earlier or later than expected due
to some side effect. If this happens, then the test case will still
be reported as passing but I think that it should be marked as failing
instead since it didn't actually display the desired behaviour.

> >  test_submodule_switch_common () {
> > +	OVERWRITING_FAIL=
> >  	command="$1"
> >  	######################### Appearing submodule #########################
> >  	# Switching to a commit letting a submodule appear creates empty dir ...
> > @@ -443,7 +446,7 @@ test_submodule_switch_common () {
> >  		(
> >  			cd submodule_update &&
> >  			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
> > -			test_must_fail $command replace_sub1_with_directory &&
> > +			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_directory &&
> 
> Here, $command may or may not be a git command and more importantly,
> it could be a shell function, right?  Then we need to take it into
> account that 
> 
> 	VAR=VAL shell_function args...
> 
> will not work, no?
> 
> Some shells do not make this a single-shot environment variable
> assignment that will not persist once the single function invocation
> returns.

I looked through POSIX specification and it says under 2.9.1 Simple
Comamnds,

	If no command name results, variable assignments shall affect
	the current execution environment. Otherwise, the variable
	assignments shall be exported for the execution environment of
	the command and shall not affect the current execution
	environment (except for special built-ins).

which makes me suspect that these shells are not POSIX-compliant. What
are some examples of shells that behave this way?

That being said, I know that we live in the real world and POSIX
standards don't justify breaking test cases. I'll reroll this to add an
`OVERWRITING_FAIL=` after the command to ensure that the variable is
cleared.

Thanks,

Denton

Re: [PATCH 4/4] lib-submodule-update: pass OVERWRITING_FAIL

[Junio C Hamano]

Denton Liu <liu.denton@gmail.com> writes:

>> Here, $command may or may not be a git command and more importantly,
>> it could be a shell function, right?  Then we need to take it into
>> account that 
>> 
>> 	VAR=VAL shell_function args...
>> 
>> will not work, no?
>> 
>> Some shells do not make this a single-shot environment variable
>> assignment that will not persist once the single function invocation
>> returns.
>
> ...
> which makes me suspect that these shells are not POSIX-compliant. What
> are some examples of shells that behave this way?

I think the most relevant thread is the one that contains this
message:

  https://public-inbox.org/git/7vljfzz0yd.fsf@alter.siamese.dyndns.org/

FWIW, shells that do not retain the assignment after a function
returns are not POSIX compliant.

Re: [PATCH 4/4] lib-submodule-update: pass OVERWRITING_FAIL

[Denton Liu]

On Wed, Apr 29, 2020 at 08:41:23PM -0700, Junio C Hamano wrote:
> Denton Liu <liu.denton@gmail.com> writes:
> 
> >> Here, $command may or may not be a git command and more importantly,
> >> it could be a shell function, right?  Then we need to take it into
> >> account that 
> >> 
> >> 	VAR=VAL shell_function args...
> >> 
> >> will not work, no?
> >> 
> >> Some shells do not make this a single-shot environment variable
> >> assignment that will not persist once the single function invocation
> >> returns.
> >
> > ...
> > which makes me suspect that these shells are not POSIX-compliant. What
> > are some examples of shells that behave this way?
> 
> I think the most relevant thread is the one that contains this
> message:
> 
>   https://public-inbox.org/git/7vljfzz0yd.fsf@alter.siamese.dyndns.org/
> 
> FWIW, shells that do not retain the assignment after a function
> returns are not POSIX compliant.

Hmm, interesting. Running an experiment:

	$ cat test.sh
	f () {
		echo $var
	}

	var=test f
	echo $var
	$ bash test.sh
	test

	$ bash --posix test.sh
	test
	test
	$ dash test.sh
	test


So it seems like there's a bug in dash. Guess it's time to file a bug
report...

[PATCH v1.1] lib-submodule-update: pass OVERWRITING_FAIL

[Denton Liu]

We are using `test_must_fail $command`. However, $command is not
necessarily a git command; it could be a test helper function.

In an effort to stop using test_must_fail with non-git commands, instead
of invoking `test_must_fail $command`, run
`OVERWRITING_FAIL=test_must_fail $command` instead. Increase the
granularity of the test helper functions by specifically choosing the
individual git invocation which is designed to fail.

This is useful because currently, when we run a test helper function, we
just mark the whole thing as `test_must_fail`. However, it's possible
that the helper function might fail earlier or later than expected due
to an introduced bug. If this happens, then the test case will still
report as passing but it should really be marked as failing since it
didn't actually display the intended behaviour.

While we're at it, some helper functions have git commands piping into
another git command. Break these pipes up into two separate invocations
with a file buffer so that the return code of the first command is not
lost.

This patch can be better viewed with `--ignore-all-space`.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
Range-diff against v1:
1:  22eacd20a8 ! 1:  f0d8dcf7dc lib-submodule-update: pass OVERWRITING_FAIL
    @@ Commit message
         granularity of the test helper functions by specifically choosing the
         individual git invocation which is designed to fail.
     
    +    This is useful because currently, when we run a test helper function, we
    +    just mark the whole thing as `test_must_fail`. However, it's possible
    +    that the helper function might fail earlier or later than expected due
    +    to an introduced bug. If this happens, then the test case will still
    +    report as passing but it should really be marked as failing since it
    +    didn't actually display the intended behaviour.
    +
         While we're at it, some helper functions have git commands piping into
         another git command. Break these pipes up into two separate invocations
         with a file buffer so that the return code of the first command is not
    @@ t/lib-submodule-update.sh: test_submodule_switch_common () {
      			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
     -			test_must_fail $command replace_sub1_with_directory &&
     +			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_directory &&
    ++			OVERWRITING_FAIL= &&
      			test_superproject_content origin/add_sub1 &&
      			test_submodule_content sub1 origin/add_sub1
      		)

 t/lib-submodule-update.sh   | 26 ++++++++++++++++++--------
 t/t3426-rebase-submodule.sh |  4 ++--
 t/t3513-revert-submodule.sh | 20 ++++++++++++--------
 t/t3906-stash-submodule.sh  | 15 +++++++++------
 t/t4137-apply-submodule.sh  |  6 ++++--
 t/t4255-am-submodule.sh     |  6 ++++--
 t/t5572-pull-submodule.sh   |  8 ++++----
 t/t6041-bisect-submodule.sh | 33 ++++++++++++++++++---------------
 8 files changed, 71 insertions(+), 47 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index cd80d77707..f61c3b43df 100644
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -304,12 +304,15 @@ test_submodule_content () {
 # a removed submodule.
 #
 # Removing a submodule containing a .git directory must fail even when forced
-# to protect the history!
+# to protect the history! If we are testing this case,
+# OVERWRITING_FAIL=test_must_fail, otherwise OVERWRITING_FAIL will be the empty
+# string.
 #
 
 # Internal function; use test_submodule_switch_func(), test_submodule_switch_func(),
 # or test_submodule_forced_switch() instead.
 test_submodule_switch_common () {
+	OVERWRITING_FAIL=
 	command="$1"
 	######################### Appearing submodule #########################
 	# Switching to a commit letting a submodule appear creates empty dir ...
@@ -443,7 +446,8 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
-			test_must_fail $command replace_sub1_with_directory &&
+			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_directory &&
+			OVERWRITING_FAIL= &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -456,7 +460,7 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_directory &&
+			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_directory &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -470,7 +474,7 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
-			test_must_fail $command replace_sub1_with_file &&
+			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_file &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -484,7 +488,7 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_file &&
+			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_file &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -559,6 +563,11 @@ test_submodule_switch_common () {
 # conditions, set the appropriate KNOWN_FAILURE_* variable used in the tests
 # below to 1.
 #
+# Removing a submodule containing a .git directory must fail even when forced
+# to protect the history! If we are testing this case,
+# OVERWRITING_FAIL=test_must_fail, otherwise OVERWRITING_FAIL will be the empty
+# string.
+#
 # Use as follows:
 #
 # my_func () {
@@ -568,6 +577,7 @@ test_submodule_switch_common () {
 # }
 # test_submodule_switch_func "my_func"
 test_submodule_switch_func () {
+	OVERWRITING_FAIL=
 	command="$1"
 	test_submodule_switch_common "$command"
 
@@ -580,7 +590,7 @@ test_submodule_switch_func () {
 			cd submodule_update &&
 			git branch -t add_sub1 origin/add_sub1 &&
 			>sub1 &&
-			test_must_fail $command add_sub1 &&
+			OVERWRITING_FAIL=test_must_fail $command add_sub1 &&
 			test_superproject_content origin/no_submodule &&
 			test_must_be_empty sub1
 		)
@@ -588,7 +598,7 @@ test_submodule_switch_func () {
 }
 
 test_submodule_switch () {
-	test_submodule_switch_func "git $1"
+	test_submodule_switch_func "eval \$OVERWRITING_FAIL git $1"
 }
 
 # Same as test_submodule_switch(), except that throwing away local changes in
@@ -596,7 +606,7 @@ test_submodule_switch () {
 test_submodule_forced_switch () {
 	command="$1"
 	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
-	test_submodule_switch_common "git $command"
+	test_submodule_switch_common "eval \$OVERWRITING_FAIL git $command"
 
 	# When forced, a file in the superproject does not prevent creating a
 	# submodule of the same name.
diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
index 788605ccc0..c6a7f584ed 100755
--- a/t/t3426-rebase-submodule.sh
+++ b/t/t3426-rebase-submodule.sh
@@ -17,7 +17,7 @@ git_rebase () {
 	git status -su >actual &&
 	ls -1pR * >>actual &&
 	test_cmp expect actual &&
-	git rebase "$1"
+	$OVERWRITING_FAIL git rebase "$1"
 }
 
 test_submodule_switch_func "git_rebase"
@@ -35,7 +35,7 @@ git_rebase_interactive () {
 	test_cmp expect actual &&
 	set_fake_editor &&
 	echo "fake-editor.sh" >.git/info/exclude &&
-	git rebase -i "$1"
+	$OVERWRITING_FAIL git rebase -i "$1"
 }
 
 test_submodule_switch_func "git_rebase_interactive"
diff --git a/t/t3513-revert-submodule.sh b/t/t3513-revert-submodule.sh
index 95a7f64471..6c899db7e1 100755
--- a/t/t3513-revert-submodule.sh
+++ b/t/t3513-revert-submodule.sh
@@ -15,14 +15,18 @@ git_revert () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
-	git checkout "$1" &&
-	git revert HEAD &&
-	rm -rf * &&
-	tar xf "$TRASH_DIRECTORY/tmp.tar" &&
-	git status -su >actual &&
-	ls -1pR * >>actual &&
-	test_cmp expect actual &&
-	git revert HEAD
+	$OVERWRITING_FAIL git checkout "$1" &&
+	if test -z "$OVERWRITING_FAIL"
+	then
+		git checkout "$1" &&
+		git revert HEAD &&
+		rm -rf * &&
+		tar xf "$TRASH_DIRECTORY/tmp.tar" &&
+		git status -su >actual &&
+		ls -1pR * >>actual &&
+		test_cmp expect actual &&
+		git revert HEAD
+	fi
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
index 6a7e801ca0..860940072d 100755
--- a/t/t3906-stash-submodule.sh
+++ b/t/t3906-stash-submodule.sh
@@ -8,12 +8,15 @@ test_description='stash can handle submodules'
 git_stash () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
-	git read-tree -u -m "$1" &&
-	git stash &&
-	git status -su >actual &&
-	ls -1pR * >>actual &&
-	test_cmp expect actual &&
-	git stash apply
+	$OVERWRITING_FAIL git read-tree -u -m "$1" &&
+	if test -z "$OVERWRITING_FAIL"
+	then
+		git stash &&
+		git status -su >actual &&
+		ls -1pR * >>actual &&
+		test_cmp expect actual &&
+		git stash apply
+	fi
 }
 
 KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES=1
diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
index b645e303a0..dd08d9e1a4 100755
--- a/t/t4137-apply-submodule.sh
+++ b/t/t4137-apply-submodule.sh
@@ -6,13 +6,15 @@ test_description='git apply handling submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 apply_index () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --index -
+	git diff --ignore-submodules=dirty "..$1" >diff &&
+	$OVERWRITING_FAIL git apply --index - <diff
 }
 
 test_submodule_switch_func "apply_index"
 
 apply_3way () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
+	git diff --ignore-submodules=dirty "..$1" >diff &&
+	$OVERWRITING_FAIL git apply --3way - <diff
 }
 
 test_submodule_switch_func "apply_3way"
diff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh
index 1b179d5f45..b0fe78a956 100755
--- a/t/t4255-am-submodule.sh
+++ b/t/t4255-am-submodule.sh
@@ -6,13 +6,15 @@ test_description='git am handling submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 am () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am -
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
+	$OVERWRITING_FAIL git am - <patch
 }
 
 test_submodule_switch_func "am"
 
 am_3way () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am --3way -
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
+	$OVERWRITING_FAIL git am --3way - <patch
 }
 
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
index f911bf631e..4dd9913b6b 100755
--- a/t/t5572-pull-submodule.sh
+++ b/t/t5572-pull-submodule.sh
@@ -13,7 +13,7 @@ reset_branch_to_HEAD () {
 
 git_pull () {
 	reset_branch_to_HEAD "$1" &&
-	git pull
+	$OVERWRITING_FAIL git pull
 }
 
 # pulls without conflicts
@@ -21,21 +21,21 @@ test_submodule_switch_func "git_pull"
 
 git_pull_ff () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --ff
+	$OVERWRITING_FAIL git pull --ff
 }
 
 test_submodule_switch_func "git_pull_ff"
 
 git_pull_ff_only () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --ff-only
+	$OVERWRITING_FAIL git pull --ff-only
 }
 
 test_submodule_switch_func "git_pull_ff_only"
 
 git_pull_noff () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --no-ff
+	$OVERWRITING_FAIL git pull --no-ff
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
diff --git a/t/t6041-bisect-submodule.sh b/t/t6041-bisect-submodule.sh
index 0e0cdf638d..714d393899 100755
--- a/t/t6041-bisect-submodule.sh
+++ b/t/t6041-bisect-submodule.sh
@@ -10,21 +10,24 @@ git_bisect () {
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
 	GOOD=$(git rev-parse --verify HEAD) &&
-	git checkout "$1" &&
-	echo "foo" >bar &&
-	git add bar &&
-	git commit -m "bisect bad" &&
-	BAD=$(git rev-parse --verify HEAD) &&
-	git reset --hard HEAD^^ &&
-	git submodule update &&
-	git bisect start &&
-	git bisect good $GOOD &&
-	rm -rf * &&
-	tar xf "$TRASH_DIRECTORY/tmp.tar" &&
-	git status -su >actual &&
-	ls -1pR * >>actual &&
-	test_cmp expect actual &&
-	git bisect bad $BAD
+	$OVERWRITING_FAIL git checkout "$1" &&
+	if test -z "$OVERWRITING_FAIL"
+	then
+		echo "foo" >bar &&
+		git add bar &&
+		git commit -m "bisect bad" &&
+		BAD=$(git rev-parse --verify HEAD) &&
+		git reset --hard HEAD^^ &&
+		git submodule update &&
+		git bisect start &&
+		git bisect good $GOOD &&
+		rm -rf * &&
+		tar xf "$TRASH_DIRECTORY/tmp.tar" &&
+		git status -su >actual &&
+		ls -1pR * >>actual &&
+		test_cmp expect actual &&
+		git bisect bad $BAD
+	fi
 }
 
 test_submodule_switch_func "git_bisect"
-- 
2.26.2.548.gbb00c8a0a9

Re: [PATCH v1.1] lib-submodule-update: pass OVERWRITING_FAIL

[Junio C Hamano]

Denton Liu <liu.denton@gmail.com> writes:

> We are using `test_must_fail $command`. However, $command is not
> necessarily a git command; it could be a test helper function.
>
> In an effort to stop using test_must_fail with non-git commands, instead
> of invoking `test_must_fail $command`, run
> `OVERWRITING_FAIL=test_must_fail $command` instead.

This description alone does not make much sense to me as a reader.
For "when OVERWRITING_FAIL environment variable is set to
test_must_fail, something magical happens while running $command" to
be true, $command (which may be "a git command", or "a test helper
function") must be aware of, and must be told how to react to the
OVERWRITING_FAIL environment variable.  For example,

	test_must_fail test a = b
	test_must_fail git cat-file leaf HEAD

may succeed because "test a = b" and "git cat-file leaf HEAD" fail
and test_must_fail would notice that these commands exited with
non-zero status without crashing.

But how would the same happen for these commands,

	OVERWRITING_FAIL=test_must_fail test a = b
	OVERWRITING_FAIL=test_must_fail git cat-file leaf HEAD

which is what the above paragraph tells me to write "instead of
using test_must_fail".  There is something gravely missing from the
description.  

Is it that $command is *NEVER* a 'git' command, but just a selected
few helper functions know how to honor this convention?  If that is
the case, perhaps can we teach these helper functions an *option* to
expect a failure instead of expecting a success?  These are all
speculations, because the above description is too vague as a
starting point for clear thinking.

> +			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_directory &&
> +			OVERWRITING_FAIL= &&

If we have given up the "single-shot environment export" for
compatibility reasons (which is a sound decision to follow), we
should make sure it is clear to our readers that we are not using
that shell feature.  I.e.

			export OVERWRITING_FAIL=test_must_fail &&
			$command replace_sub1_with_directory &&
			unset OVERWRITING_FAIL &&

I still do not understand how you are forcing an arbitrary $command
to honor this environment variable, though.

Re: [PATCH v1.1] lib-submodule-update: pass OVERWRITING_FAIL

[Denton Liu]

Hi Junio,

On Thu, Apr 30, 2020 at 01:38:59PM -0700, Junio C Hamano wrote:
> Denton Liu <liu.denton@gmail.com> writes:
> 
> > We are using `test_must_fail $command`. However, $command is not
> > necessarily a git command; it could be a test helper function.
> >
> > In an effort to stop using test_must_fail with non-git commands, instead
> > of invoking `test_must_fail $command`, run
> > `OVERWRITING_FAIL=test_must_fail $command` instead.
> 
> This description alone does not make much sense to me as a reader.
> For "when OVERWRITING_FAIL environment variable is set to
> test_must_fail, something magical happens while running $command" to
> be true, $command (which may be "a git command", or "a test helper
> function") must be aware of, and must be told how to react to the
> OVERWRITING_FAIL environment variable.  For example,
> 
> 	test_must_fail test a = b
> 	test_must_fail git cat-file leaf HEAD
> 
> may succeed because "test a = b" and "git cat-file leaf HEAD" fail
> and test_must_fail would notice that these commands exited with
> non-zero status without crashing.
> 
> But how would the same happen for these commands,
> 
> 	OVERWRITING_FAIL=test_must_fail test a = b
> 	OVERWRITING_FAIL=test_must_fail git cat-file leaf HEAD
> 
> which is what the above paragraph tells me to write "instead of
> using test_must_fail".  There is something gravely missing from the
> description.  

Because of the previous patch in this series, helper functions are
called using test_submodule_switch_func() while git commands are invoked
using test_submodule_switch() and test_submodule_forced_switch(). As a
result, helper functions that are invoked must learn to handle
$OVERWRITING_FAIL appropriately. The latter two functions now feature
something like

	test_submodule_switch_func "eval \$OVERWRITING_FAIL git $1"

Does this commit message increase the clarity?

	lib-submodule-update: pass OVERWRITING_FAIL

	We are using `test_must_fail $command`. However, $command is not
	necessarily a git command; it could be a test helper function.

	In an effort to stop using test_must_fail with non-git commands, instead
	of invoking `test_must_fail $command`, run
	`OVERWRITING_FAIL=test_must_fail $command` instead in
	test_submodule_switch_common().

	In the case where $command is a git command,
	test_submodule_switch_common() is called by one of
	test_submodule_switch() or test_submodule_forced_switch(). In those two
	functions, pass $command like this:

		test_submodule_switch_common "eval \$OVERWRITING_FAIL git $command"

	In the case where $command is a test helper function, increase the
	granularity of the test helper function by marking the git invocation
	which was meant to fail like this:

		$OVERWRITING_FAIL git checkout "$1"

	This is useful because currently, when we run a test helper function, we
	just mark the whole thing as `test_must_fail`. However, it's possible
	that the helper function might fail earlier or later than expected due
	to an introduced bug. If this happens, then the test case will still
	report as passing but it should really be marked as failing since it
	didn't actually display the intended behaviour.

	While we're at it, some helper functions have git commands piping into
	another git command. Break these pipes up into two separate invocations
	with a file buffer so that the return code of the first command is not
	lost.

	This patch can be better viewed with `--ignore-all-space`.

> Is it that $command is *NEVER* a 'git' command, but just a selected
> few helper functions know how to honor this convention?

With the changes made, it is now either a helper function or an eval of
a git command, as described above.

> If that is
> the case, perhaps can we teach these helper functions an *option* to
> expect a failure instead of expecting a success?

I don't think this is possible because $command is some arbitrary
command string, in particular due to the eval stuff. I suppose we could
write something like

	test_submodule_switch_common "f () {
			# handle potential --expect-fail
			git $command"
		} && f"

but I'm not sure if this would even work. I haven't tested it and I feel
like doing this would be far too unwieldy. Also, since
test_submodule_switch_common() uses $command as the test name, I don't
think it's feasible to take this approach.

> These are all
> speculations, because the above description is too vague as a
> starting point for clear thinking.
> 
> > +			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_directory &&
> > +			OVERWRITING_FAIL= &&
> 
> If we have given up the "single-shot environment export" for
> compatibility reasons (which is a sound decision to follow), we
> should make sure it is clear to our readers that we are not using
> that shell feature.  I.e.
> 
> 			export OVERWRITING_FAIL=test_must_fail &&
> 			$command replace_sub1_with_directory &&
> 			unset OVERWRITING_FAIL &&

Makes sense. I would drop the export, though, because $OVERWRITING_FAIL
should only be handled within the shell environment. We're never calling
any external commands that need to know about this variable.

On a tangent, I got a response[1] from the dash people about the
"single-shot environment export" propagating past a function. It seems
like in the most updated version of POSIX addresses this and it's up to
the implementers whether or not variables propagate past a function
invocation.

Thanks,

Denton

[1]: https://www.mail-archive.com/dash@vger.kernel.org/msg01925.html

Re: [PATCH v1.1] lib-submodule-update: pass OVERWRITING_FAIL

[Junio C Hamano]

Denton Liu <liu.denton@gmail.com> writes:

> Does this commit message increase the clarity?
> 
> 	lib-submodule-update: pass OVERWRITING_FAIL
>
> 	We are using `test_must_fail $command`. However, $command is not
> 	necessarily a git command; it could be a test helper function.
>
> 	In an effort to stop using test_must_fail with non-git commands, instead
> 	of invoking `test_must_fail $command`, run
> 	`OVERWRITING_FAIL=test_must_fail $command` instead in
> 	test_submodule_switch_common().
>
> 	In the case where $command is a git command,
> 	test_submodule_switch_common() is called by one of
> 	test_submodule_switch() or test_submodule_forced_switch(). In those two
> 	functions, pass $command like this:
>
> 		test_submodule_switch_common "eval \$OVERWRITING_FAIL git $command"
>
> 	In the case where $command is a test helper function, increase the
> 	granularity of the test helper function by marking the git invocation
> 	which was meant to fail like this:
>
> 		$OVERWRITING_FAIL git checkout "$1"
>
> 	This is useful because currently, when we run a test helper function, we
> 	just mark the whole thing as `test_must_fail`. However, it's possible
> 	that the helper function might fail earlier or later than expected due
> 	to an introduced bug. If this happens, then the test case will still
> 	report as passing but it should really be marked as failing since it
> 	didn't actually display the intended behaviour.
>
> 	While we're at it, some helper functions have git commands piping into
> 	another git command. Break these pipes up into two separate invocations
> 	with a file buffer so that the return code of the first command is not
> 	lost.
>
> 	This patch can be better viewed with `--ignore-all-space`.

It may be better, but not all that much.  I think it comes from the
design that this change is hard to understand.  Anybody who wants to
write more of these tests would need a much better guidance than
"just use OVERWRITING_FAIL=test_must_fail where you would normally
write test_must_fail and you'd be OK", as that is not what is going
on.  Before doing so, they would make sure that the place where they
are tempted to write test_must_fail MUST be called by these three
wrappers, or this guidance does not apply, for example.

>> If we have given up the "single-shot environment export" for
>> compatibility reasons (which is a sound decision to follow), we
>> should make sure it is clear to our readers that we are not using
>> that shell feature.  I.e.
>> 
>> 			export OVERWRITING_FAIL=test_must_fail &&
>> 			$command replace_sub1_with_directory &&
>> 			unset OVERWRITING_FAIL &&
>
> Makes sense. I would drop the export, though, because $OVERWRITING_FAIL
> should only be handled within the shell environment. We're never calling
> any external commands that need to know about this variable.

Yup, not pretending that this affects anywhere outside of shell by
exporting is a bad idea.

> On a tangent, I got a response[1] from the dash people about the
> "single-shot environment export" propagating past a function. It seems
> like in the most updated version of POSIX addresses this and it's up to
> the implementers whether or not variables propagate past a function
> invocation.

Yes, it is the reason why we discourage the unportable use of the
pattern in our tests.

[PATCH v1.2] lib-submodule-update: pass OVERWRITING_FAIL

[Denton Liu]

We are using `test_must_fail $command`. However, $command is not
necessarily a git command; it could be a test helper function.

In an effort to stop using test_must_fail with non-git commands, instead
of invoking `test_must_fail $command`, run
`OVERWRITING_FAIL=test_must_fail $command` instead in
test_submodule_switch_common().

In the case where $command is a git command,
test_submodule_switch_common() is called by one of
test_submodule_switch() or test_submodule_forced_switch(). In those two
functions, pass $command like this:

	test_submodule_switch_common "eval \$OVERWRITING_FAIL git $command"

When $command is a helper function, the parent function calling
test_submodule_switch_common() is test_submodule_switch_func(). For all
test_submodule_switch_func() invocations, increase the granularity of
the argument test helper function by prefixing the git invocation which is
meant to fail with $OVERWRITING_FAIL like this:

	$OVERWRITING_FAIL git checkout "$1"

This is useful because currently, when we run a test helper function, we
just mark the whole thing as `test_must_fail`. However, it's possible
that the helper function might fail earlier or later than expected due
to an introduced bug. If this happens, then the test case will still
report as passing but it should really be marked as failing since it
didn't actually display the intended behaviour.

While we're at it, some helper functions have git commands piping into
another git command. Break these pipes up into two separate invocations
with a file buffer so that the return code of the first command is not
lost.

This patch can be better viewed with `--ignore-all-space`.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
Changes since v1.1:

	* unset OVERWRITING_FAIL in more places that I missed earlier

	* clarify that the helper functions passed to
	  test_submodule_switch_func() are the ones that have been
	  modified (I'm not 100% if I understood your complaint with
	  the text so perhaps the commit message may still need further
	  refinement)

 t/lib-submodule-update.sh   | 33 +++++++++++++++++++++++++--------
 t/t3426-rebase-submodule.sh |  4 ++--
 t/t3513-revert-submodule.sh | 20 ++++++++++++--------
 t/t3906-stash-submodule.sh  | 15 +++++++++------
 t/t4137-apply-submodule.sh  |  6 ++++--
 t/t4255-am-submodule.sh     |  6 ++++--
 t/t5572-pull-submodule.sh   |  8 ++++----
 t/t6041-bisect-submodule.sh | 33 ++++++++++++++++++---------------
 8 files changed, 78 insertions(+), 47 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index cd80d77707..67d9d8615a 100644
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -304,12 +304,15 @@ test_submodule_content () {
 # a removed submodule.
 #
 # Removing a submodule containing a .git directory must fail even when forced
-# to protect the history!
+# to protect the history! If we are testing this case,
+# OVERWRITING_FAIL=test_must_fail, otherwise OVERWRITING_FAIL will be the empty
+# string.
 #
 
 # Internal function; use test_submodule_switch_func(), test_submodule_switch_func(),
 # or test_submodule_forced_switch() instead.
 test_submodule_switch_common () {
+	OVERWRITING_FAIL=
 	command="$1"
 	######################### Appearing submodule #########################
 	# Switching to a commit letting a submodule appear creates empty dir ...
@@ -443,7 +446,9 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
-			test_must_fail $command replace_sub1_with_directory &&
+			OVERWRITING_FAIL=test_must_fail &&
+			$command replace_sub1_with_directory &&
+			OVERWRITING_FAIL= &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -456,7 +461,9 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_directory &&
+			OVERWRITING_FAIL=test_must_fail &&
+			$command replace_sub1_with_directory &&
+			OVERWRITING_FAIL= &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -470,7 +477,9 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
-			test_must_fail $command replace_sub1_with_file &&
+			OVERWRITING_FAIL=test_must_fail &&
+			$command replace_sub1_with_file &&
+			OVERWRITING_FAIL= &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -484,7 +493,9 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_file &&
+			OVERWRITING_FAIL=test_must_fail &&
+			$command replace_sub1_with_file &&
+			OVERWRITING_FAIL= &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -559,6 +570,11 @@ test_submodule_switch_common () {
 # conditions, set the appropriate KNOWN_FAILURE_* variable used in the tests
 # below to 1.
 #
+# Removing a submodule containing a .git directory must fail even when forced
+# to protect the history! If we are testing this case,
+# OVERWRITING_FAIL=test_must_fail, otherwise OVERWRITING_FAIL will be the empty
+# string.
+#
 # Use as follows:
 #
 # my_func () {
@@ -568,6 +584,7 @@ test_submodule_switch_common () {
 # }
 # test_submodule_switch_func "my_func"
 test_submodule_switch_func () {
+	OVERWRITING_FAIL=
 	command="$1"
 	test_submodule_switch_common "$command"
 
@@ -580,7 +597,7 @@ test_submodule_switch_func () {
 			cd submodule_update &&
 			git branch -t add_sub1 origin/add_sub1 &&
 			>sub1 &&
-			test_must_fail $command add_sub1 &&
+			OVERWRITING_FAIL=test_must_fail $command add_sub1 &&
 			test_superproject_content origin/no_submodule &&
 			test_must_be_empty sub1
 		)
@@ -588,7 +605,7 @@ test_submodule_switch_func () {
 }
 
 test_submodule_switch () {
-	test_submodule_switch_func "git $1"
+	test_submodule_switch_func "eval \$OVERWRITING_FAIL git $1"
 }
 
 # Same as test_submodule_switch(), except that throwing away local changes in
@@ -596,7 +613,7 @@ test_submodule_switch () {
 test_submodule_forced_switch () {
 	command="$1"
 	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
-	test_submodule_switch_common "git $command"
+	test_submodule_switch_common "eval \$OVERWRITING_FAIL git $command"
 
 	# When forced, a file in the superproject does not prevent creating a
 	# submodule of the same name.
diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
index 788605ccc0..c6a7f584ed 100755
--- a/t/t3426-rebase-submodule.sh
+++ b/t/t3426-rebase-submodule.sh
@@ -17,7 +17,7 @@ git_rebase () {
 	git status -su >actual &&
 	ls -1pR * >>actual &&
 	test_cmp expect actual &&
-	git rebase "$1"
+	$OVERWRITING_FAIL git rebase "$1"
 }
 
 test_submodule_switch_func "git_rebase"
@@ -35,7 +35,7 @@ git_rebase_interactive () {
 	test_cmp expect actual &&
 	set_fake_editor &&
 	echo "fake-editor.sh" >.git/info/exclude &&
-	git rebase -i "$1"
+	$OVERWRITING_FAIL git rebase -i "$1"
 }
 
 test_submodule_switch_func "git_rebase_interactive"
diff --git a/t/t3513-revert-submodule.sh b/t/t3513-revert-submodule.sh
index 95a7f64471..6c899db7e1 100755
--- a/t/t3513-revert-submodule.sh
+++ b/t/t3513-revert-submodule.sh
@@ -15,14 +15,18 @@ git_revert () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
-	git checkout "$1" &&
-	git revert HEAD &&
-	rm -rf * &&
-	tar xf "$TRASH_DIRECTORY/tmp.tar" &&
-	git status -su >actual &&
-	ls -1pR * >>actual &&
-	test_cmp expect actual &&
-	git revert HEAD
+	$OVERWRITING_FAIL git checkout "$1" &&
+	if test -z "$OVERWRITING_FAIL"
+	then
+		git checkout "$1" &&
+		git revert HEAD &&
+		rm -rf * &&
+		tar xf "$TRASH_DIRECTORY/tmp.tar" &&
+		git status -su >actual &&
+		ls -1pR * >>actual &&
+		test_cmp expect actual &&
+		git revert HEAD
+	fi
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
index 6a7e801ca0..860940072d 100755
--- a/t/t3906-stash-submodule.sh
+++ b/t/t3906-stash-submodule.sh
@@ -8,12 +8,15 @@ test_description='stash can handle submodules'
 git_stash () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
-	git read-tree -u -m "$1" &&
-	git stash &&
-	git status -su >actual &&
-	ls -1pR * >>actual &&
-	test_cmp expect actual &&
-	git stash apply
+	$OVERWRITING_FAIL git read-tree -u -m "$1" &&
+	if test -z "$OVERWRITING_FAIL"
+	then
+		git stash &&
+		git status -su >actual &&
+		ls -1pR * >>actual &&
+		test_cmp expect actual &&
+		git stash apply
+	fi
 }
 
 KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES=1
diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
index b645e303a0..dd08d9e1a4 100755
--- a/t/t4137-apply-submodule.sh
+++ b/t/t4137-apply-submodule.sh
@@ -6,13 +6,15 @@ test_description='git apply handling submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 apply_index () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --index -
+	git diff --ignore-submodules=dirty "..$1" >diff &&
+	$OVERWRITING_FAIL git apply --index - <diff
 }
 
 test_submodule_switch_func "apply_index"
 
 apply_3way () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
+	git diff --ignore-submodules=dirty "..$1" >diff &&
+	$OVERWRITING_FAIL git apply --3way - <diff
 }
 
 test_submodule_switch_func "apply_3way"
diff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh
index 1b179d5f45..b0fe78a956 100755
--- a/t/t4255-am-submodule.sh
+++ b/t/t4255-am-submodule.sh
@@ -6,13 +6,15 @@ test_description='git am handling submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 am () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am -
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
+	$OVERWRITING_FAIL git am - <patch
 }
 
 test_submodule_switch_func "am"
 
 am_3way () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am --3way -
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
+	$OVERWRITING_FAIL git am --3way - <patch
 }
 
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
index f911bf631e..4dd9913b6b 100755
--- a/t/t5572-pull-submodule.sh
+++ b/t/t5572-pull-submodule.sh
@@ -13,7 +13,7 @@ reset_branch_to_HEAD () {
 
 git_pull () {
 	reset_branch_to_HEAD "$1" &&
-	git pull
+	$OVERWRITING_FAIL git pull
 }
 
 # pulls without conflicts
@@ -21,21 +21,21 @@ test_submodule_switch_func "git_pull"
 
 git_pull_ff () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --ff
+	$OVERWRITING_FAIL git pull --ff
 }
 
 test_submodule_switch_func "git_pull_ff"
 
 git_pull_ff_only () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --ff-only
+	$OVERWRITING_FAIL git pull --ff-only
 }
 
 test_submodule_switch_func "git_pull_ff_only"
 
 git_pull_noff () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --no-ff
+	$OVERWRITING_FAIL git pull --no-ff
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
diff --git a/t/t6041-bisect-submodule.sh b/t/t6041-bisect-submodule.sh
index 0e0cdf638d..714d393899 100755
--- a/t/t6041-bisect-submodule.sh
+++ b/t/t6041-bisect-submodule.sh
@@ -10,21 +10,24 @@ git_bisect () {
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
 	GOOD=$(git rev-parse --verify HEAD) &&
-	git checkout "$1" &&
-	echo "foo" >bar &&
-	git add bar &&
-	git commit -m "bisect bad" &&
-	BAD=$(git rev-parse --verify HEAD) &&
-	git reset --hard HEAD^^ &&
-	git submodule update &&
-	git bisect start &&
-	git bisect good $GOOD &&
-	rm -rf * &&
-	tar xf "$TRASH_DIRECTORY/tmp.tar" &&
-	git status -su >actual &&
-	ls -1pR * >>actual &&
-	test_cmp expect actual &&
-	git bisect bad $BAD
+	$OVERWRITING_FAIL git checkout "$1" &&
+	if test -z "$OVERWRITING_FAIL"
+	then
+		echo "foo" >bar &&
+		git add bar &&
+		git commit -m "bisect bad" &&
+		BAD=$(git rev-parse --verify HEAD) &&
+		git reset --hard HEAD^^ &&
+		git submodule update &&
+		git bisect start &&
+		git bisect good $GOOD &&
+		rm -rf * &&
+		tar xf "$TRASH_DIRECTORY/tmp.tar" &&
+		git status -su >actual &&
+		ls -1pR * >>actual &&
+		test_cmp expect actual &&
+		git bisect bad $BAD
+	fi
 }
 
 test_submodule_switch_func "git_bisect"
-- 
2.26.2.548.gbb00c8a0a9

Re: [PATCH 0/4] t: replace incorrect test_must_fail usage (part 5)

[Taylor Blau]

On Wed, Apr 29, 2020 at 08:22:22AM -0400, Denton Liu wrote:
> The overall scope of these patches is to replace inappropriate uses of
> test_must_fail. IOW, we should only allow test_must_fail to run on `git`
> and `test-tool`. Ultimately, we will conclude by making test_must_fail
> error out on non-git commands. An advance view of the final series can
> be found here[1].

This comment has nothing to do with your series, but I am curious if you
are planning on touching 'test_might_fail' at all. These can be useful
for non-Git commands, too, such as 'test_might_fail umask 022' on
systems that may or may not do something sensible with umask.

Sorry for injecting myself into this thread on an otherwise unrelated
topic.

Thanks,
Taylor

Re: [PATCH 0/4] t: replace incorrect test_must_fail usage (part 5)

[Johannes Sixt]

Am 29.04.20 um 21:50 schrieb Taylor Blau:
> This comment has nothing to do with your series, but I am curious if you
> are planning on touching 'test_might_fail' at all. These can be useful
> for non-Git commands, too, such as 'test_might_fail umask 022' on
> systems that may or may not do something sensible with umask.

When it's not a git command that might fail, the idiom is

	... &&
	{ umask 022 || :; } &&
	...

-- Hannes

Re* [PATCH 0/4] t: replace incorrect test_must_fail usage (part 5)

[Junio C Hamano]

Johannes Sixt <j6t@kdbg.org> writes:

> Am 29.04.20 um 21:50 schrieb Taylor Blau:
>> This comment has nothing to do with your series, but I am curious if you
>> are planning on touching 'test_might_fail' at all. These can be useful
>> for non-Git commands, too, such as 'test_might_fail umask 022' on
>> systems that may or may not do something sensible with umask.
>
> When it's not a git command that might fail, the idiom is
>
> 	... &&
> 	{ umask 022 || :; } &&
> 	...
>
> -- Hannes

I hoped to find this documented in t/README, but ended up writing
this.  Overkill?  I dunno.

-- >8 --
Subject: [PATCH] t/README: document when not to use test_might_fail

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/README | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/t/README b/t/README
index 13747f1344..870950c7d1 100644
--- a/t/README
+++ b/t/README
@@ -875,7 +875,9 @@ library for your script to use.
  - test_might_fail [<options>] <git-command>
 
    Similar to test_must_fail, but tolerate success, too.  Use this
-   instead of "<git-command> || :" to catch failures due to segv.
+   instead of "<git-command> || :" to catch failures due to segv,
+   but do use "{ <non-git-command> || :; }" to ignore a failure from
+   a command that is not git.
 
    Accepts the same options as test_must_fail.
 

Re: Re* [PATCH 0/4] t: replace incorrect test_must_fail usage (part 5)

[Taylor Blau]

On Wed, Apr 29, 2020 at 02:42:02PM -0700, Junio C Hamano wrote:
> Johannes Sixt <j6t@kdbg.org> writes:
>
> > Am 29.04.20 um 21:50 schrieb Taylor Blau:
> >> This comment has nothing to do with your series, but I am curious if you
> >> are planning on touching 'test_might_fail' at all. These can be useful
> >> for non-Git commands, too, such as 'test_might_fail umask 022' on
> >> systems that may or may not do something sensible with umask.
> >
> > When it's not a git command that might fail, the idiom is
> >
> > 	... &&
> > 	{ umask 022 || :; } &&
> > 	...
> >
> > -- Hannes
>
> I hoped to find this documented in t/README, but ended up writing
> this.  Overkill?  I dunno.
>
> -- >8 --
> Subject: [PATCH] t/README: document when not to use test_might_fail
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>  t/README | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/t/README b/t/README
> index 13747f1344..870950c7d1 100644
> --- a/t/README
> +++ b/t/README
> @@ -875,7 +875,9 @@ library for your script to use.
>   - test_might_fail [<options>] <git-command>
>
>     Similar to test_must_fail, but tolerate success, too.  Use this
> -   instead of "<git-command> || :" to catch failures due to segv.
> +   instead of "<git-command> || :" to catch failures due to segv,
> +   but do use "{ <non-git-command> || :; }" to ignore a failure from
> +   a command that is not git.

Hmm. I say this as somebody who just re-rolled a series to add two
'test_might_fail umask 022' lines, so am a little disappointed to learn
that this is not considered to be idiomatic.

To me this seems a little overkill, but it may not be on environments
where an extra subshell incurred by 'test_might_fail' might be overly
expensive.

Junio: do you want another reroll of that series? :/

>     Accepts the same options as test_must_fail.

Thanks,
Taylor

Re: Re* [PATCH 0/4] t: replace incorrect test_must_fail usage (part 5)

[Junio C Hamano]

Taylor Blau <me@ttaylorr.com> writes:

> To me this seems a little overkill, but it may not be on environments
> where an extra subshell incurred by 'test_might_fail' might be overly
> expensive.

It comes from the same principle as "we are not in the business of
catching segv from system tools---don't use test_must_fail on
non-git commands".  Adopting the convention happened quite some time
ago and that was why I checked if we failed to document it.

What I wondered was if it is overkill to document the convention; if
the convention was overkill is not a question at this point.

Re: Re* [PATCH 0/4] t: replace incorrect test_must_fail usage (part 5)

[Taylor Blau]

On Wed, Apr 29, 2020 at 03:10:50PM -0700, Junio C Hamano wrote:
> Taylor Blau <me@ttaylorr.com> writes:
>
> > To me this seems a little overkill, but it may not be on environments
> > where an extra subshell incurred by 'test_might_fail' might be overly
> > expensive.
>
> It comes from the same principle as "we are not in the business of
> catching segv from system tools---don't use test_must_fail on
> non-git commands".  Adopting the convention happened quite some time
> ago and that was why I checked if we failed to document it.
>
> What I wondered was if it is overkill to document the convention; if
> the convention was overkill is not a question at this point.

Ah, fair enough. Thanks for a patient explanation.

Thanks,
Taylor

Re: Re* [PATCH 0/4] t: replace incorrect test_must_fail usage (part 5)

[Junio C Hamano]

Taylor Blau <me@ttaylorr.com> writes:

> Hmm. I say this as somebody who just re-rolled a series to add two
> 'test_might_fail umask 022' lines, so am a little disappointed to learn
> that this is not considered to be idiomatic.
> ...
> Junio: do you want another reroll of that series? :/

The one I saw and remember was two new umask calls protected in POSIXPERM
prerequisite but without test-might-fail involved.

Perhaps there is nothing to reroll?  Or perhaps I am not checking my
mailbox often enough?

Re: Re* [PATCH 0/4] t: replace incorrect test_must_fail usage (part 5)

[Taylor Blau]

On Wed, Apr 29, 2020 at 03:36:25PM -0700, Junio C Hamano wrote:
> Taylor Blau <me@ttaylorr.com> writes:
>
> > Hmm. I say this as somebody who just re-rolled a series to add two
> > 'test_might_fail umask 022' lines, so am a little disappointed to learn
> > that this is not considered to be idiomatic.
> > ...
> > Junio: do you want another reroll of that series? :/
>
> The one I saw and remember was two new umask calls protected in POSIXPERM
> prerequisite but without test-might-fail involved.
>
> Perhaps there is nothing to reroll?  Or perhaps I am not checking my
> mailbox often enough?

You are checking your mailbox often enough, but unfortunately my memory
isn't as good as I thought ;). You're right: those calls are in
POSIXPERM-only tests, and don't have a 'test_might_fail' in front of
them as such.

That was easy ;).

Thanks,
Taylor

Re: Re* [PATCH 0/4] t: replace incorrect test_must_fail usage (part 5)

[Eric Sunshine]

On Wed, Apr 29, 2020 at 5:42 PM Junio C Hamano <gitster@pobox.com> wrote:
> Subject: [PATCH] t/README: document when not to use test_might_fail
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
> diff --git a/t/README b/t/README
> @@ -875,7 +875,9 @@ library for your script to use.
>   - test_might_fail [<options>] <git-command>
>
>     Similar to test_must_fail, but tolerate success, too.  Use this
> -   instead of "<git-command> || :" to catch failures due to segv.
> +   instead of "<git-command> || :" to catch failures due to segv,
> +   but do use "{ <non-git-command> || :; }" to ignore a failure from
> +   a command that is not git.

This seems like a good addition and perhaps may help reduce reviewer
burden (not that this comes up very often, but I've recommended it
here and there when reviewing patches). Here's a possible alternate
wording:

    Similar to test_must_fail, but tolerate success, too. Use this
    instead of "<git-command> || :" to catch failures due to segv.
    To ignore failure of non-git commands, however, use
    "{ <non-git-command> || :; }".

Re: Re* [PATCH 0/4] t: replace incorrect test_must_fail usage (part 5)

[Junio C Hamano]

Eric Sunshine <sunshine@sunshineco.com> writes:

> This seems like a good addition and perhaps may help reduce reviewer
> burden (not that this comes up very often, but I've recommended it
> here and there when reviewing patches). Here's a possible alternate
> wording:
>
>     Similar to test_must_fail, but tolerate success, too. Use this
>     instead of "<git-command> || :" to catch failures due to segv.
>     To ignore failure of non-git commands, however, use
>     "{ <non-git-command> || :; }".

That is certainly more succinct.  "git grep -e '|| :' t/" gives some
hits that are candidates to convert to test_might_fail, by the way.

[PATCH v2 0/4] t: replace incorrect test_must_fail usage (part 5)

[Denton Liu]

Hi all,

This is mostly a resend of what's currently queued in
"dl/test-must-fail-fixes-5" except with a tiny bit of cleanup on the tip
patch. I'd appreciate a review on this series so that we can finally get
rid of that "Needs review" on the What's Cooking messages ;)


The overall scope of these patches is to replace inappropriate uses of
test_must_fail. IOW, we should only allow test_must_fail to run on `git`
and `test-tool`. Ultimately, we will conclude by making test_must_fail
error out on non-git commands. An advance view of the final series can
be found here[1].

This is the fifth part. It focuses on lib-submodule-update.sh and tests
that make use of it.

The first part can be found here[2]. The second part can be found
here[3]. The third part can be found here[4]. The fourth part can be
found here[5].

Changes since v1.2:

* In "lib-submodule-update: pass OVERWRITING_FAIL", use if-then return
  to reduce the amount of code churn

[1]: (may be rebased at any time) https://github.com/Denton-L/git/tree/ready/cleanup-test-must-fail2
[2]: https://lore.kernel.org/git/cover.1576583819.git.liu.denton@gmail.com/
[3]: https://lore.kernel.org/git/cover.1577454401.git.liu.denton@gmail.com/
[4]: https://lore.kernel.org/git/cover.1585209554.git.liu.denton@gmail.com/
[5]: https://lore.kernel.org/git/cover.1587372771.git.liu.denton@gmail.com/

Denton Liu (4):
  lib-submodule-update: add space after function name
  lib-submodule-update: consolidate --recurse-submodules
  lib-submodule-update: prepend "git" to $command
  lib-submodule-update: pass OVERWRITING_FAIL

 t/lib-submodule-update.sh        | 55 ++++++++++++++++++++++----------
 t/t1013-read-tree-submodule.sh   |  4 +--
 t/t2013-checkout-submodule.sh    |  4 +--
 t/t3426-rebase-submodule.sh      |  8 ++---
 t/t3512-cherry-pick-submodule.sh |  2 +-
 t/t3513-revert-submodule.sh      |  7 +++-
 t/t3906-stash-submodule.sh       |  8 +++--
 t/t4137-apply-submodule.sh       | 10 +++---
 t/t4255-am-submodule.sh          | 10 +++---
 t/t5572-pull-submodule.sh        | 16 +++++-----
 t/t6041-bisect-submodule.sh      |  8 +++--
 t/t7112-reset-submodule.sh       |  6 ++--
 t/t7613-merge-submodule.sh       |  8 ++---
 13 files changed, 92 insertions(+), 54 deletions(-)

Range-diff against v1:
1:  f148526dae = 1:  ba2f642e0f lib-submodule-update: add space after function name
2:  b7d7d0d761 = 2:  16d0a3eb9a lib-submodule-update: consolidate --recurse-submodules
3:  4922e75f2a = 3:  578bab6f1a lib-submodule-update: prepend "git" to $command
4:  a0a8fbd881 ! 4:  48598e3f98 lib-submodule-update: pass OVERWRITING_FAIL
    @@ Commit message
         with a file buffer so that the return code of the first command is not
         lost.
     
    -    This patch can be better viewed with `--ignore-all-space`.
    -
      ## t/lib-submodule-update.sh ##
     @@ t/lib-submodule-update.sh: test_submodule_content () {
      # a removed submodule.
    @@ t/t3513-revert-submodule.sh: git_revert () {
      	git status -su >expect &&
      	ls -1pR * >>expect &&
      	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
    --	git checkout "$1" &&
    --	git revert HEAD &&
    --	rm -rf * &&
    --	tar xf "$TRASH_DIRECTORY/tmp.tar" &&
    --	git status -su >actual &&
    --	ls -1pR * >>actual &&
    --	test_cmp expect actual &&
    --	git revert HEAD
     +	$OVERWRITING_FAIL git checkout "$1" &&
    -+	if test -z "$OVERWRITING_FAIL"
    ++	if test -n "$OVERWRITING_FAIL"
     +	then
    -+		git checkout "$1" &&
    -+		git revert HEAD &&
    -+		rm -rf * &&
    -+		tar xf "$TRASH_DIRECTORY/tmp.tar" &&
    -+		git status -su >actual &&
    -+		ls -1pR * >>actual &&
    -+		test_cmp expect actual &&
    -+		git revert HEAD
    -+	fi
    - }
    - 
    - KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
    ++		return
    ++	fi &&
    + 	git checkout "$1" &&
    + 	git revert HEAD &&
    + 	rm -rf * &&
     
      ## t/t3906-stash-submodule.sh ##
     @@ t/t3906-stash-submodule.sh: test_description='stash can handle submodules'
    @@ t/t3906-stash-submodule.sh: test_description='stash can handle submodules'
      	git status -su >expect &&
      	ls -1pR * >>expect &&
     -	git read-tree -u -m "$1" &&
    --	git stash &&
    --	git status -su >actual &&
    --	ls -1pR * >>actual &&
    --	test_cmp expect actual &&
    --	git stash apply
     +	$OVERWRITING_FAIL git read-tree -u -m "$1" &&
    -+	if test -z "$OVERWRITING_FAIL"
    ++	if test -n "$OVERWRITING_FAIL"
     +	then
    -+		git stash &&
    -+		git status -su >actual &&
    -+		ls -1pR * >>actual &&
    -+		test_cmp expect actual &&
    -+		git stash apply
    -+	fi
    - }
    - 
    - KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES=1
    ++		return
    ++	fi &&
    + 	git stash &&
    + 	git status -su >actual &&
    + 	ls -1pR * >>actual &&
     
      ## t/t4137-apply-submodule.sh ##
     @@ t/t4137-apply-submodule.sh: test_description='git apply handling submodules'
    @@ t/t6041-bisect-submodule.sh: git_bisect () {
      	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
      	GOOD=$(git rev-parse --verify HEAD) &&
     -	git checkout "$1" &&
    --	echo "foo" >bar &&
    --	git add bar &&
    --	git commit -m "bisect bad" &&
    --	BAD=$(git rev-parse --verify HEAD) &&
    --	git reset --hard HEAD^^ &&
    --	git submodule update &&
    --	git bisect start &&
    --	git bisect good $GOOD &&
    --	rm -rf * &&
    --	tar xf "$TRASH_DIRECTORY/tmp.tar" &&
    --	git status -su >actual &&
    --	ls -1pR * >>actual &&
    --	test_cmp expect actual &&
    --	git bisect bad $BAD
     +	$OVERWRITING_FAIL git checkout "$1" &&
    -+	if test -z "$OVERWRITING_FAIL"
    ++	if test -n "$OVERWRITING_FAIL"
     +	then
    -+		echo "foo" >bar &&
    -+		git add bar &&
    -+		git commit -m "bisect bad" &&
    -+		BAD=$(git rev-parse --verify HEAD) &&
    -+		git reset --hard HEAD^^ &&
    -+		git submodule update &&
    -+		git bisect start &&
    -+		git bisect good $GOOD &&
    -+		rm -rf * &&
    -+		tar xf "$TRASH_DIRECTORY/tmp.tar" &&
    -+		git status -su >actual &&
    -+		ls -1pR * >>actual &&
    -+		test_cmp expect actual &&
    -+		git bisect bad $BAD
    -+	fi
    - }
    - 
    - test_submodule_switch_func "git_bisect"
    ++		return
    ++	fi &&
    + 	echo "foo" >bar &&
    + 	git add bar &&
    + 	git commit -m "bisect bad" &&
-- 
2.27.0.rc0.114.g141fe7d276

[PATCH v2 1/4] lib-submodule-update: add space after function name

[Denton Liu]

In the shell scripts in this codebase, the usual style is to include a
space between the function name and the (). Add these missing spaces to
conform to the usual style of the code.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 1dd17fc03e..122554acd6 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -183,7 +183,7 @@ test_git_directory_is_unchanged () {
 	)
 }
 
-test_git_directory_exists() {
+test_git_directory_exists () {
 	test -e ".git/modules/$1" &&
 	if test -f sub1/.git
 	then
@@ -309,7 +309,7 @@ test_submodule_content () {
 
 # Internal function; use test_submodule_switch() or
 # test_submodule_forced_switch() instead.
-test_submodule_switch_common() {
+test_submodule_switch_common () {
 	command="$1"
 	######################### Appearing submodule #########################
 	# Switching to a commit letting a submodule appear creates empty dir ...
@@ -629,7 +629,7 @@ test_submodule_forced_switch () {
 
 # Internal function; use test_submodule_switch_recursing_with_args() or
 # test_submodule_forced_switch_recursing_with_args() instead.
-test_submodule_recursing_with_args_common() {
+test_submodule_recursing_with_args_common () {
 	command="$1"
 
 	######################### Appearing submodule #########################
-- 
2.27.0.rc0.114.g141fe7d276

[PATCH v2 2/4] lib-submodule-update: consolidate --recurse-submodules

[Denton Liu]

Both test_submodule_switch_recursing_with_args() and
test_submodule_forced_switch_recursing_with_args() call the internal
function test_submodule_recursing_with_args_common() with the final
argument of `--recurse-submodules`. Consolidate this duplication by
appending the argument in test_submodule_recursing_with_args_common().

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 122554acd6..bb36287803 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -630,7 +630,7 @@ test_submodule_forced_switch () {
 # Internal function; use test_submodule_switch_recursing_with_args() or
 # test_submodule_forced_switch_recursing_with_args() instead.
 test_submodule_recursing_with_args_common () {
-	command="$1"
+	command="$1 --recurse-submodules"
 
 	######################### Appearing submodule #########################
 	# Switching to a commit letting a submodule appear checks it out ...
@@ -809,7 +809,7 @@ test_submodule_recursing_with_args_common () {
 # test_submodule_switch_recursing_with_args "$GIT_COMMAND"
 test_submodule_switch_recursing_with_args () {
 	cmd_args="$1"
-	command="git $cmd_args --recurse-submodules"
+	command="git $cmd_args"
 	test_submodule_recursing_with_args_common "$command"
 
 	RESULTDS=success
@@ -927,7 +927,7 @@ test_submodule_switch_recursing_with_args () {
 # away local changes in the superproject is allowed.
 test_submodule_forced_switch_recursing_with_args () {
 	cmd_args="$1"
-	command="git $cmd_args --recurse-submodules"
+	command="git $cmd_args"
 	test_submodule_recursing_with_args_common "$command"
 
 	RESULT=success
-- 
2.27.0.rc0.114.g141fe7d276

[PATCH v2 3/4] lib-submodule-update: prepend "git" to $command

[Denton Liu]

Since all invocations of test_submodule_forced_switch() are git
commands, automatically prepend "git" before invoking
test_submodule_switch_common().

Similarly, many invocations of test_submodule_switch() are also git
commands so automatically prepend "git" before invoking
test_submodule_switch_common() as well.

Finally, for invocations of test_submodule_switch() that invoke a custom
function, rename the old function to test_submodule_switch_func().

This is necessary because in a future commit, we will be adding some
logic that needs to distinguish between an invocation of a plain git
comamnd and an invocation of a test helper function.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh        | 14 +++++++++-----
 t/t1013-read-tree-submodule.sh   |  4 ++--
 t/t2013-checkout-submodule.sh    |  4 ++--
 t/t3426-rebase-submodule.sh      |  4 ++--
 t/t3512-cherry-pick-submodule.sh |  2 +-
 t/t3513-revert-submodule.sh      |  2 +-
 t/t3906-stash-submodule.sh       |  2 +-
 t/t4137-apply-submodule.sh       |  4 ++--
 t/t4255-am-submodule.sh          |  4 ++--
 t/t5572-pull-submodule.sh        |  8 ++++----
 t/t6041-bisect-submodule.sh      |  2 +-
 t/t7112-reset-submodule.sh       |  6 +++---
 t/t7613-merge-submodule.sh       |  8 ++++----
 13 files changed, 34 insertions(+), 30 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index bb36287803..fb6c0f3d4f 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -307,8 +307,8 @@ test_submodule_content () {
 # to protect the history!
 #
 
-# Internal function; use test_submodule_switch() or
-# test_submodule_forced_switch() instead.
+# Internal function; use test_submodule_switch_func(), test_submodule_switch_func(),
+# or test_submodule_forced_switch() instead.
 test_submodule_switch_common () {
 	command="$1"
 	######################### Appearing submodule #########################
@@ -566,8 +566,8 @@ test_submodule_switch_common () {
 #   # Do something here that updates the worktree and index to match target,
 #   # but not any submodule directories.
 # }
-# test_submodule_switch "my_func"
-test_submodule_switch () {
+# test_submodule_switch_func "my_func"
+test_submodule_switch_func () {
 	command="$1"
 	test_submodule_switch_common "$command"
 
@@ -587,12 +587,16 @@ test_submodule_switch () {
 	'
 }
 
+test_submodule_switch () {
+	test_submodule_switch_func "git $1"
+}
+
 # Same as test_submodule_switch(), except that throwing away local changes in
 # the superproject is allowed.
 test_submodule_forced_switch () {
 	command="$1"
 	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
-	test_submodule_switch_common "$command"
+	test_submodule_switch_common "git $command"
 
 	# When forced, a file in the superproject does not prevent creating a
 	# submodule of the same name.
diff --git a/t/t1013-read-tree-submodule.sh b/t/t1013-read-tree-submodule.sh
index 91a6fafcb4..b6df7444c0 100755
--- a/t/t1013-read-tree-submodule.sh
+++ b/t/t1013-read-tree-submodule.sh
@@ -12,8 +12,8 @@ test_submodule_switch_recursing_with_args "read-tree -u -m"
 
 test_submodule_forced_switch_recursing_with_args "read-tree -u --reset"
 
-test_submodule_switch "git read-tree -u -m"
+test_submodule_switch "read-tree -u -m"
 
-test_submodule_forced_switch "git read-tree -u --reset"
+test_submodule_forced_switch "read-tree -u --reset"
 
 test_done
diff --git a/t/t2013-checkout-submodule.sh b/t/t2013-checkout-submodule.sh
index 8f86b5f4b2..b2bdd1fcb4 100755
--- a/t/t2013-checkout-submodule.sh
+++ b/t/t2013-checkout-submodule.sh
@@ -68,8 +68,8 @@ test_submodule_switch_recursing_with_args "checkout"
 
 test_submodule_forced_switch_recursing_with_args "checkout -f"
 
-test_submodule_switch "git checkout"
+test_submodule_switch "checkout"
 
-test_submodule_forced_switch "git checkout -f"
+test_submodule_forced_switch "checkout -f"
 
 test_done
diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
index a2bba04ba9..788605ccc0 100755
--- a/t/t3426-rebase-submodule.sh
+++ b/t/t3426-rebase-submodule.sh
@@ -20,7 +20,7 @@ git_rebase () {
 	git rebase "$1"
 }
 
-test_submodule_switch "git_rebase"
+test_submodule_switch_func "git_rebase"
 
 git_rebase_interactive () {
 	git status -su >expect &&
@@ -38,7 +38,7 @@ git_rebase_interactive () {
 	git rebase -i "$1"
 }
 
-test_submodule_switch "git_rebase_interactive"
+test_submodule_switch_func "git_rebase_interactive"
 
 test_expect_success 'rebase interactive ignores modified submodules' '
 	test_when_finished "rm -rf super sub" &&
diff --git a/t/t3512-cherry-pick-submodule.sh b/t/t3512-cherry-pick-submodule.sh
index bd78287841..6ece1d8573 100755
--- a/t/t3512-cherry-pick-submodule.sh
+++ b/t/t3512-cherry-pick-submodule.sh
@@ -7,7 +7,7 @@ test_description='cherry-pick can handle submodules'
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "git cherry-pick"
+test_submodule_switch "cherry-pick"
 
 test_expect_success 'unrelated submodule/file conflict is ignored' '
 	test_create_repo sub &&
diff --git a/t/t3513-revert-submodule.sh b/t/t3513-revert-submodule.sh
index 5e39fcdb66..95a7f64471 100755
--- a/t/t3513-revert-submodule.sh
+++ b/t/t3513-revert-submodule.sh
@@ -26,6 +26,6 @@ git_revert () {
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
-test_submodule_switch "git_revert"
+test_submodule_switch_func "git_revert"
 
 test_done
diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
index b93d1d74da..6a7e801ca0 100755
--- a/t/t3906-stash-submodule.sh
+++ b/t/t3906-stash-submodule.sh
@@ -19,7 +19,7 @@ git_stash () {
 KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES=1
 KNOWN_FAILURE_CHERRY_PICK_SEES_EMPTY_COMMIT=1
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
-test_submodule_switch "git_stash"
+test_submodule_switch_func "git_stash"
 
 setup_basic () {
 	test_when_finished "rm -rf main sub" &&
diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
index a9bd40a6d0..b645e303a0 100755
--- a/t/t4137-apply-submodule.sh
+++ b/t/t4137-apply-submodule.sh
@@ -9,12 +9,12 @@ apply_index () {
 	git diff --ignore-submodules=dirty "..$1" | git apply --index -
 }
 
-test_submodule_switch "apply_index"
+test_submodule_switch_func "apply_index"
 
 apply_3way () {
 	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
 }
 
-test_submodule_switch "apply_3way"
+test_submodule_switch_func "apply_3way"
 
 test_done
diff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh
index 0ba8194403..1b179d5f45 100755
--- a/t/t4255-am-submodule.sh
+++ b/t/t4255-am-submodule.sh
@@ -9,14 +9,14 @@ am () {
 	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am -
 }
 
-test_submodule_switch "am"
+test_submodule_switch_func "am"
 
 am_3way () {
 	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am --3way -
 }
 
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "am_3way"
+test_submodule_switch_func "am_3way"
 
 test_expect_success 'setup diff.submodule' '
 	test_commit one &&
diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
index f916729a12..f911bf631e 100755
--- a/t/t5572-pull-submodule.sh
+++ b/t/t5572-pull-submodule.sh
@@ -17,21 +17,21 @@ git_pull () {
 }
 
 # pulls without conflicts
-test_submodule_switch "git_pull"
+test_submodule_switch_func "git_pull"
 
 git_pull_ff () {
 	reset_branch_to_HEAD "$1" &&
 	git pull --ff
 }
 
-test_submodule_switch "git_pull_ff"
+test_submodule_switch_func "git_pull_ff"
 
 git_pull_ff_only () {
 	reset_branch_to_HEAD "$1" &&
 	git pull --ff-only
 }
 
-test_submodule_switch "git_pull_ff_only"
+test_submodule_switch_func "git_pull_ff_only"
 
 git_pull_noff () {
 	reset_branch_to_HEAD "$1" &&
@@ -40,7 +40,7 @@ git_pull_noff () {
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "git_pull_noff"
+test_submodule_switch_func "git_pull_noff"
 
 test_expect_success 'pull --recurse-submodule setup' '
 	test_create_repo child &&
diff --git a/t/t6041-bisect-submodule.sh b/t/t6041-bisect-submodule.sh
index 62b8a2e7bb..0e0cdf638d 100755
--- a/t/t6041-bisect-submodule.sh
+++ b/t/t6041-bisect-submodule.sh
@@ -27,6 +27,6 @@ git_bisect () {
 	git bisect bad $BAD
 }
 
-test_submodule_switch "git_bisect"
+test_submodule_switch_func "git_bisect"
 
 test_done
diff --git a/t/t7112-reset-submodule.sh b/t/t7112-reset-submodule.sh
index a1cb9ff858..8741b665c9 100755
--- a/t/t7112-reset-submodule.sh
+++ b/t/t7112-reset-submodule.sh
@@ -13,10 +13,10 @@ test_submodule_switch_recursing_with_args "reset --keep"
 
 test_submodule_forced_switch_recursing_with_args "reset --hard"
 
-test_submodule_switch "git reset --keep"
+test_submodule_switch "reset --keep"
 
-test_submodule_switch "git reset --merge"
+test_submodule_switch "reset --merge"
 
-test_submodule_forced_switch "git reset --hard"
+test_submodule_forced_switch "reset --hard"
 
 test_done
diff --git a/t/t7613-merge-submodule.sh b/t/t7613-merge-submodule.sh
index d1e9fcc781..04bf4be7d7 100755
--- a/t/t7613-merge-submodule.sh
+++ b/t/t7613-merge-submodule.sh
@@ -6,14 +6,14 @@ test_description='merge can handle submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 # merges without conflicts
-test_submodule_switch "git merge"
+test_submodule_switch "merge"
 
-test_submodule_switch "git merge --ff"
+test_submodule_switch "merge --ff"
 
-test_submodule_switch "git merge --ff-only"
+test_submodule_switch "merge --ff-only"
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "git merge --no-ff"
+test_submodule_switch "merge --no-ff"
 
 test_done
-- 
2.27.0.rc0.114.g141fe7d276

Re: [PATCH v2 3/4] lib-submodule-update: prepend "git" to $command

[Philip Oakley]

Hi Denton,

On 21/05/2020 01:24, Denton Liu wrote:
> Since all invocations of test_submodule_forced_switch() are git
> commands, automatically prepend "git" before invoking
> test_submodule_switch_common().
>
> Similarly, many invocations of test_submodule_switch() are also git
> commands so automatically prepend "git" before invoking
> test_submodule_switch_common() as well.
>
> Finally, for invocations of test_submodule_switch() that invoke a custom
> function, rename the old function to test_submodule_switch_func().
>
> This is necessary because in a future commit, we will be adding some
> logic that needs to distinguish between an invocation of a plain git
> comamnd and an invocation of a test helper function.
>
> Signed-off-by: Denton Liu <liu.denton@gmail.com>
> ---
>  t/lib-submodule-update.sh        | 14 +++++++++-----
>  t/t1013-read-tree-submodule.sh   |  4 ++--
>  t/t2013-checkout-submodule.sh    |  4 ++--
>  t/t3426-rebase-submodule.sh      |  4 ++--
>  t/t3512-cherry-pick-submodule.sh |  2 +-
>  t/t3513-revert-submodule.sh      |  2 +-
>  t/t3906-stash-submodule.sh       |  2 +-
>  t/t4137-apply-submodule.sh       |  4 ++--
>  t/t4255-am-submodule.sh          |  4 ++--
>  t/t5572-pull-submodule.sh        |  8 ++++----
>  t/t6041-bisect-submodule.sh      |  2 +-
>  t/t7112-reset-submodule.sh       |  6 +++---
>  t/t7613-merge-submodule.sh       |  8 ++++----
>  13 files changed, 34 insertions(+), 30 deletions(-)
>
> diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
> index bb36287803..fb6c0f3d4f 100755
> --- a/t/lib-submodule-update.sh
> +++ b/t/lib-submodule-update.sh
> @@ -307,8 +307,8 @@ test_submodule_content () {
>  # to protect the history!
>  #
>  
> -# Internal function; use test_submodule_switch() or
> -# test_submodule_forced_switch() instead.
> +# Internal function; use test_submodule_switch_func(), test_submodule_switch_func(),
Is this an accidental duplication of  test_submodule_switch_func(),?
(spotted in passing)
--
Philip
> +# or test_submodule_forced_switch() instead.
>  test_submodule_switch_common () {
>  	command="$1"
>  	######################### Appearing submodule #########################
> @@ -566,8 +566,8 @@ test_submodule_switch_common () {
>  #   # Do something here that updates the worktree and index to match target,
>  #   # but not any submodule directories.
>  # }
> -# test_submodule_switch "my_func"
> -test_submodule_switch () {
> +# test_submodule_switch_func "my_func"
> +test_submodule_switch_func () {
>  	command="$1"
>  	test_submodule_switch_common "$command"
>  
> @@ -587,12 +587,16 @@ test_submodule_switch () {
>  	'
>  }
>  
> +test_submodule_switch () {
> +	test_submodule_switch_func "git $1"
> +}
> +
>  # Same as test_submodule_switch(), except that throwing away local changes in
>  # the superproject is allowed.
>  test_submodule_forced_switch () {
>  	command="$1"
>  	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
> -	test_submodule_switch_common "$command"
> +	test_submodule_switch_common "git $command"
>  
>  	# When forced, a file in the superproject does not prevent creating a
>  	# submodule of the same name.
> diff --git a/t/t1013-read-tree-submodule.sh b/t/t1013-read-tree-submodule.sh
> index 91a6fafcb4..b6df7444c0 100755
> --- a/t/t1013-read-tree-submodule.sh
> +++ b/t/t1013-read-tree-submodule.sh
> @@ -12,8 +12,8 @@ test_submodule_switch_recursing_with_args "read-tree -u -m"
>  
>  test_submodule_forced_switch_recursing_with_args "read-tree -u --reset"
>  
> -test_submodule_switch "git read-tree -u -m"
> +test_submodule_switch "read-tree -u -m"
>  
> -test_submodule_forced_switch "git read-tree -u --reset"
> +test_submodule_forced_switch "read-tree -u --reset"
>  
>  test_done
> diff --git a/t/t2013-checkout-submodule.sh b/t/t2013-checkout-submodule.sh
> index 8f86b5f4b2..b2bdd1fcb4 100755
> --- a/t/t2013-checkout-submodule.sh
> +++ b/t/t2013-checkout-submodule.sh
> @@ -68,8 +68,8 @@ test_submodule_switch_recursing_with_args "checkout"
>  
>  test_submodule_forced_switch_recursing_with_args "checkout -f"
>  
> -test_submodule_switch "git checkout"
> +test_submodule_switch "checkout"
>  
> -test_submodule_forced_switch "git checkout -f"
> +test_submodule_forced_switch "checkout -f"
>  
>  test_done
> diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
> index a2bba04ba9..788605ccc0 100755
> --- a/t/t3426-rebase-submodule.sh
> +++ b/t/t3426-rebase-submodule.sh
> @@ -20,7 +20,7 @@ git_rebase () {
>  	git rebase "$1"
>  }
>  
> -test_submodule_switch "git_rebase"
> +test_submodule_switch_func "git_rebase"
>  
>  git_rebase_interactive () {
>  	git status -su >expect &&
> @@ -38,7 +38,7 @@ git_rebase_interactive () {
>  	git rebase -i "$1"
>  }
>  
> -test_submodule_switch "git_rebase_interactive"
> +test_submodule_switch_func "git_rebase_interactive"
>  
>  test_expect_success 'rebase interactive ignores modified submodules' '
>  	test_when_finished "rm -rf super sub" &&
> diff --git a/t/t3512-cherry-pick-submodule.sh b/t/t3512-cherry-pick-submodule.sh
> index bd78287841..6ece1d8573 100755
> --- a/t/t3512-cherry-pick-submodule.sh
> +++ b/t/t3512-cherry-pick-submodule.sh
> @@ -7,7 +7,7 @@ test_description='cherry-pick can handle submodules'
>  
>  KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
>  KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
> -test_submodule_switch "git cherry-pick"
> +test_submodule_switch "cherry-pick"
>  
>  test_expect_success 'unrelated submodule/file conflict is ignored' '
>  	test_create_repo sub &&
> diff --git a/t/t3513-revert-submodule.sh b/t/t3513-revert-submodule.sh
> index 5e39fcdb66..95a7f64471 100755
> --- a/t/t3513-revert-submodule.sh
> +++ b/t/t3513-revert-submodule.sh
> @@ -26,6 +26,6 @@ git_revert () {
>  }
>  
>  KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
> -test_submodule_switch "git_revert"
> +test_submodule_switch_func "git_revert"
>  
>  test_done
> diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
> index b93d1d74da..6a7e801ca0 100755
> --- a/t/t3906-stash-submodule.sh
> +++ b/t/t3906-stash-submodule.sh
> @@ -19,7 +19,7 @@ git_stash () {
>  KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES=1
>  KNOWN_FAILURE_CHERRY_PICK_SEES_EMPTY_COMMIT=1
>  KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
> -test_submodule_switch "git_stash"
> +test_submodule_switch_func "git_stash"
>  
>  setup_basic () {
>  	test_when_finished "rm -rf main sub" &&
> diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
> index a9bd40a6d0..b645e303a0 100755
> --- a/t/t4137-apply-submodule.sh
> +++ b/t/t4137-apply-submodule.sh
> @@ -9,12 +9,12 @@ apply_index () {
>  	git diff --ignore-submodules=dirty "..$1" | git apply --index -
>  }
>  
> -test_submodule_switch "apply_index"
> +test_submodule_switch_func "apply_index"
>  
>  apply_3way () {
>  	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
>  }
>  
> -test_submodule_switch "apply_3way"
> +test_submodule_switch_func "apply_3way"
>  
>  test_done
> diff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh
> index 0ba8194403..1b179d5f45 100755
> --- a/t/t4255-am-submodule.sh
> +++ b/t/t4255-am-submodule.sh
> @@ -9,14 +9,14 @@ am () {
>  	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am -
>  }
>  
> -test_submodule_switch "am"
> +test_submodule_switch_func "am"
>  
>  am_3way () {
>  	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am --3way -
>  }
>  
>  KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
> -test_submodule_switch "am_3way"
> +test_submodule_switch_func "am_3way"
>  
>  test_expect_success 'setup diff.submodule' '
>  	test_commit one &&
> diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
> index f916729a12..f911bf631e 100755
> --- a/t/t5572-pull-submodule.sh
> +++ b/t/t5572-pull-submodule.sh
> @@ -17,21 +17,21 @@ git_pull () {
>  }
>  
>  # pulls without conflicts
> -test_submodule_switch "git_pull"
> +test_submodule_switch_func "git_pull"
>  
>  git_pull_ff () {
>  	reset_branch_to_HEAD "$1" &&
>  	git pull --ff
>  }
>  
> -test_submodule_switch "git_pull_ff"
> +test_submodule_switch_func "git_pull_ff"
>  
>  git_pull_ff_only () {
>  	reset_branch_to_HEAD "$1" &&
>  	git pull --ff-only
>  }
>  
> -test_submodule_switch "git_pull_ff_only"
> +test_submodule_switch_func "git_pull_ff_only"
>  
>  git_pull_noff () {
>  	reset_branch_to_HEAD "$1" &&
> @@ -40,7 +40,7 @@ git_pull_noff () {
>  
>  KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
>  KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
> -test_submodule_switch "git_pull_noff"
> +test_submodule_switch_func "git_pull_noff"
>  
>  test_expect_success 'pull --recurse-submodule setup' '
>  	test_create_repo child &&
> diff --git a/t/t6041-bisect-submodule.sh b/t/t6041-bisect-submodule.sh
> index 62b8a2e7bb..0e0cdf638d 100755
> --- a/t/t6041-bisect-submodule.sh
> +++ b/t/t6041-bisect-submodule.sh
> @@ -27,6 +27,6 @@ git_bisect () {
>  	git bisect bad $BAD
>  }
>  
> -test_submodule_switch "git_bisect"
> +test_submodule_switch_func "git_bisect"
>  
>  test_done
> diff --git a/t/t7112-reset-submodule.sh b/t/t7112-reset-submodule.sh
> index a1cb9ff858..8741b665c9 100755
> --- a/t/t7112-reset-submodule.sh
> +++ b/t/t7112-reset-submodule.sh
> @@ -13,10 +13,10 @@ test_submodule_switch_recursing_with_args "reset --keep"
>  
>  test_submodule_forced_switch_recursing_with_args "reset --hard"
>  
> -test_submodule_switch "git reset --keep"
> +test_submodule_switch "reset --keep"
>  
> -test_submodule_switch "git reset --merge"
> +test_submodule_switch "reset --merge"
>  
> -test_submodule_forced_switch "git reset --hard"
> +test_submodule_forced_switch "reset --hard"
>  
>  test_done
> diff --git a/t/t7613-merge-submodule.sh b/t/t7613-merge-submodule.sh
> index d1e9fcc781..04bf4be7d7 100755
> --- a/t/t7613-merge-submodule.sh
> +++ b/t/t7613-merge-submodule.sh
> @@ -6,14 +6,14 @@ test_description='merge can handle submodules'
>  . "$TEST_DIRECTORY"/lib-submodule-update.sh
>  
>  # merges without conflicts
> -test_submodule_switch "git merge"
> +test_submodule_switch "merge"
>  
> -test_submodule_switch "git merge --ff"
> +test_submodule_switch "merge --ff"
>  
> -test_submodule_switch "git merge --ff-only"
> +test_submodule_switch "merge --ff-only"
>  
>  KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
>  KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
> -test_submodule_switch "git merge --no-ff"
> +test_submodule_switch "merge --no-ff"
>  
>  test_done

Re: [PATCH v2 3/4] lib-submodule-update: prepend "git" to $command

[Denton Liu]

Hi Philip,

On Thu, May 21, 2020 at 11:39:48AM +0100, Philip Oakley wrote:
> > diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
> > index bb36287803..fb6c0f3d4f 100755
> > --- a/t/lib-submodule-update.sh
> > +++ b/t/lib-submodule-update.sh
> > @@ -307,8 +307,8 @@ test_submodule_content () {
> >  # to protect the history!
> >  #
> >  
> > -# Internal function; use test_submodule_switch() or
> > -# test_submodule_forced_switch() instead.
> > +# Internal function; use test_submodule_switch_func(), test_submodule_switch_func(),
> Is this an accidental duplication of  test_submodule_switch_func(),?
> (spotted in passing)

Thanks, good catch. The second one should be test_submodule_switch().

> --
> Philip

[PATCH v2 4/4] lib-submodule-update: pass OVERWRITING_FAIL

[Denton Liu]

We are using `test_must_fail $command`. However, $command is not
necessarily a git command; it could be a test helper function.

In an effort to stop using test_must_fail with non-git commands, instead
of invoking `test_must_fail $command`, run
`OVERWRITING_FAIL=test_must_fail $command` instead in
test_submodule_switch_common().

In the case where $command is a git command,
test_submodule_switch_common() is called by one of
test_submodule_switch() or test_submodule_forced_switch(). In those two
functions, pass $command like this:

	test_submodule_switch_common "eval \$OVERWRITING_FAIL git $command"

When $command is a helper function, the parent function calling
test_submodule_switch_common() is test_submodule_switch_func(). For all
test_submodule_switch_func() invocations, increase the granularity of
the argument test helper function by prefixing the git invocation which is
meant to fail with $OVERWRITING_FAIL like this:

	$OVERWRITING_FAIL git checkout "$1"

This is useful because currently, when we run a test helper function, we
just mark the whole thing as `test_must_fail`. However, it's possible
that the helper function might fail earlier or later than expected due
to an introduced bug. If this happens, then the test case will still
report as passing but it should really be marked as failing since it
didn't actually display the intended behaviour.

While we're at it, some helper functions have git commands piping into
another git command. Break these pipes up into two separate invocations
with a file buffer so that the return code of the first command is not
lost.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh   | 33 +++++++++++++++++++++++++--------
 t/t3426-rebase-submodule.sh |  4 ++--
 t/t3513-revert-submodule.sh |  5 +++++
 t/t3906-stash-submodule.sh  |  6 +++++-
 t/t4137-apply-submodule.sh  |  6 ++++--
 t/t4255-am-submodule.sh     |  6 ++++--
 t/t5572-pull-submodule.sh   |  8 ++++----
 t/t6041-bisect-submodule.sh |  6 +++++-
 8 files changed, 54 insertions(+), 20 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index fb6c0f3d4f..a0274c03de 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -304,12 +304,15 @@ test_submodule_content () {
 # a removed submodule.
 #
 # Removing a submodule containing a .git directory must fail even when forced
-# to protect the history!
+# to protect the history! If we are testing this case,
+# OVERWRITING_FAIL=test_must_fail, otherwise OVERWRITING_FAIL will be the empty
+# string.
 #
 
 # Internal function; use test_submodule_switch_func(), test_submodule_switch_func(),
 # or test_submodule_forced_switch() instead.
 test_submodule_switch_common () {
+	OVERWRITING_FAIL=
 	command="$1"
 	######################### Appearing submodule #########################
 	# Switching to a commit letting a submodule appear creates empty dir ...
@@ -443,7 +446,9 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
-			test_must_fail $command replace_sub1_with_directory &&
+			OVERWRITING_FAIL=test_must_fail &&
+			$command replace_sub1_with_directory &&
+			OVERWRITING_FAIL= &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -456,7 +461,9 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_directory &&
+			OVERWRITING_FAIL=test_must_fail &&
+			$command replace_sub1_with_directory &&
+			OVERWRITING_FAIL= &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -470,7 +477,9 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
-			test_must_fail $command replace_sub1_with_file &&
+			OVERWRITING_FAIL=test_must_fail &&
+			$command replace_sub1_with_file &&
+			OVERWRITING_FAIL= &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -484,7 +493,9 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_file &&
+			OVERWRITING_FAIL=test_must_fail &&
+			$command replace_sub1_with_file &&
+			OVERWRITING_FAIL= &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -559,6 +570,11 @@ test_submodule_switch_common () {
 # conditions, set the appropriate KNOWN_FAILURE_* variable used in the tests
 # below to 1.
 #
+# Removing a submodule containing a .git directory must fail even when forced
+# to protect the history! If we are testing this case,
+# OVERWRITING_FAIL=test_must_fail, otherwise OVERWRITING_FAIL will be the empty
+# string.
+#
 # Use as follows:
 #
 # my_func () {
@@ -568,6 +584,7 @@ test_submodule_switch_common () {
 # }
 # test_submodule_switch_func "my_func"
 test_submodule_switch_func () {
+	OVERWRITING_FAIL=
 	command="$1"
 	test_submodule_switch_common "$command"
 
@@ -580,7 +597,7 @@ test_submodule_switch_func () {
 			cd submodule_update &&
 			git branch -t add_sub1 origin/add_sub1 &&
 			>sub1 &&
-			test_must_fail $command add_sub1 &&
+			OVERWRITING_FAIL=test_must_fail $command add_sub1 &&
 			test_superproject_content origin/no_submodule &&
 			test_must_be_empty sub1
 		)
@@ -588,7 +605,7 @@ test_submodule_switch_func () {
 }
 
 test_submodule_switch () {
-	test_submodule_switch_func "git $1"
+	test_submodule_switch_func "eval \$OVERWRITING_FAIL git $1"
 }
 
 # Same as test_submodule_switch(), except that throwing away local changes in
@@ -596,7 +613,7 @@ test_submodule_switch () {
 test_submodule_forced_switch () {
 	command="$1"
 	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
-	test_submodule_switch_common "git $command"
+	test_submodule_switch_common "eval \$OVERWRITING_FAIL git $command"
 
 	# When forced, a file in the superproject does not prevent creating a
 	# submodule of the same name.
diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
index 788605ccc0..c6a7f584ed 100755
--- a/t/t3426-rebase-submodule.sh
+++ b/t/t3426-rebase-submodule.sh
@@ -17,7 +17,7 @@ git_rebase () {
 	git status -su >actual &&
 	ls -1pR * >>actual &&
 	test_cmp expect actual &&
-	git rebase "$1"
+	$OVERWRITING_FAIL git rebase "$1"
 }
 
 test_submodule_switch_func "git_rebase"
@@ -35,7 +35,7 @@ git_rebase_interactive () {
 	test_cmp expect actual &&
 	set_fake_editor &&
 	echo "fake-editor.sh" >.git/info/exclude &&
-	git rebase -i "$1"
+	$OVERWRITING_FAIL git rebase -i "$1"
 }
 
 test_submodule_switch_func "git_rebase_interactive"
diff --git a/t/t3513-revert-submodule.sh b/t/t3513-revert-submodule.sh
index 95a7f64471..e41f0f2009 100755
--- a/t/t3513-revert-submodule.sh
+++ b/t/t3513-revert-submodule.sh
@@ -15,6 +15,11 @@ git_revert () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
+	$OVERWRITING_FAIL git checkout "$1" &&
+	if test -n "$OVERWRITING_FAIL"
+	then
+		return
+	fi &&
 	git checkout "$1" &&
 	git revert HEAD &&
 	rm -rf * &&
diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
index 6a7e801ca0..6e8dac9669 100755
--- a/t/t3906-stash-submodule.sh
+++ b/t/t3906-stash-submodule.sh
@@ -8,7 +8,11 @@ test_description='stash can handle submodules'
 git_stash () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
-	git read-tree -u -m "$1" &&
+	$OVERWRITING_FAIL git read-tree -u -m "$1" &&
+	if test -n "$OVERWRITING_FAIL"
+	then
+		return
+	fi &&
 	git stash &&
 	git status -su >actual &&
 	ls -1pR * >>actual &&
diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
index b645e303a0..dd08d9e1a4 100755
--- a/t/t4137-apply-submodule.sh
+++ b/t/t4137-apply-submodule.sh
@@ -6,13 +6,15 @@ test_description='git apply handling submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 apply_index () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --index -
+	git diff --ignore-submodules=dirty "..$1" >diff &&
+	$OVERWRITING_FAIL git apply --index - <diff
 }
 
 test_submodule_switch_func "apply_index"
 
 apply_3way () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
+	git diff --ignore-submodules=dirty "..$1" >diff &&
+	$OVERWRITING_FAIL git apply --3way - <diff
 }
 
 test_submodule_switch_func "apply_3way"
diff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh
index 1b179d5f45..b0fe78a956 100755
--- a/t/t4255-am-submodule.sh
+++ b/t/t4255-am-submodule.sh
@@ -6,13 +6,15 @@ test_description='git am handling submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 am () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am -
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
+	$OVERWRITING_FAIL git am - <patch
 }
 
 test_submodule_switch_func "am"
 
 am_3way () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am --3way -
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
+	$OVERWRITING_FAIL git am --3way - <patch
 }
 
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
index f911bf631e..4dd9913b6b 100755
--- a/t/t5572-pull-submodule.sh
+++ b/t/t5572-pull-submodule.sh
@@ -13,7 +13,7 @@ reset_branch_to_HEAD () {
 
 git_pull () {
 	reset_branch_to_HEAD "$1" &&
-	git pull
+	$OVERWRITING_FAIL git pull
 }
 
 # pulls without conflicts
@@ -21,21 +21,21 @@ test_submodule_switch_func "git_pull"
 
 git_pull_ff () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --ff
+	$OVERWRITING_FAIL git pull --ff
 }
 
 test_submodule_switch_func "git_pull_ff"
 
 git_pull_ff_only () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --ff-only
+	$OVERWRITING_FAIL git pull --ff-only
 }
 
 test_submodule_switch_func "git_pull_ff_only"
 
 git_pull_noff () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --no-ff
+	$OVERWRITING_FAIL git pull --no-ff
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
diff --git a/t/t6041-bisect-submodule.sh b/t/t6041-bisect-submodule.sh
index 0e0cdf638d..759890b721 100755
--- a/t/t6041-bisect-submodule.sh
+++ b/t/t6041-bisect-submodule.sh
@@ -10,7 +10,11 @@ git_bisect () {
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
 	GOOD=$(git rev-parse --verify HEAD) &&
-	git checkout "$1" &&
+	$OVERWRITING_FAIL git checkout "$1" &&
+	if test -n "$OVERWRITING_FAIL"
+	then
+		return
+	fi &&
 	echo "foo" >bar &&
 	git add bar &&
 	git commit -m "bisect bad" &&
-- 
2.27.0.rc0.114.g141fe7d276

Re: [PATCH v2 4/4] lib-submodule-update: pass OVERWRITING_FAIL

[Jeff King]

On Wed, May 20, 2020 at 08:24:18PM -0400, Denton Liu wrote:

> We are using `test_must_fail $command`. However, $command is not
> necessarily a git command; it could be a test helper function.
>
> In an effort to stop using test_must_fail with non-git commands, instead
> of invoking `test_must_fail $command`, run
> `OVERWRITING_FAIL=test_must_fail $command` instead in
> test_submodule_switch_common().

I think there are really two separate issues being addressed by this
patch.

One is using "test_must_fail" with a helper function. IMHO this is not
something we should go too far in trying to deal with. It's mostly a
style issue about how test_must_fail is meant to be used, and there's no
real downside if we occasionally use it for a non-git command.

I don't mind cleaning up spots where it's obviously misused, but in this
case we're introducing a lot of new complexity to handle the "sometimes"
nature of this call. And I'm not sure it's worth it on its own.

> This is useful because currently, when we run a test helper function, we
> just mark the whole thing as `test_must_fail`. However, it's possible
> that the helper function might fail earlier or later than expected due
> to an introduced bug. If this happens, then the test case will still
> report as passing but it should really be marked as failing since it
> didn't actually display the intended behaviour.

Now this second concern I think is much more interesting, because it
impacts the test results. And it's really not even about test_must_fail
in particular, but is a general problem with checking failure in any
compound operation. We may expect the early parts to succeed, but the
later parts to fail, and we want to tell the difference. And that's true
whether you're using "!", test_must_fail, etc.

You solve it here by passing OVERWRITING_FAIL down into the callback
functions. And that does work. But I think it may be easier to
understand if we invert the responsibility. Let the outer caller specify
two callbacks: one for setup/prep that must succeed, and one for a
single operation where we might expect success or failure.

The changes in lib-submodule-update look something like:

  @@ -326,7 +327,8 @@ test_submodule_switch_common() {
                  (
                          cd submodule_update &&
                          git branch -t add_sub1 origin/add_sub1 &&
  -                       $command add_sub1 &&
  +                       $prep add_sub1 &&
  +                       $command &&
                          test_superproject_content origin/add_sub1 &&
                          test_dir_is_empty sub1 &&
                          git submodule update --init --recursive &&

And in the caller we can simplify greatly:

  diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
  index f916729a12..e3ae7c89f1 100755
  --- a/t/t5572-pull-submodule.sh
  +++ b/t/t5572-pull-submodule.sh
  @@ -11,36 +11,13 @@ reset_branch_to_HEAD () {
   	git branch --set-upstream-to="origin/$1" "$1"
   }
   
  -git_pull () {
  -	reset_branch_to_HEAD "$1" &&
  -	git pull
  -}
  -
   # pulls without conflicts
  -test_submodule_switch "git_pull"
  -
  -git_pull_ff () {
  -	reset_branch_to_HEAD "$1" &&
  -	git pull --ff
  -}
  -
  -test_submodule_switch "git_pull_ff"
  -
  -git_pull_ff_only () {
  -	reset_branch_to_HEAD "$1" &&
  -	git pull --ff-only
  -}
  -
  -test_submodule_switch "git_pull_ff_only"
  -
  -git_pull_noff () {
  -	reset_branch_to_HEAD "$1" &&
  -	git pull --no-ff
  -}
  -
  +test_submodule_switch "reset_branch_to_HEAD" "git pull"
  +test_submodule_switch "reset_branch_to_HEAD" "git pull -ff"
  +test_submodule_switch "reset_branch_to_HEAD" "git pull --ff-only"
   KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
   KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
  -test_submodule_switch "git_pull_noff"
  +test_submodule_switch "reset_branch_to_HEAD" "git pull --no-ff"
   
   test_expect_success 'pull --recurse-submodule setup' '
   	test_create_repo child &&

I suspect this approach involves touching more lines than yours (it has
to add $prep everywhere $command is used). But IMHO the result is much
simpler to understand, because there's no spooky-action-at-a-distance
from magic environment variables.

A more complete patch is below, which is enough to get t5572 passing. I
think it would need the other test_submodule_switch() function updated,
and other scripts would need to adapt to the 2-arg style.

-Peff

Re: [PATCH v2 4/4] lib-submodule-update: pass OVERWRITING_FAIL

[Denton Liu]

Hi Peff,

On Thu, May 21, 2020 at 02:29:28PM -0400, Jeff King wrote:
> > This is useful because currently, when we run a test helper function, we
> > just mark the whole thing as `test_must_fail`. However, it's possible
> > that the helper function might fail earlier or later than expected due
> > to an introduced bug. If this happens, then the test case will still
> > report as passing but it should really be marked as failing since it
> > didn't actually display the intended behaviour.
> 
> Now this second concern I think is much more interesting, because it
> impacts the test results. And it's really not even about test_must_fail
> in particular, but is a general problem with checking failure in any
> compound operation. We may expect the early parts to succeed, but the
> later parts to fail, and we want to tell the difference. And that's true
> whether you're using "!", test_must_fail, etc.
> 
> You solve it here by passing OVERWRITING_FAIL down into the callback
> functions. And that does work. But I think it may be easier to
> understand if we invert the responsibility. Let the outer caller specify
> two callbacks: one for setup/prep that must succeed, and one for a
> single operation where we might expect success or failure.

I believe that we'll need a third optional argument. For example, in
t3906, we have the following diff

	diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
	index 6a7e801ca0..6e8dac9669 100755
	--- a/t/t3906-stash-submodule.sh
	+++ b/t/t3906-stash-submodule.sh
	@@ -8,7 +8,11 @@ test_description='stash can handle submodules'
	 git_stash () {
		git status -su >expect &&
		ls -1pR * >>expect &&
	-	git read-tree -u -m "$1" &&
	+	$OVERWRITING_FAIL git read-tree -u -m "$1" &&
	+	if test -n "$OVERWRITING_FAIL"
	+	then
	+		return
	+	fi &&
		git stash &&
		git status -su >actual &&
		ls -1pR * >>actual &&

which means, when we're doing test_must_fail, we have to skip the
remainder of the function otherwise, if the first command succeeds but
it fails later in the test, then we report success even though it didn't
fail at the point where we were expecting. I think that we'll have to
have an optional third arg for $cleanup or something. The only thing
that I'm worried about is that having three separate functions being
passed in may be too messy.

Aside from that, I like this approach much more than mine.

Thanks,

Denton

Re: [PATCH v2 4/4] lib-submodule-update: pass OVERWRITING_FAIL

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> You solve it here by passing OVERWRITING_FAIL down into the callback
> functions. And that does work. But I think it may be easier to
> understand if we invert the responsibility. Let the outer caller specify
> two callbacks: one for setup/prep that must succeed, and one for a
> single operation where we might expect success or failure.
>
> The changes in lib-submodule-update look something like:
> ...
> And in the caller we can simplify greatly:
> ...
>   +test_submodule_switch "reset_branch_to_HEAD" "git pull"
>   +test_submodule_switch "reset_branch_to_HEAD" "git pull -ff"
>   +test_submodule_switch "reset_branch_to_HEAD" "git pull --ff-only"
>    KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
>    KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
>   -test_submodule_switch "git_pull_noff"
>   +test_submodule_switch "reset_branch_to_HEAD" "git pull --no-ff"
>    
>    test_expect_success 'pull --recurse-submodule setup' '
>    	test_create_repo child &&
>
> I suspect this approach involves touching more lines than yours (it has
> to add $prep everywhere $command is used). But IMHO the result is much
> simpler to understand, because there's no spooky-action-at-a-distance
> from magic environment variables.

Yes, spooky-action-at-a-distance was what made me go Eeeeek when I
saw the original approach.

> A more complete patch is below, which is enough to get t5572 passing. I
> think it would need the other test_submodule_switch() function updated,
> and other scripts would need to adapt to the 2-arg style.

below where?

Re: [PATCH v2 4/4] lib-submodule-update: pass OVERWRITING_FAIL

[Jeff King]

On Thu, May 21, 2020 at 02:11:53PM -0700, Junio C Hamano wrote:

> > A more complete patch is below, which is enough to get t5572 passing. I
> > think it would need the other test_submodule_switch() function updated,
> > and other scripts would need to adapt to the 2-arg style.
> 
> below where?

Oops.

---
 t/lib-submodule-update.sh | 46 ++++++++++++++++++++++++++++++----------------
 t/t5572-pull-submodule.sh | 31 ++++---------------------------
 2 files changed, 34 insertions(+), 43 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 64fc6487dd..34da02b7fa 100644
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -310,7 +310,8 @@ test_submodule_content () {
 # Internal function; use test_submodule_switch() or
 # test_submodule_forced_switch() instead.
 test_submodule_switch_common() {
-	command="$1"
+	prep="$1"
+	command="$2"
 	######################### Appearing submodule #########################
 	# Switching to a commit letting a submodule appear creates empty dir ...
 	if test "$KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES" = 1
@@ -326,7 +327,8 @@ test_submodule_switch_common() {
 		(
 			cd submodule_update &&
 			git branch -t add_sub1 origin/add_sub1 &&
-			$command add_sub1 &&
+			$prep add_sub1 &&
+			$command &&
 			test_superproject_content origin/add_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -341,7 +343,8 @@ test_submodule_switch_common() {
 			cd submodule_update &&
 			mkdir sub1 &&
 			git branch -t add_sub1 origin/add_sub1 &&
-			$command add_sub1 &&
+			$prep add_sub1 &&
+			$command &&
 			test_superproject_content origin/add_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -356,7 +359,8 @@ test_submodule_switch_common() {
 		(
 			cd submodule_update &&
 			git branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&
-			$command replace_file_with_sub1 &&
+			$prep replace_file_with_sub1 &&
+			$command &&
 			test_superproject_content origin/replace_file_with_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -380,7 +384,8 @@ test_submodule_switch_common() {
 		(
 			cd submodule_update &&
 			git branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&
-			$command replace_directory_with_sub1 &&
+			$prep replace_directory_with_sub1 &&
+			$command &&
 			test_superproject_content origin/replace_directory_with_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -402,7 +407,8 @@ test_submodule_switch_common() {
 		(
 			cd submodule_update &&
 			git branch -t remove_sub1 origin/remove_sub1 &&
-			$command remove_sub1 &&
+			$prep remove_sub1 &&
+			$command &&
 			test_superproject_content origin/remove_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -415,7 +421,8 @@ test_submodule_switch_common() {
 			cd submodule_update &&
 			git branch -t remove_sub1 origin/remove_sub1 &&
 			replace_gitfile_with_git_dir sub1 &&
-			$command remove_sub1 &&
+			$prep remove_sub1 &&
+			$command &&
 			test_superproject_content origin/remove_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -443,7 +450,8 @@ test_submodule_switch_common() {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
-			test_must_fail $command replace_sub1_with_directory &&
+			$prep replace_sub1_with_directory &&
+			test_must_fail $command &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -456,7 +464,8 @@ test_submodule_switch_common() {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_directory &&
+			$prep replace_sub1_with_directory &&
+			test_must_fail $command &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -470,7 +479,8 @@ test_submodule_switch_common() {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
-			test_must_fail $command replace_sub1_with_file &&
+			$prep replace_sub1_with_file &&
+			test_must_fail $command &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -484,7 +494,8 @@ test_submodule_switch_common() {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_file &&
+			$prep replace_sub1_with_file &&
+			test_must_fail $command &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -508,7 +519,8 @@ test_submodule_switch_common() {
 		(
 			cd submodule_update &&
 			git branch -t modify_sub1 origin/modify_sub1 &&
-			$command modify_sub1 &&
+			$prep modify_sub1 &&
+			$command &&
 			test_superproject_content origin/modify_sub1 &&
 			test_submodule_content sub1 origin/add_sub1 &&
 			git submodule update &&
@@ -523,7 +535,8 @@ test_submodule_switch_common() {
 		(
 			cd submodule_update &&
 			git branch -t invalid_sub1 origin/invalid_sub1 &&
-			$command invalid_sub1 &&
+			$prep invalid_sub1 &&
+			$command &&
 			test_superproject_content origin/invalid_sub1 &&
 			test_submodule_content sub1 origin/add_sub1 &&
 			test_must_fail git submodule update &&
@@ -538,7 +551,8 @@ test_submodule_switch_common() {
 		(
 			cd submodule_update &&
 			git branch -t valid_sub1 origin/valid_sub1 &&
-			$command valid_sub1 &&
+			$prep valid_sub1 &&
+			$command &&
 			test_superproject_content origin/valid_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -568,8 +582,8 @@ test_submodule_switch_common() {
 # }
 # test_submodule_switch "my_func"
 test_submodule_switch () {
-	command="$1"
-	test_submodule_switch_common "$command"
+	command="$2"
+	test_submodule_switch_common "$@"
 
 	# An empty directory does not prevent the creation of a submodule of
 	# the same name, but a file does.
diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
index f916729a12..e3ae7c89f1 100755
--- a/t/t5572-pull-submodule.sh
+++ b/t/t5572-pull-submodule.sh
@@ -11,36 +11,13 @@ reset_branch_to_HEAD () {
 	git branch --set-upstream-to="origin/$1" "$1"
 }
 
-git_pull () {
-	reset_branch_to_HEAD "$1" &&
-	git pull
-}
-
 # pulls without conflicts
-test_submodule_switch "git_pull"
-
-git_pull_ff () {
-	reset_branch_to_HEAD "$1" &&
-	git pull --ff
-}
-
-test_submodule_switch "git_pull_ff"
-
-git_pull_ff_only () {
-	reset_branch_to_HEAD "$1" &&
-	git pull --ff-only
-}
-
-test_submodule_switch "git_pull_ff_only"
-
-git_pull_noff () {
-	reset_branch_to_HEAD "$1" &&
-	git pull --no-ff
-}
-
+test_submodule_switch "reset_branch_to_HEAD" "git pull"
+test_submodule_switch "reset_branch_to_HEAD" "git pull -ff"
+test_submodule_switch "reset_branch_to_HEAD" "git pull --ff-only"
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "git_pull_noff"
+test_submodule_switch "reset_branch_to_HEAD" "git pull --no-ff"
 
 test_expect_success 'pull --recurse-submodule setup' '
 	test_create_repo child &&

Re: [PATCH v2 4/4] lib-submodule-update: pass OVERWRITING_FAIL

[Jeff King]

On Wed, May 20, 2020 at 08:24:18PM -0400, Denton Liu wrote:

>  test_submodule_switch () {
> -	test_submodule_switch_func "git $1"
> +	test_submodule_switch_func "eval \$OVERWRITING_FAIL git $1"
>  }
> [...]
>  test_submodule_forced_switch () {
>  	command="$1"
>  	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
> -	test_submodule_switch_common "git $command"
> +	test_submodule_switch_common "eval \$OVERWRITING_FAIL git $command"

These both subject $1 to an extra layer of shell eval. That may not
matter since we seem to only pass basic stuff. It could probably be
avoided with an extra variable (though since we're passing along the
eval to be eval'd, I think it gets tricky), so it may not be worth
addressing.

Though IMHO it is nicer still if we can avoid the eval altogether by
doing something like the prep/command split I suggested in a sibling
reply.

-Peff

Re: [PATCH v2 0/4] t: replace incorrect test_must_fail usage (part 5)

[Junio C Hamano]

Denton Liu <liu.denton@gmail.com> writes:

> Hi all,
>
> This is mostly a resend of what's currently queued in
> "dl/test-must-fail-fixes-5" except with a tiny bit of cleanup on the tip
> patch. I'd appreciate a review on this series so that we can finally get
> rid of that "Needs review" on the What's Cooking messages ;)

Thanks.  

The OVERWRITING_FAIL one was the only one I was unhappy about, so it
would be good to have more eyeballs on it---perhaps other people
find the approach acceptable, or can suggest more readable and
understandable approach.

Re: [PATCH v2 0/4] t: replace incorrect test_must_fail usage (part 5)

[Jeff King]

On Thu, May 21, 2020 at 09:47:22AM -0700, Junio C Hamano wrote:

> Denton Liu <liu.denton@gmail.com> writes:
> 
> > Hi all,
> >
> > This is mostly a resend of what's currently queued in
> > "dl/test-must-fail-fixes-5" except with a tiny bit of cleanup on the tip
> > patch. I'd appreciate a review on this series so that we can finally get
> > rid of that "Needs review" on the What's Cooking messages ;)
> 
> Thanks.  
> 
> The OVERWRITING_FAIL one was the only one I was unhappy about, so it
> would be good to have more eyeballs on it---perhaps other people
> find the approach acceptable, or can suggest more readable and
> understandable approach.

FWIW, I don't really like it either. :) I gave my best shot at an
alternative in reply to that patch.

-Peff

[PATCH v3 0/4] t: replace incorrect test_must_fail usage (part 5)

[Denton Liu]

Hi all,

Sorry for taking so long with the reroll. I've taken Peff's suggestion
and hopefully this approach is a lot more understandable. Unfortunately,
we introduce a few evals in test_submodule_switch_common() in the
process but I think that the end result looks pretty clean.


The overall scope of these patches is to replace inappropriate uses of
test_must_fail. IOW, we should only allow test_must_fail to run on `git`
and `test-tool`. Ultimately, we will conclude by making test_must_fail
error out on non-git commands. An advance view of the final series can
be found here[1].

This is the fifth part. It focuses on lib-submodule-update.sh and tests
that make use of it.

The first part can be found here[2]. The second part can be found
here[3]. The third part can be found here[4]. The fourth part can be
found here[5].

Changes since v1.2:

* In "lib-submodule-update: pass OVERWRITING_FAIL", use if-then return
  to reduce the amount of code churn

Changes since v2:

* Replace the OVERWRITING_FAIL approach with callback functions as
  suggested by Peff[6]

[1]: (may be rebased at any time) https://github.com/Denton-L/git/tree/ready/cleanup-test-must-fail2
[2]: https://lore.kernel.org/git/cover.1576583819.git.liu.denton@gmail.com/
[3]: https://lore.kernel.org/git/cover.1577454401.git.liu.denton@gmail.com/
[4]: https://lore.kernel.org/git/cover.1585209554.git.liu.denton@gmail.com/
[5]: https://lore.kernel.org/git/cover.1587372771.git.liu.denton@gmail.com/
[6]: https://lore.kernel.org/git/20200521182928.GA1308647@coredump.intra.peff.net/

Denton Liu (4):
  lib-submodule-update: add space after function name
  lib-submodule-update: consolidate --recurse-submodules
  lib-submodule-update: prepend "git" to $command
  lib-submodule-update: use callbacks in test_submodule_switch_common()

 t/lib-submodule-update.sh        | 115 +++++++++++++++++++++++--------
 t/t1013-read-tree-submodule.sh   |   4 +-
 t/t2013-checkout-submodule.sh    |   4 +-
 t/t3426-rebase-submodule.sh      |  10 ++-
 t/t3512-cherry-pick-submodule.sh |   2 +-
 t/t3513-revert-submodule.sh      |  10 +--
 t/t3906-stash-submodule.sh       |  10 +--
 t/t4137-apply-submodule.sh       |  12 ++--
 t/t4255-am-submodule.sh          |  12 ++--
 t/t5572-pull-submodule.sh        |  28 ++------
 t/t6041-bisect-submodule.sh      |  10 +--
 t/t7112-reset-submodule.sh       |   6 +-
 t/t7613-merge-submodule.sh       |   8 +--
 13 files changed, 132 insertions(+), 99 deletions(-)

Range-diff against v2:
1:  ba2f642e0f = 1:  ba2f642e0f lib-submodule-update: add space after function name
2:  16d0a3eb9a = 2:  16d0a3eb9a lib-submodule-update: consolidate --recurse-submodules
3:  578bab6f1a ! 3:  09446be5b9 lib-submodule-update: prepend "git" to $command
    @@ t/lib-submodule-update.sh: test_submodule_content () {
      
     -# Internal function; use test_submodule_switch() or
     -# test_submodule_forced_switch() instead.
    -+# Internal function; use test_submodule_switch_func(), test_submodule_switch_func(),
    ++# Internal function; use test_submodule_switch_func(), test_submodule_switch(),
     +# or test_submodule_forced_switch() instead.
      test_submodule_switch_common () {
      	command="$1"
4:  48598e3f98 < -:  ---------- lib-submodule-update: pass OVERWRITING_FAIL
-:  ---------- > 4:  74e6086da4 lib-submodule-update: use callbacks in test_submodule_switch_common()
-- 
2.27.0.132.g321788e831

[PATCH v3 1/4] lib-submodule-update: add space after function name

[Denton Liu]

In the shell scripts in this codebase, the usual style is to include a
space between the function name and the (). Add these missing spaces to
conform to the usual style of the code.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 1dd17fc03e..122554acd6 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -183,7 +183,7 @@ test_git_directory_is_unchanged () {
 	)
 }
 
-test_git_directory_exists() {
+test_git_directory_exists () {
 	test -e ".git/modules/$1" &&
 	if test -f sub1/.git
 	then
@@ -309,7 +309,7 @@ test_submodule_content () {
 
 # Internal function; use test_submodule_switch() or
 # test_submodule_forced_switch() instead.
-test_submodule_switch_common() {
+test_submodule_switch_common () {
 	command="$1"
 	######################### Appearing submodule #########################
 	# Switching to a commit letting a submodule appear creates empty dir ...
@@ -629,7 +629,7 @@ test_submodule_forced_switch () {
 
 # Internal function; use test_submodule_switch_recursing_with_args() or
 # test_submodule_forced_switch_recursing_with_args() instead.
-test_submodule_recursing_with_args_common() {
+test_submodule_recursing_with_args_common () {
 	command="$1"
 
 	######################### Appearing submodule #########################
-- 
2.27.0.132.g321788e831

[PATCH v3 2/4] lib-submodule-update: consolidate --recurse-submodules

[Denton Liu]

Both test_submodule_switch_recursing_with_args() and
test_submodule_forced_switch_recursing_with_args() call the internal
function test_submodule_recursing_with_args_common() with the final
argument of `--recurse-submodules`. Consolidate this duplication by
appending the argument in test_submodule_recursing_with_args_common().

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 122554acd6..bb36287803 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -630,7 +630,7 @@ test_submodule_forced_switch () {
 # Internal function; use test_submodule_switch_recursing_with_args() or
 # test_submodule_forced_switch_recursing_with_args() instead.
 test_submodule_recursing_with_args_common () {
-	command="$1"
+	command="$1 --recurse-submodules"
 
 	######################### Appearing submodule #########################
 	# Switching to a commit letting a submodule appear checks it out ...
@@ -809,7 +809,7 @@ test_submodule_recursing_with_args_common () {
 # test_submodule_switch_recursing_with_args "$GIT_COMMAND"
 test_submodule_switch_recursing_with_args () {
 	cmd_args="$1"
-	command="git $cmd_args --recurse-submodules"
+	command="git $cmd_args"
 	test_submodule_recursing_with_args_common "$command"
 
 	RESULTDS=success
@@ -927,7 +927,7 @@ test_submodule_switch_recursing_with_args () {
 # away local changes in the superproject is allowed.
 test_submodule_forced_switch_recursing_with_args () {
 	cmd_args="$1"
-	command="git $cmd_args --recurse-submodules"
+	command="git $cmd_args"
 	test_submodule_recursing_with_args_common "$command"
 
 	RESULT=success
-- 
2.27.0.132.g321788e831

[PATCH v3 3/4] lib-submodule-update: prepend "git" to $command

[Denton Liu]

Since all invocations of test_submodule_forced_switch() are git
commands, automatically prepend "git" before invoking
test_submodule_switch_common().

Similarly, many invocations of test_submodule_switch() are also git
commands so automatically prepend "git" before invoking
test_submodule_switch_common() as well.

Finally, for invocations of test_submodule_switch() that invoke a custom
function, rename the old function to test_submodule_switch_func().

This is necessary because in a future commit, we will be adding some
logic that needs to distinguish between an invocation of a plain git
comamnd and an invocation of a test helper function.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh        | 14 +++++++++-----
 t/t1013-read-tree-submodule.sh   |  4 ++--
 t/t2013-checkout-submodule.sh    |  4 ++--
 t/t3426-rebase-submodule.sh      |  4 ++--
 t/t3512-cherry-pick-submodule.sh |  2 +-
 t/t3513-revert-submodule.sh      |  2 +-
 t/t3906-stash-submodule.sh       |  2 +-
 t/t4137-apply-submodule.sh       |  4 ++--
 t/t4255-am-submodule.sh          |  4 ++--
 t/t5572-pull-submodule.sh        |  8 ++++----
 t/t6041-bisect-submodule.sh      |  2 +-
 t/t7112-reset-submodule.sh       |  6 +++---
 t/t7613-merge-submodule.sh       |  8 ++++----
 13 files changed, 34 insertions(+), 30 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index bb36287803..7c3ba1be00 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -307,8 +307,8 @@ test_submodule_content () {
 # to protect the history!
 #
 
-# Internal function; use test_submodule_switch() or
-# test_submodule_forced_switch() instead.
+# Internal function; use test_submodule_switch_func(), test_submodule_switch(),
+# or test_submodule_forced_switch() instead.
 test_submodule_switch_common () {
 	command="$1"
 	######################### Appearing submodule #########################
@@ -566,8 +566,8 @@ test_submodule_switch_common () {
 #   # Do something here that updates the worktree and index to match target,
 #   # but not any submodule directories.
 # }
-# test_submodule_switch "my_func"
-test_submodule_switch () {
+# test_submodule_switch_func "my_func"
+test_submodule_switch_func () {
 	command="$1"
 	test_submodule_switch_common "$command"
 
@@ -587,12 +587,16 @@ test_submodule_switch () {
 	'
 }
 
+test_submodule_switch () {
+	test_submodule_switch_func "git $1"
+}
+
 # Same as test_submodule_switch(), except that throwing away local changes in
 # the superproject is allowed.
 test_submodule_forced_switch () {
 	command="$1"
 	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
-	test_submodule_switch_common "$command"
+	test_submodule_switch_common "git $command"
 
 	# When forced, a file in the superproject does not prevent creating a
 	# submodule of the same name.
diff --git a/t/t1013-read-tree-submodule.sh b/t/t1013-read-tree-submodule.sh
index 91a6fafcb4..b6df7444c0 100755
--- a/t/t1013-read-tree-submodule.sh
+++ b/t/t1013-read-tree-submodule.sh
@@ -12,8 +12,8 @@ test_submodule_switch_recursing_with_args "read-tree -u -m"
 
 test_submodule_forced_switch_recursing_with_args "read-tree -u --reset"
 
-test_submodule_switch "git read-tree -u -m"
+test_submodule_switch "read-tree -u -m"
 
-test_submodule_forced_switch "git read-tree -u --reset"
+test_submodule_forced_switch "read-tree -u --reset"
 
 test_done
diff --git a/t/t2013-checkout-submodule.sh b/t/t2013-checkout-submodule.sh
index 8f86b5f4b2..b2bdd1fcb4 100755
--- a/t/t2013-checkout-submodule.sh
+++ b/t/t2013-checkout-submodule.sh
@@ -68,8 +68,8 @@ test_submodule_switch_recursing_with_args "checkout"
 
 test_submodule_forced_switch_recursing_with_args "checkout -f"
 
-test_submodule_switch "git checkout"
+test_submodule_switch "checkout"
 
-test_submodule_forced_switch "git checkout -f"
+test_submodule_forced_switch "checkout -f"
 
 test_done
diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
index a2bba04ba9..788605ccc0 100755
--- a/t/t3426-rebase-submodule.sh
+++ b/t/t3426-rebase-submodule.sh
@@ -20,7 +20,7 @@ git_rebase () {
 	git rebase "$1"
 }
 
-test_submodule_switch "git_rebase"
+test_submodule_switch_func "git_rebase"
 
 git_rebase_interactive () {
 	git status -su >expect &&
@@ -38,7 +38,7 @@ git_rebase_interactive () {
 	git rebase -i "$1"
 }
 
-test_submodule_switch "git_rebase_interactive"
+test_submodule_switch_func "git_rebase_interactive"
 
 test_expect_success 'rebase interactive ignores modified submodules' '
 	test_when_finished "rm -rf super sub" &&
diff --git a/t/t3512-cherry-pick-submodule.sh b/t/t3512-cherry-pick-submodule.sh
index bd78287841..6ece1d8573 100755
--- a/t/t3512-cherry-pick-submodule.sh
+++ b/t/t3512-cherry-pick-submodule.sh
@@ -7,7 +7,7 @@ test_description='cherry-pick can handle submodules'
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "git cherry-pick"
+test_submodule_switch "cherry-pick"
 
 test_expect_success 'unrelated submodule/file conflict is ignored' '
 	test_create_repo sub &&
diff --git a/t/t3513-revert-submodule.sh b/t/t3513-revert-submodule.sh
index 5e39fcdb66..95a7f64471 100755
--- a/t/t3513-revert-submodule.sh
+++ b/t/t3513-revert-submodule.sh
@@ -26,6 +26,6 @@ git_revert () {
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
-test_submodule_switch "git_revert"
+test_submodule_switch_func "git_revert"
 
 test_done
diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
index b93d1d74da..6a7e801ca0 100755
--- a/t/t3906-stash-submodule.sh
+++ b/t/t3906-stash-submodule.sh
@@ -19,7 +19,7 @@ git_stash () {
 KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES=1
 KNOWN_FAILURE_CHERRY_PICK_SEES_EMPTY_COMMIT=1
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
-test_submodule_switch "git_stash"
+test_submodule_switch_func "git_stash"
 
 setup_basic () {
 	test_when_finished "rm -rf main sub" &&
diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
index a9bd40a6d0..b645e303a0 100755
--- a/t/t4137-apply-submodule.sh
+++ b/t/t4137-apply-submodule.sh
@@ -9,12 +9,12 @@ apply_index () {
 	git diff --ignore-submodules=dirty "..$1" | git apply --index -
 }
 
-test_submodule_switch "apply_index"
+test_submodule_switch_func "apply_index"
 
 apply_3way () {
 	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
 }
 
-test_submodule_switch "apply_3way"
+test_submodule_switch_func "apply_3way"
 
 test_done
diff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh
index 0ba8194403..1b179d5f45 100755
--- a/t/t4255-am-submodule.sh
+++ b/t/t4255-am-submodule.sh
@@ -9,14 +9,14 @@ am () {
 	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am -
 }
 
-test_submodule_switch "am"
+test_submodule_switch_func "am"
 
 am_3way () {
 	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am --3way -
 }
 
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "am_3way"
+test_submodule_switch_func "am_3way"
 
 test_expect_success 'setup diff.submodule' '
 	test_commit one &&
diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
index f916729a12..f911bf631e 100755
--- a/t/t5572-pull-submodule.sh
+++ b/t/t5572-pull-submodule.sh
@@ -17,21 +17,21 @@ git_pull () {
 }
 
 # pulls without conflicts
-test_submodule_switch "git_pull"
+test_submodule_switch_func "git_pull"
 
 git_pull_ff () {
 	reset_branch_to_HEAD "$1" &&
 	git pull --ff
 }
 
-test_submodule_switch "git_pull_ff"
+test_submodule_switch_func "git_pull_ff"
 
 git_pull_ff_only () {
 	reset_branch_to_HEAD "$1" &&
 	git pull --ff-only
 }
 
-test_submodule_switch "git_pull_ff_only"
+test_submodule_switch_func "git_pull_ff_only"
 
 git_pull_noff () {
 	reset_branch_to_HEAD "$1" &&
@@ -40,7 +40,7 @@ git_pull_noff () {
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "git_pull_noff"
+test_submodule_switch_func "git_pull_noff"
 
 test_expect_success 'pull --recurse-submodule setup' '
 	test_create_repo child &&
diff --git a/t/t6041-bisect-submodule.sh b/t/t6041-bisect-submodule.sh
index 62b8a2e7bb..0e0cdf638d 100755
--- a/t/t6041-bisect-submodule.sh
+++ b/t/t6041-bisect-submodule.sh
@@ -27,6 +27,6 @@ git_bisect () {
 	git bisect bad $BAD
 }
 
-test_submodule_switch "git_bisect"
+test_submodule_switch_func "git_bisect"
 
 test_done
diff --git a/t/t7112-reset-submodule.sh b/t/t7112-reset-submodule.sh
index a1cb9ff858..8741b665c9 100755
--- a/t/t7112-reset-submodule.sh
+++ b/t/t7112-reset-submodule.sh
@@ -13,10 +13,10 @@ test_submodule_switch_recursing_with_args "reset --keep"
 
 test_submodule_forced_switch_recursing_with_args "reset --hard"
 
-test_submodule_switch "git reset --keep"
+test_submodule_switch "reset --keep"
 
-test_submodule_switch "git reset --merge"
+test_submodule_switch "reset --merge"
 
-test_submodule_forced_switch "git reset --hard"
+test_submodule_forced_switch "reset --hard"
 
 test_done
diff --git a/t/t7613-merge-submodule.sh b/t/t7613-merge-submodule.sh
index d1e9fcc781..04bf4be7d7 100755
--- a/t/t7613-merge-submodule.sh
+++ b/t/t7613-merge-submodule.sh
@@ -6,14 +6,14 @@ test_description='merge can handle submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 # merges without conflicts
-test_submodule_switch "git merge"
+test_submodule_switch "merge"
 
-test_submodule_switch "git merge --ff"
+test_submodule_switch "merge --ff"
 
-test_submodule_switch "git merge --ff-only"
+test_submodule_switch "merge --ff-only"
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "git merge --no-ff"
+test_submodule_switch "merge --no-ff"
 
 test_done
-- 
2.27.0.132.g321788e831

[PATCH v3 4/4] lib-submodule-update: use callbacks in test_submodule_switch_common()

[Denton Liu]

When we run a test helper function in test_submodule_switch_common(), we
sometimes specify a whole helper function as the $command. When we do
this, in some test cases, we just mark the whole function with
`test_must_fail`. However, it's possible that the helper function might
fail earlier or later than expected due to an introduced bug. If this
happens, then the test case will still report as passing but it should
really be marked as failing since it didn't actually display the
intended behaviour.

Instead of invoking $command as one monolithic helper function, break it
up into three parts:

	1. $command which is always a git command.
	2. $before which is a callback function that runs just prior to
	   $command.
	3. $after which is a callback function that runs just after
	   $command.

If the command requires a filename argument, specify it as `\$arg` since
that variable will be set and the whole $command string will be eval'd.
Unfortunately, there is no way to get rid of the eval as some of the
commands are passed (such as the `git pull` tests) require that no
additional arguments are passed so we must have some mechanism for the
caller to specify whether or not it wants the filename argument.

The $before and $after callback functions will be passed the filename as
the first arg. These callback functions are optional and, if missing,
will be replaced with `true`. Also, in the case where we have a
`test_must_fail` test, $after will not be executed, similar to how the
helper functions currently behave when the git command fails and exits
the &&-chain.

Finally, as an added bonus, `test_must_fail` will only run on $command
which is guaranteed to be a git command.

An alternate design was considered where the $OVERWRITING_FAIL is set
from the test_submodule_switch_common() function and passed to the
helper function. This approach was considered too difficult to
understand due to the fact that using a signalling magic environment
variable might be too indirect.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh   | 95 +++++++++++++++++++++++++++++--------
 t/t3426-rebase-submodule.sh | 10 ++--
 t/t3513-revert-submodule.sh | 10 ++--
 t/t3906-stash-submodule.sh  | 10 ++--
 t/t4137-apply-submodule.sh  | 12 ++---
 t/t4255-am-submodule.sh     | 12 ++---
 t/t5572-pull-submodule.sh   | 28 ++---------
 t/t6041-bisect-submodule.sh | 10 ++--
 8 files changed, 108 insertions(+), 79 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 7c3ba1be00..988d85bd7d 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -310,7 +310,20 @@ test_submodule_content () {
 # Internal function; use test_submodule_switch_func(), test_submodule_switch(),
 # or test_submodule_forced_switch() instead.
 test_submodule_switch_common () {
-	command="$1"
+	command="$1" # should be a git command
+	before="$2"
+	after="$3"
+
+	if test -z "$before"
+	then
+		before=true
+	fi
+
+	if test -z "$after"
+	then
+		after=true
+	fi
+
 	######################### Appearing submodule #########################
 	# Switching to a commit letting a submodule appear creates empty dir ...
 	if test "$KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES" = 1
@@ -326,7 +339,10 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t add_sub1 origin/add_sub1 &&
-			$command add_sub1 &&
+			arg=add_sub1 &&
+			$before "$arg" &&
+			eval $command &&
+			$after "$arg" &&
 			test_superproject_content origin/add_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -341,7 +357,10 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			mkdir sub1 &&
 			git branch -t add_sub1 origin/add_sub1 &&
-			$command add_sub1 &&
+			arg=add_sub1 &&
+			$before "$arg" &&
+			eval $command &&
+			$after "$arg" &&
 			test_superproject_content origin/add_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -356,7 +375,10 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&
-			$command replace_file_with_sub1 &&
+			arg=replace_file_with_sub1 &&
+			$before "$arg" &&
+			eval $command &&
+			$after "$arg" &&
 			test_superproject_content origin/replace_file_with_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -380,7 +402,10 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&
-			$command replace_directory_with_sub1 &&
+			arg=replace_directory_with_sub1  &&
+			$before "$arg" &&
+			eval $command &&
+			$after "$arg" &&
 			test_superproject_content origin/replace_directory_with_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -402,7 +427,10 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t remove_sub1 origin/remove_sub1 &&
-			$command remove_sub1 &&
+			arg=remove_sub1 &&
+			$before "$arg" &&
+			eval $command &&
+			$after "$arg" &&
 			test_superproject_content origin/remove_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -415,7 +443,10 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t remove_sub1 origin/remove_sub1 &&
 			replace_gitfile_with_git_dir sub1 &&
-			$command remove_sub1 &&
+			arg=remove_sub1 &&
+			$before "$arg" &&
+			eval $command &&
+			$after "$arg" &&
 			test_superproject_content origin/remove_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -443,7 +474,9 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
-			test_must_fail $command replace_sub1_with_directory &&
+			arg=replace_sub1_with_directory &&
+			$before "$arg" &&
+			eval test_must_fail $command &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -456,7 +489,9 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_directory &&
+			arg=replace_sub1_with_directory &&
+			$before "$arg" &&
+			eval test_must_fail $command &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -470,7 +505,9 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
-			test_must_fail $command replace_sub1_with_file &&
+			arg=replace_sub1_with_file &&
+			$before "$arg" &&
+			eval test_must_fail $command &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -484,7 +521,9 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_file &&
+			arg=replace_sub1_with_file &&
+			$before "$arg" &&
+			eval test_must_fail $command &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -508,7 +547,10 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t modify_sub1 origin/modify_sub1 &&
-			$command modify_sub1 &&
+			arg=modify_sub1 &&
+			$before "$arg" &&
+			eval $command &&
+			$after "$arg" &&
 			test_superproject_content origin/modify_sub1 &&
 			test_submodule_content sub1 origin/add_sub1 &&
 			git submodule update &&
@@ -523,7 +565,10 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t invalid_sub1 origin/invalid_sub1 &&
-			$command invalid_sub1 &&
+			arg=invalid_sub1 &&
+			$before "$arg" &&
+			eval $command &&
+			$after "$arg" &&
 			test_superproject_content origin/invalid_sub1 &&
 			test_submodule_content sub1 origin/add_sub1 &&
 			test_must_fail git submodule update &&
@@ -538,7 +583,10 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t valid_sub1 origin/valid_sub1 &&
-			$command valid_sub1 &&
+			arg=valid_sub1 &&
+			$before "$arg" &&
+			eval $command &&
+			$after "$arg" &&
 			test_superproject_content origin/valid_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -568,8 +616,10 @@ test_submodule_switch_common () {
 # }
 # test_submodule_switch_func "my_func"
 test_submodule_switch_func () {
-	command="$1"
-	test_submodule_switch_common "$command"
+	command="git $1"
+	before="$2"
+	after="$3"
+	test_submodule_switch_common "$command" "$before" "$after"
 
 	# An empty directory does not prevent the creation of a submodule of
 	# the same name, but a file does.
@@ -580,7 +630,9 @@ test_submodule_switch_func () {
 			cd submodule_update &&
 			git branch -t add_sub1 origin/add_sub1 &&
 			>sub1 &&
-			test_must_fail $command add_sub1 &&
+			arg=add_sub1 &&
+			$before "$arg" &&
+			eval test_must_fail $command &&
 			test_superproject_content origin/no_submodule &&
 			test_must_be_empty sub1
 		)
@@ -588,15 +640,15 @@ test_submodule_switch_func () {
 }
 
 test_submodule_switch () {
-	test_submodule_switch_func "git $1"
+	test_submodule_switch_func "$1 \$arg"
 }
 
 # Same as test_submodule_switch(), except that throwing away local changes in
 # the superproject is allowed.
 test_submodule_forced_switch () {
-	command="$1"
+	command="git $1 \$arg"
 	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
-	test_submodule_switch_common "git $command"
+	test_submodule_switch_common "$command"
 
 	# When forced, a file in the superproject does not prevent creating a
 	# submodule of the same name.
@@ -607,7 +659,8 @@ test_submodule_forced_switch () {
 			cd submodule_update &&
 			git branch -t add_sub1 origin/add_sub1 &&
 			>sub1 &&
-			$command add_sub1 &&
+			arg=add_sub1 &&
+			eval $command &&
 			test_superproject_content origin/add_sub1 &&
 			test_dir_is_empty sub1
 		)
diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
index 788605ccc0..d31e6487bd 100755
--- a/t/t3426-rebase-submodule.sh
+++ b/t/t3426-rebase-submodule.sh
@@ -16,11 +16,10 @@ git_rebase () {
 	git revert HEAD &&
 	git status -su >actual &&
 	ls -1pR * >>actual &&
-	test_cmp expect actual &&
-	git rebase "$1"
+	test_cmp expect actual
 }
 
-test_submodule_switch_func "git_rebase"
+test_submodule_switch_func "rebase \$arg" "git_rebase"
 
 git_rebase_interactive () {
 	git status -su >expect &&
@@ -34,11 +33,10 @@ git_rebase_interactive () {
 	ls -1pR * >>actual &&
 	test_cmp expect actual &&
 	set_fake_editor &&
-	echo "fake-editor.sh" >.git/info/exclude &&
-	git rebase -i "$1"
+	echo "fake-editor.sh" >.git/info/exclude
 }
 
-test_submodule_switch_func "git_rebase_interactive"
+test_submodule_switch_func "rebase -i \$arg" "git_rebase_interactive"
 
 test_expect_success 'rebase interactive ignores modified submodules' '
 	test_when_finished "rm -rf super sub" &&
diff --git a/t/t3513-revert-submodule.sh b/t/t3513-revert-submodule.sh
index 95a7f64471..994cdc40f0 100755
--- a/t/t3513-revert-submodule.sh
+++ b/t/t3513-revert-submodule.sh
@@ -11,11 +11,13 @@ test_description='revert can handle submodules'
 # first so we can restore the work tree test setup after doing the checkout
 # and revert.  We test here that the restored work tree content is identical
 # to that at the beginning. The last revert is then tested by the framework.
-git_revert () {
+git_revert_before () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
-	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
-	git checkout "$1" &&
+	tar cf "$TRASH_DIRECTORY/tmp.tar" *
+}
+
+git_revert_after () {
 	git revert HEAD &&
 	rm -rf * &&
 	tar xf "$TRASH_DIRECTORY/tmp.tar" &&
@@ -26,6 +28,6 @@ git_revert () {
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
-test_submodule_switch_func "git_revert"
+test_submodule_switch_func "checkout \$arg" "git_revert_before" "git_revert_after"
 
 test_done
diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
index 6a7e801ca0..358a625124 100755
--- a/t/t3906-stash-submodule.sh
+++ b/t/t3906-stash-submodule.sh
@@ -5,10 +5,12 @@ test_description='stash can handle submodules'
 . ./test-lib.sh
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
-git_stash () {
+git_stash_before () {
 	git status -su >expect &&
-	ls -1pR * >>expect &&
-	git read-tree -u -m "$1" &&
+	ls -1pR * >>expect
+}
+
+git_stash_after () {
 	git stash &&
 	git status -su >actual &&
 	ls -1pR * >>actual &&
@@ -19,7 +21,7 @@ git_stash () {
 KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES=1
 KNOWN_FAILURE_CHERRY_PICK_SEES_EMPTY_COMMIT=1
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
-test_submodule_switch_func "git_stash"
+test_submodule_switch_func "read-tree -u -m \$arg" "git_stash_before" "git_stash_after"
 
 setup_basic () {
 	test_when_finished "rm -rf main sub" &&
diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
index b645e303a0..fe5ebeaa41 100755
--- a/t/t4137-apply-submodule.sh
+++ b/t/t4137-apply-submodule.sh
@@ -5,16 +5,12 @@ test_description='git apply handling submodules'
 . ./test-lib.sh
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
-apply_index () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --index -
+create_diff () {
+	git diff --ignore-submodules=dirty "..$1" >diff
 }
 
-test_submodule_switch_func "apply_index"
+test_submodule_switch_func "apply --index diff" "create_diff"
 
-apply_3way () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
-}
-
-test_submodule_switch_func "apply_3way"
+test_submodule_switch_func "apply --3way diff" "create_diff"
 
 test_done
diff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh
index 1b179d5f45..5305280dfd 100755
--- a/t/t4255-am-submodule.sh
+++ b/t/t4255-am-submodule.sh
@@ -5,18 +5,14 @@ test_description='git am handling submodules'
 . ./test-lib.sh
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
-am () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am -
+create_patch () {
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch
 }
 
-test_submodule_switch_func "am"
-
-am_3way () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am --3way -
-}
+test_submodule_switch_func "am patch" "create_patch"
 
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch_func "am_3way"
+test_submodule_switch_func "am --3way patch" "create_patch"
 
 test_expect_success 'setup diff.submodule' '
 	test_commit one &&
diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
index f911bf631e..3b37853537 100755
--- a/t/t5572-pull-submodule.sh
+++ b/t/t5572-pull-submodule.sh
@@ -11,36 +11,16 @@ reset_branch_to_HEAD () {
 	git branch --set-upstream-to="origin/$1" "$1"
 }
 
-git_pull () {
-	reset_branch_to_HEAD "$1" &&
-	git pull
-}
-
 # pulls without conflicts
-test_submodule_switch_func "git_pull"
+test_submodule_switch_func "pull" "reset_branch_to_HEAD"
 
-git_pull_ff () {
-	reset_branch_to_HEAD "$1" &&
-	git pull --ff
-}
+test_submodule_switch_func "pull --ff" "reset_branch_to_HEAD"
 
-test_submodule_switch_func "git_pull_ff"
-
-git_pull_ff_only () {
-	reset_branch_to_HEAD "$1" &&
-	git pull --ff-only
-}
-
-test_submodule_switch_func "git_pull_ff_only"
-
-git_pull_noff () {
-	reset_branch_to_HEAD "$1" &&
-	git pull --no-ff
-}
+test_submodule_switch_func "pull --ff-only" "reset_branch_to_HEAD"
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch_func "git_pull_noff"
+test_submodule_switch_func "pull --no-ff" "reset_branch_to_HEAD"
 
 test_expect_success 'pull --recurse-submodule setup' '
 	test_create_repo child &&
diff --git a/t/t6041-bisect-submodule.sh b/t/t6041-bisect-submodule.sh
index 0e0cdf638d..b99e81d55d 100755
--- a/t/t6041-bisect-submodule.sh
+++ b/t/t6041-bisect-submodule.sh
@@ -5,12 +5,14 @@ test_description='bisect can handle submodules'
 . ./test-lib.sh
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
-git_bisect () {
+git_bisect_before () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
-	GOOD=$(git rev-parse --verify HEAD) &&
-	git checkout "$1" &&
+	GOOD=$(git rev-parse --verify HEAD)
+}
+
+git_bisect_after () {
 	echo "foo" >bar &&
 	git add bar &&
 	git commit -m "bisect bad" &&
@@ -27,6 +29,6 @@ git_bisect () {
 	git bisect bad $BAD
 }
 
-test_submodule_switch_func "git_bisect"
+test_submodule_switch_func "checkout \$arg" "git_bisect_before" "git_bisect_after"
 
 test_done
-- 
2.27.0.132.g321788e831

Re: [PATCH v3 4/4] lib-submodule-update: use callbacks in test_submodule_switch_common()

[Junio C Hamano]

Denton Liu <liu.denton@gmail.com> writes:

>  test_submodule_switch_common () {
> -	command="$1"
> +	command="$1" # should be a git command
> +	before="$2"
> +	after="$3"
> +
> +	if test -z "$before"
> +	then
> +		before=true
> +	fi
> +
> +	if test -z "$after"
> +	then
> +		after=true
> +	fi
> +

	command="$1" before=${2:-true} after=${3:-true}

should be sufficient?  Spelling 'true' as ':' would make it even
shorter but probably too cryptic ;-)

> diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
> index 788605ccc0..d31e6487bd 100755
> --- a/t/t3426-rebase-submodule.sh
> +++ b/t/t3426-rebase-submodule.sh
> @@ -16,11 +16,10 @@ git_rebase () {
>  	git revert HEAD &&
>  	git status -su >actual &&
>  	ls -1pR * >>actual &&
> -	test_cmp expect actual &&
> -	git rebase "$1"
> +	test_cmp expect actual

We used to run "git rebase" in this helper, but that is the thing we
expect to succeed or to fail, so with the revamped test framework,
this is split out to just be the "before testing" helper function?

Makes sense.

> -test_submodule_switch_func "git_rebase"
> +test_submodule_switch_func "rebase \$arg" "git_rebase"

>  git_rebase_interactive () {
>  	git status -su >expect &&
> @@ -34,11 +33,10 @@ git_rebase_interactive () {
>  	ls -1pR * >>actual &&
>  	test_cmp expect actual &&
>  	set_fake_editor &&
> -	echo "fake-editor.sh" >.git/info/exclude &&
> -	git rebase -i "$1"
> +	echo "fake-editor.sh" >.git/info/exclude
>  }
>  
> -test_submodule_switch_func "git_rebase_interactive"
> +test_submodule_switch_func "rebase -i \$arg" "git_rebase_interactive"

Likewise.

Re: [PATCH v3 4/4] lib-submodule-update: use callbacks in test_submodule_switch_common()

[Junio C Hamano]

Denton Liu <liu.denton@gmail.com> writes:

> When we run a test helper function in test_submodule_switch_common(), we
> sometimes specify a whole helper function as the $command. When we do
> this, in some test cases, we just mark the whole function with
> `test_must_fail`. However, it's possible that the helper function might
> fail earlier or later than expected due to an introduced bug. If this
> happens, then the test case will still report as passing but it should
> really be marked as failing since it didn't actually display the
> intended behaviour.
>
> Instead of invoking $command as one monolithic helper function, break it
> up into three parts:
>
> 	1. $command which is always a git command.
> 	2. $before which is a callback function that runs just prior to
> 	   $command.
> 	3. $after which is a callback function that runs just after
> 	   $command.
>
> If the command requires a filename argument, specify it as `\$arg` since
> that variable will be set and the whole $command string will be eval'd.
> Unfortunately, there is no way to get rid of the eval as some of the
> commands are passed (such as the `git pull` tests) require that no
> additional arguments are passed so we must have some mechanism for the
> caller to specify whether or not it wants the filename argument.
>
> The $before and $after callback functions will be passed the filename as
> the first arg. These callback functions are optional and, if missing,
> will be replaced with `true`. Also, in the case where we have a
> `test_must_fail` test, $after will not be executed, similar to how the
> helper functions currently behave when the git command fails and exits
> the &&-chain.
>
> Finally, as an added bonus, `test_must_fail` will only run on $command
> which is guaranteed to be a git command.
>
> An alternate design was considered where the $OVERWRITING_FAIL is set
> from the test_submodule_switch_common() function and passed to the
> helper function. This approach was considered too difficult to
> understand due to the fact that using a signalling magic environment
> variable might be too indirect.
>
> Signed-off-by: Denton Liu <liu.denton@gmail.com>
> ---

Peff, I think the previous round of this had all-caps OVERWRITING_FAIL
but the reversal of the role to feed the actual code that gets run
before and after the tested snippet by the caller was made in
response to your sugestion <20200521182928.GA1308647@coredump.intra.peff.net>.

I think I've already said that the helper function in the early part
can be vastly simplified but in any case the calling sideprobably
has become somewhat easier to follow (but not all that much, I am
afraid).  

Is there anything else we missed?

Thanks.

>  t/lib-submodule-update.sh   | 95 +++++++++++++++++++++++++++++--------
>  t/t3426-rebase-submodule.sh | 10 ++--
>  t/t3513-revert-submodule.sh | 10 ++--
>  t/t3906-stash-submodule.sh  | 10 ++--
>  t/t4137-apply-submodule.sh  | 12 ++---
>  t/t4255-am-submodule.sh     | 12 ++---
>  t/t5572-pull-submodule.sh   | 28 ++---------
>  t/t6041-bisect-submodule.sh | 10 ++--
>  8 files changed, 108 insertions(+), 79 deletions(-)
>
> diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
> index 7c3ba1be00..988d85bd7d 100755
> --- a/t/lib-submodule-update.sh
> +++ b/t/lib-submodule-update.sh
> @@ -310,7 +310,20 @@ test_submodule_content () {
>  # Internal function; use test_submodule_switch_func(), test_submodule_switch(),
>  # or test_submodule_forced_switch() instead.
>  test_submodule_switch_common () {
> -	command="$1"
> +	command="$1" # should be a git command
> +	before="$2"
> +	after="$3"
> +
> +	if test -z "$before"
> +	then
> +		before=true
> +	fi
> +
> +	if test -z "$after"
> +	then
> +		after=true
> +	fi
> +
>  	######################### Appearing submodule #########################
>  	# Switching to a commit letting a submodule appear creates empty dir ...
>  	if test "$KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES" = 1
> @@ -326,7 +339,10 @@ test_submodule_switch_common () {
>  		(
>  			cd submodule_update &&
>  			git branch -t add_sub1 origin/add_sub1 &&
> -			$command add_sub1 &&
> +			arg=add_sub1 &&
> +			$before "$arg" &&
> +			eval $command &&
> +			$after "$arg" &&
>  			test_superproject_content origin/add_sub1 &&
>  			test_dir_is_empty sub1 &&
>  			git submodule update --init --recursive &&
> @@ -341,7 +357,10 @@ test_submodule_switch_common () {
>  			cd submodule_update &&
>  			mkdir sub1 &&
>  			git branch -t add_sub1 origin/add_sub1 &&
> -			$command add_sub1 &&
> +			arg=add_sub1 &&
> +			$before "$arg" &&
> +			eval $command &&
> +			$after "$arg" &&
>  			test_superproject_content origin/add_sub1 &&
>  			test_dir_is_empty sub1 &&
>  			git submodule update --init --recursive &&
> @@ -356,7 +375,10 @@ test_submodule_switch_common () {
>  		(
>  			cd submodule_update &&
>  			git branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&
> -			$command replace_file_with_sub1 &&
> +			arg=replace_file_with_sub1 &&
> +			$before "$arg" &&
> +			eval $command &&
> +			$after "$arg" &&
>  			test_superproject_content origin/replace_file_with_sub1 &&
>  			test_dir_is_empty sub1 &&
>  			git submodule update --init --recursive &&
> @@ -380,7 +402,10 @@ test_submodule_switch_common () {
>  		(
>  			cd submodule_update &&
>  			git branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&
> -			$command replace_directory_with_sub1 &&
> +			arg=replace_directory_with_sub1  &&
> +			$before "$arg" &&
> +			eval $command &&
> +			$after "$arg" &&
>  			test_superproject_content origin/replace_directory_with_sub1 &&
>  			test_dir_is_empty sub1 &&
>  			git submodule update --init --recursive &&
> @@ -402,7 +427,10 @@ test_submodule_switch_common () {
>  		(
>  			cd submodule_update &&
>  			git branch -t remove_sub1 origin/remove_sub1 &&
> -			$command remove_sub1 &&
> +			arg=remove_sub1 &&
> +			$before "$arg" &&
> +			eval $command &&
> +			$after "$arg" &&
>  			test_superproject_content origin/remove_sub1 &&
>  			test_submodule_content sub1 origin/add_sub1
>  		)
> @@ -415,7 +443,10 @@ test_submodule_switch_common () {
>  			cd submodule_update &&
>  			git branch -t remove_sub1 origin/remove_sub1 &&
>  			replace_gitfile_with_git_dir sub1 &&
> -			$command remove_sub1 &&
> +			arg=remove_sub1 &&
> +			$before "$arg" &&
> +			eval $command &&
> +			$after "$arg" &&
>  			test_superproject_content origin/remove_sub1 &&
>  			test_git_directory_is_unchanged sub1 &&
>  			test_submodule_content sub1 origin/add_sub1
> @@ -443,7 +474,9 @@ test_submodule_switch_common () {
>  		(
>  			cd submodule_update &&
>  			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
> -			test_must_fail $command replace_sub1_with_directory &&
> +			arg=replace_sub1_with_directory &&
> +			$before "$arg" &&
> +			eval test_must_fail $command &&
>  			test_superproject_content origin/add_sub1 &&
>  			test_submodule_content sub1 origin/add_sub1
>  		)
> @@ -456,7 +489,9 @@ test_submodule_switch_common () {
>  			cd submodule_update &&
>  			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
>  			replace_gitfile_with_git_dir sub1 &&
> -			test_must_fail $command replace_sub1_with_directory &&
> +			arg=replace_sub1_with_directory &&
> +			$before "$arg" &&
> +			eval test_must_fail $command &&
>  			test_superproject_content origin/add_sub1 &&
>  			test_git_directory_is_unchanged sub1 &&
>  			test_submodule_content sub1 origin/add_sub1
> @@ -470,7 +505,9 @@ test_submodule_switch_common () {
>  		(
>  			cd submodule_update &&
>  			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
> -			test_must_fail $command replace_sub1_with_file &&
> +			arg=replace_sub1_with_file &&
> +			$before "$arg" &&
> +			eval test_must_fail $command &&
>  			test_superproject_content origin/add_sub1 &&
>  			test_submodule_content sub1 origin/add_sub1
>  		)
> @@ -484,7 +521,9 @@ test_submodule_switch_common () {
>  			cd submodule_update &&
>  			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
>  			replace_gitfile_with_git_dir sub1 &&
> -			test_must_fail $command replace_sub1_with_file &&
> +			arg=replace_sub1_with_file &&
> +			$before "$arg" &&
> +			eval test_must_fail $command &&
>  			test_superproject_content origin/add_sub1 &&
>  			test_git_directory_is_unchanged sub1 &&
>  			test_submodule_content sub1 origin/add_sub1
> @@ -508,7 +547,10 @@ test_submodule_switch_common () {
>  		(
>  			cd submodule_update &&
>  			git branch -t modify_sub1 origin/modify_sub1 &&
> -			$command modify_sub1 &&
> +			arg=modify_sub1 &&
> +			$before "$arg" &&
> +			eval $command &&
> +			$after "$arg" &&
>  			test_superproject_content origin/modify_sub1 &&
>  			test_submodule_content sub1 origin/add_sub1 &&
>  			git submodule update &&
> @@ -523,7 +565,10 @@ test_submodule_switch_common () {
>  		(
>  			cd submodule_update &&
>  			git branch -t invalid_sub1 origin/invalid_sub1 &&
> -			$command invalid_sub1 &&
> +			arg=invalid_sub1 &&
> +			$before "$arg" &&
> +			eval $command &&
> +			$after "$arg" &&
>  			test_superproject_content origin/invalid_sub1 &&
>  			test_submodule_content sub1 origin/add_sub1 &&
>  			test_must_fail git submodule update &&
> @@ -538,7 +583,10 @@ test_submodule_switch_common () {
>  		(
>  			cd submodule_update &&
>  			git branch -t valid_sub1 origin/valid_sub1 &&
> -			$command valid_sub1 &&
> +			arg=valid_sub1 &&
> +			$before "$arg" &&
> +			eval $command &&
> +			$after "$arg" &&
>  			test_superproject_content origin/valid_sub1 &&
>  			test_dir_is_empty sub1 &&
>  			git submodule update --init --recursive &&
> @@ -568,8 +616,10 @@ test_submodule_switch_common () {
>  # }
>  # test_submodule_switch_func "my_func"
>  test_submodule_switch_func () {
> -	command="$1"
> -	test_submodule_switch_common "$command"
> +	command="git $1"
> +	before="$2"
> +	after="$3"
> +	test_submodule_switch_common "$command" "$before" "$after"
>  
>  	# An empty directory does not prevent the creation of a submodule of
>  	# the same name, but a file does.
> @@ -580,7 +630,9 @@ test_submodule_switch_func () {
>  			cd submodule_update &&
>  			git branch -t add_sub1 origin/add_sub1 &&
>  			>sub1 &&
> -			test_must_fail $command add_sub1 &&
> +			arg=add_sub1 &&
> +			$before "$arg" &&
> +			eval test_must_fail $command &&
>  			test_superproject_content origin/no_submodule &&
>  			test_must_be_empty sub1
>  		)
> @@ -588,15 +640,15 @@ test_submodule_switch_func () {
>  }
>  
>  test_submodule_switch () {
> -	test_submodule_switch_func "git $1"
> +	test_submodule_switch_func "$1 \$arg"
>  }
>  
>  # Same as test_submodule_switch(), except that throwing away local changes in
>  # the superproject is allowed.
>  test_submodule_forced_switch () {
> -	command="$1"
> +	command="git $1 \$arg"
>  	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
> -	test_submodule_switch_common "git $command"
> +	test_submodule_switch_common "$command"
>  
>  	# When forced, a file in the superproject does not prevent creating a
>  	# submodule of the same name.
> @@ -607,7 +659,8 @@ test_submodule_forced_switch () {
>  			cd submodule_update &&
>  			git branch -t add_sub1 origin/add_sub1 &&
>  			>sub1 &&
> -			$command add_sub1 &&
> +			arg=add_sub1 &&
> +			eval $command &&
>  			test_superproject_content origin/add_sub1 &&
>  			test_dir_is_empty sub1
>  		)
> diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
> index 788605ccc0..d31e6487bd 100755
> --- a/t/t3426-rebase-submodule.sh
> +++ b/t/t3426-rebase-submodule.sh
> @@ -16,11 +16,10 @@ git_rebase () {
>  	git revert HEAD &&
>  	git status -su >actual &&
>  	ls -1pR * >>actual &&
> -	test_cmp expect actual &&
> -	git rebase "$1"
> +	test_cmp expect actual
>  }
>  
> -test_submodule_switch_func "git_rebase"
> +test_submodule_switch_func "rebase \$arg" "git_rebase"
>  
>  git_rebase_interactive () {
>  	git status -su >expect &&
> @@ -34,11 +33,10 @@ git_rebase_interactive () {
>  	ls -1pR * >>actual &&
>  	test_cmp expect actual &&
>  	set_fake_editor &&
> -	echo "fake-editor.sh" >.git/info/exclude &&
> -	git rebase -i "$1"
> +	echo "fake-editor.sh" >.git/info/exclude
>  }
>  
> -test_submodule_switch_func "git_rebase_interactive"
> +test_submodule_switch_func "rebase -i \$arg" "git_rebase_interactive"
>  
>  test_expect_success 'rebase interactive ignores modified submodules' '
>  	test_when_finished "rm -rf super sub" &&
> diff --git a/t/t3513-revert-submodule.sh b/t/t3513-revert-submodule.sh
> index 95a7f64471..994cdc40f0 100755
> --- a/t/t3513-revert-submodule.sh
> +++ b/t/t3513-revert-submodule.sh
> @@ -11,11 +11,13 @@ test_description='revert can handle submodules'
>  # first so we can restore the work tree test setup after doing the checkout
>  # and revert.  We test here that the restored work tree content is identical
>  # to that at the beginning. The last revert is then tested by the framework.
> -git_revert () {
> +git_revert_before () {
>  	git status -su >expect &&
>  	ls -1pR * >>expect &&
> -	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
> -	git checkout "$1" &&
> +	tar cf "$TRASH_DIRECTORY/tmp.tar" *
> +}
> +
> +git_revert_after () {
>  	git revert HEAD &&
>  	rm -rf * &&
>  	tar xf "$TRASH_DIRECTORY/tmp.tar" &&
> @@ -26,6 +28,6 @@ git_revert () {
>  }
>  
>  KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
> -test_submodule_switch_func "git_revert"
> +test_submodule_switch_func "checkout \$arg" "git_revert_before" "git_revert_after"
>  
>  test_done
> diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
> index 6a7e801ca0..358a625124 100755
> --- a/t/t3906-stash-submodule.sh
> +++ b/t/t3906-stash-submodule.sh
> @@ -5,10 +5,12 @@ test_description='stash can handle submodules'
>  . ./test-lib.sh
>  . "$TEST_DIRECTORY"/lib-submodule-update.sh
>  
> -git_stash () {
> +git_stash_before () {
>  	git status -su >expect &&
> -	ls -1pR * >>expect &&
> -	git read-tree -u -m "$1" &&
> +	ls -1pR * >>expect
> +}
> +
> +git_stash_after () {
>  	git stash &&
>  	git status -su >actual &&
>  	ls -1pR * >>actual &&
> @@ -19,7 +21,7 @@ git_stash () {
>  KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES=1
>  KNOWN_FAILURE_CHERRY_PICK_SEES_EMPTY_COMMIT=1
>  KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
> -test_submodule_switch_func "git_stash"
> +test_submodule_switch_func "read-tree -u -m \$arg" "git_stash_before" "git_stash_after"
>  
>  setup_basic () {
>  	test_when_finished "rm -rf main sub" &&
> diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
> index b645e303a0..fe5ebeaa41 100755
> --- a/t/t4137-apply-submodule.sh
> +++ b/t/t4137-apply-submodule.sh
> @@ -5,16 +5,12 @@ test_description='git apply handling submodules'
>  . ./test-lib.sh
>  . "$TEST_DIRECTORY"/lib-submodule-update.sh
>  
> -apply_index () {
> -	git diff --ignore-submodules=dirty "..$1" | git apply --index -
> +create_diff () {
> +	git diff --ignore-submodules=dirty "..$1" >diff
>  }
>  
> -test_submodule_switch_func "apply_index"
> +test_submodule_switch_func "apply --index diff" "create_diff"
>  
> -apply_3way () {
> -	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
> -}
> -
> -test_submodule_switch_func "apply_3way"
> +test_submodule_switch_func "apply --3way diff" "create_diff"
>  
>  test_done
> diff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh
> index 1b179d5f45..5305280dfd 100755
> --- a/t/t4255-am-submodule.sh
> +++ b/t/t4255-am-submodule.sh
> @@ -5,18 +5,14 @@ test_description='git am handling submodules'
>  . ./test-lib.sh
>  . "$TEST_DIRECTORY"/lib-submodule-update.sh
>  
> -am () {
> -	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am -
> +create_patch () {
> +	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch
>  }
>  
> -test_submodule_switch_func "am"
> -
> -am_3way () {
> -	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am --3way -
> -}
> +test_submodule_switch_func "am patch" "create_patch"
>  
>  KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
> -test_submodule_switch_func "am_3way"
> +test_submodule_switch_func "am --3way patch" "create_patch"
>  
>  test_expect_success 'setup diff.submodule' '
>  	test_commit one &&
> diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
> index f911bf631e..3b37853537 100755
> --- a/t/t5572-pull-submodule.sh
> +++ b/t/t5572-pull-submodule.sh
> @@ -11,36 +11,16 @@ reset_branch_to_HEAD () {
>  	git branch --set-upstream-to="origin/$1" "$1"
>  }
>  
> -git_pull () {
> -	reset_branch_to_HEAD "$1" &&
> -	git pull
> -}
> -
>  # pulls without conflicts
> -test_submodule_switch_func "git_pull"
> +test_submodule_switch_func "pull" "reset_branch_to_HEAD"
>  
> -git_pull_ff () {
> -	reset_branch_to_HEAD "$1" &&
> -	git pull --ff
> -}
> +test_submodule_switch_func "pull --ff" "reset_branch_to_HEAD"
>  
> -test_submodule_switch_func "git_pull_ff"
> -
> -git_pull_ff_only () {
> -	reset_branch_to_HEAD "$1" &&
> -	git pull --ff-only
> -}
> -
> -test_submodule_switch_func "git_pull_ff_only"
> -
> -git_pull_noff () {
> -	reset_branch_to_HEAD "$1" &&
> -	git pull --no-ff
> -}
> +test_submodule_switch_func "pull --ff-only" "reset_branch_to_HEAD"
>  
>  KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
>  KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
> -test_submodule_switch_func "git_pull_noff"
> +test_submodule_switch_func "pull --no-ff" "reset_branch_to_HEAD"
>  
>  test_expect_success 'pull --recurse-submodule setup' '
>  	test_create_repo child &&
> diff --git a/t/t6041-bisect-submodule.sh b/t/t6041-bisect-submodule.sh
> index 0e0cdf638d..b99e81d55d 100755
> --- a/t/t6041-bisect-submodule.sh
> +++ b/t/t6041-bisect-submodule.sh
> @@ -5,12 +5,14 @@ test_description='bisect can handle submodules'
>  . ./test-lib.sh
>  . "$TEST_DIRECTORY"/lib-submodule-update.sh
>  
> -git_bisect () {
> +git_bisect_before () {
>  	git status -su >expect &&
>  	ls -1pR * >>expect &&
>  	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
> -	GOOD=$(git rev-parse --verify HEAD) &&
> -	git checkout "$1" &&
> +	GOOD=$(git rev-parse --verify HEAD)
> +}
> +
> +git_bisect_after () {
>  	echo "foo" >bar &&
>  	git add bar &&
>  	git commit -m "bisect bad" &&
> @@ -27,6 +29,6 @@ git_bisect () {
>  	git bisect bad $BAD
>  }
>  
> -test_submodule_switch_func "git_bisect"
> +test_submodule_switch_func "checkout \$arg" "git_bisect_before" "git_bisect_after"
>  
>  test_done

[PATCH v4 0/4] t: replace incorrect test_must_fail usage (part 5)

[Denton Liu]

Hi all,

This revision of the series just improves on some of the documentation
and add addresses one of Junio's comments. Aside from that, the code
remains the same.


The overall scope of these patches is to replace inappropriate uses of
test_must_fail. IOW, we should only allow test_must_fail to run on `git`
and `test-tool`. Ultimately, we will conclude by making test_must_fail
error out on non-git commands. An advance view of the final series can
be found here[1].

This is the fifth part. It focuses on lib-submodule-update.sh and tests
that make use of it.

The first part can be found here[2]. The second part can be found
here[3]. The third part can be found here[4]. The fourth part can be
found here[5].

Changes since v1.2:

* In "lib-submodule-update: pass OVERWRITING_FAIL", use if-then return
  to reduce the amount of code churn

Changes since v2:

* Replace the OVERWRITING_FAIL approach with callback functions as
  suggested by Peff[6]

Changes since v3:

* Simply handling of empty $before and $after

* Add more comments on the usage of the helper functions

[1]: (may be rebased at any time) https://github.com/Denton-L/git/tree/ready/cleanup-test-must-fail2
[2]: https://lore.kernel.org/git/cover.1576583819.git.liu.denton@gmail.com/
[3]: https://lore.kernel.org/git/cover.1577454401.git.liu.denton@gmail.com/
[4]: https://lore.kernel.org/git/cover.1585209554.git.liu.denton@gmail.com/
[5]: https://lore.kernel.org/git/cover.1587372771.git.liu.denton@gmail.com/
[6]: https://lore.kernel.org/git/20200521182928.GA1308647@coredump.intra.peff.net/

Denton Liu (4):
  lib-submodule-update: add space after function name
  lib-submodule-update: consolidate --recurse-submodules
  lib-submodule-update: prepend "git" to $command
  lib-submodule-update: use callbacks in test_submodule_switch_common()

 t/lib-submodule-update.sh        | 135 +++++++++++++++++++++++--------
 t/t1013-read-tree-submodule.sh   |   4 +-
 t/t2013-checkout-submodule.sh    |   4 +-
 t/t3426-rebase-submodule.sh      |  10 +--
 t/t3512-cherry-pick-submodule.sh |   2 +-
 t/t3513-revert-submodule.sh      |  10 ++-
 t/t3906-stash-submodule.sh       |  10 ++-
 t/t4137-apply-submodule.sh       |  12 +--
 t/t4255-am-submodule.sh          |  12 +--
 t/t5572-pull-submodule.sh        |  28 +------
 t/t6041-bisect-submodule.sh      |  10 ++-
 t/t7112-reset-submodule.sh       |   6 +-
 t/t7613-merge-submodule.sh       |   8 +-
 13 files changed, 148 insertions(+), 103 deletions(-)

Range-diff against v3:
1:  ba2f642e0f = 1:  ba2f642e0f lib-submodule-update: add space after function name
2:  16d0a3eb9a = 2:  16d0a3eb9a lib-submodule-update: consolidate --recurse-submodules
3:  09446be5b9 = 3:  09446be5b9 lib-submodule-update: prepend "git" to $command
4:  74e6086da4 ! 4:  35d07117e6 lib-submodule-update: use callbacks in test_submodule_switch_common()
    @@ Commit message
         If the command requires a filename argument, specify it as `\$arg` since
         that variable will be set and the whole $command string will be eval'd.
         Unfortunately, there is no way to get rid of the eval as some of the
    -    commands are passed (such as the `git pull` tests) require that no
    +    commands that are passed (such as the `git pull` tests) require that no
         additional arguments are passed so we must have some mechanism for the
         caller to specify whether or not it wants the filename argument.
     
    @@ Commit message
         Finally, as an added bonus, `test_must_fail` will only run on $command
         which is guaranteed to be a git command.
     
    -    An alternate design was considered where the $OVERWRITING_FAIL is set
    -    from the test_submodule_switch_common() function and passed to the
    -    helper function. This approach was considered too difficult to
    -    understand due to the fact that using a signalling magic environment
    -    variable might be too indirect.
    +    An alternate design was considered where $OVERWRITING_FAIL is set from
    +    test_submodule_switch_common() and exposed to the helper function. This
    +    approach was considered too difficult to understand due to the fact that
    +    using a signalling magic environment variable might be too indirect.
     
      ## t/lib-submodule-update.sh ##
     @@ t/lib-submodule-update.sh: test_submodule_content () {
    + # Removing a submodule containing a .git directory must fail even when forced
    + # to protect the history!
    + #
    ++# $1: The git command to be eval'd and tested. The submodule being operated on
    ++# will be available as $arg.
    ++#
    ++# $2: The function that will run before the git command. It will be passed the
    ++# submodule being operated on as the only argument. This argument is optional.
    ++#
    ++# $3: The function that will run after $1. It will be passed the submodule
    ++# being operated on as the only argument. This argument is optional. It will
    ++# not be run when testing a case where the git command is expected to fail.
    + 
      # Internal function; use test_submodule_switch_func(), test_submodule_switch(),
      # or test_submodule_forced_switch() instead.
      test_submodule_switch_common () {
     -	command="$1"
    -+	command="$1" # should be a git command
    -+	before="$2"
    -+	after="$3"
    -+
    -+	if test -z "$before"
    -+	then
    -+		before=true
    -+	fi
    -+
    -+	if test -z "$after"
    -+	then
    -+		after=true
    -+	fi
    ++	command="$1" before="${2:-true}" after="${3:-true}"
     +
      	######################### Appearing submodule #########################
      	# Switching to a commit letting a submodule appear creates empty dir ...
    @@ t/lib-submodule-update.sh: test_submodule_switch_common () {
      			test_dir_is_empty sub1 &&
      			git submodule update --init --recursive &&
     @@ t/lib-submodule-update.sh: test_submodule_switch_common () {
    + # conditions, set the appropriate KNOWN_FAILURE_* variable used in the tests
    + # below to 1.
    + #
    +-# Use as follows:
    ++# $1: The git command to be eval'd and tested. The submodule being operated on
    ++# will be available as $arg. Do not include the leading "git".
    + #
    +-# my_func () {
    ++# $2: The function that will run before the git command. It will be passed the
    ++# submodule being operated on as the only argument. This argument is optional.
    ++#
    ++# $3: The function that will run after $1. It will be passed the submodule
    ++# being operated on as the only argument. This argument is optional. It will
    ++# not be run when testing a case where the git command is expected to fail.
    ++#
    ++# The following example uses `git some-command` as an example command to be
    ++# tested. It updates the worktree and index to match a target, but not any
    ++# submodule directories.
    ++#
    ++# my_func_before () {
    + #   target=$1
    +-#   # Do something here that updates the worktree and index to match target,
    +-#   # but not any submodule directories.
    ++#   # Prepare for git some-command to be run
      # }
    - # test_submodule_switch_func "my_func"
    +-# test_submodule_switch_func "my_func"
    ++# my_func_after () {
    ++#   target=$1
    ++#   # Check the state after git some-command is run
    ++# }
    ++# test_submodule_switch_func "some-command \$arg" "my_func_before" "my_func_after"
      test_submodule_switch_func () {
     -	command="$1"
     -	test_submodule_switch_common "$command"
-- 
2.27.0.132.g321788e831

[PATCH v4 1/4] lib-submodule-update: add space after function name

[Denton Liu]

In the shell scripts in this codebase, the usual style is to include a
space between the function name and the (). Add these missing spaces to
conform to the usual style of the code.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 1dd17fc03e..122554acd6 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -183,7 +183,7 @@ test_git_directory_is_unchanged () {
 	)
 }
 
-test_git_directory_exists() {
+test_git_directory_exists () {
 	test -e ".git/modules/$1" &&
 	if test -f sub1/.git
 	then
@@ -309,7 +309,7 @@ test_submodule_content () {
 
 # Internal function; use test_submodule_switch() or
 # test_submodule_forced_switch() instead.
-test_submodule_switch_common() {
+test_submodule_switch_common () {
 	command="$1"
 	######################### Appearing submodule #########################
 	# Switching to a commit letting a submodule appear creates empty dir ...
@@ -629,7 +629,7 @@ test_submodule_forced_switch () {
 
 # Internal function; use test_submodule_switch_recursing_with_args() or
 # test_submodule_forced_switch_recursing_with_args() instead.
-test_submodule_recursing_with_args_common() {
+test_submodule_recursing_with_args_common () {
 	command="$1"
 
 	######################### Appearing submodule #########################
-- 
2.27.0.132.g321788e831

[PATCH v4 2/4] lib-submodule-update: consolidate --recurse-submodules

[Denton Liu]

Both test_submodule_switch_recursing_with_args() and
test_submodule_forced_switch_recursing_with_args() call the internal
function test_submodule_recursing_with_args_common() with the final
argument of `--recurse-submodules`. Consolidate this duplication by
appending the argument in test_submodule_recursing_with_args_common().

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 122554acd6..bb36287803 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -630,7 +630,7 @@ test_submodule_forced_switch () {
 # Internal function; use test_submodule_switch_recursing_with_args() or
 # test_submodule_forced_switch_recursing_with_args() instead.
 test_submodule_recursing_with_args_common () {
-	command="$1"
+	command="$1 --recurse-submodules"
 
 	######################### Appearing submodule #########################
 	# Switching to a commit letting a submodule appear checks it out ...
@@ -809,7 +809,7 @@ test_submodule_recursing_with_args_common () {
 # test_submodule_switch_recursing_with_args "$GIT_COMMAND"
 test_submodule_switch_recursing_with_args () {
 	cmd_args="$1"
-	command="git $cmd_args --recurse-submodules"
+	command="git $cmd_args"
 	test_submodule_recursing_with_args_common "$command"
 
 	RESULTDS=success
@@ -927,7 +927,7 @@ test_submodule_switch_recursing_with_args () {
 # away local changes in the superproject is allowed.
 test_submodule_forced_switch_recursing_with_args () {
 	cmd_args="$1"
-	command="git $cmd_args --recurse-submodules"
+	command="git $cmd_args"
 	test_submodule_recursing_with_args_common "$command"
 
 	RESULT=success
-- 
2.27.0.132.g321788e831

[PATCH v4 3/4] lib-submodule-update: prepend "git" to $command

[Denton Liu]

Since all invocations of test_submodule_forced_switch() are git
commands, automatically prepend "git" before invoking
test_submodule_switch_common().

Similarly, many invocations of test_submodule_switch() are also git
commands so automatically prepend "git" before invoking
test_submodule_switch_common() as well.

Finally, for invocations of test_submodule_switch() that invoke a custom
function, rename the old function to test_submodule_switch_func().

This is necessary because in a future commit, we will be adding some
logic that needs to distinguish between an invocation of a plain git
comamnd and an invocation of a test helper function.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh        | 14 +++++++++-----
 t/t1013-read-tree-submodule.sh   |  4 ++--
 t/t2013-checkout-submodule.sh    |  4 ++--
 t/t3426-rebase-submodule.sh      |  4 ++--
 t/t3512-cherry-pick-submodule.sh |  2 +-
 t/t3513-revert-submodule.sh      |  2 +-
 t/t3906-stash-submodule.sh       |  2 +-
 t/t4137-apply-submodule.sh       |  4 ++--
 t/t4255-am-submodule.sh          |  4 ++--
 t/t5572-pull-submodule.sh        |  8 ++++----
 t/t6041-bisect-submodule.sh      |  2 +-
 t/t7112-reset-submodule.sh       |  6 +++---
 t/t7613-merge-submodule.sh       |  8 ++++----
 13 files changed, 34 insertions(+), 30 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index bb36287803..7c3ba1be00 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -307,8 +307,8 @@ test_submodule_content () {
 # to protect the history!
 #
 
-# Internal function; use test_submodule_switch() or
-# test_submodule_forced_switch() instead.
+# Internal function; use test_submodule_switch_func(), test_submodule_switch(),
+# or test_submodule_forced_switch() instead.
 test_submodule_switch_common () {
 	command="$1"
 	######################### Appearing submodule #########################
@@ -566,8 +566,8 @@ test_submodule_switch_common () {
 #   # Do something here that updates the worktree and index to match target,
 #   # but not any submodule directories.
 # }
-# test_submodule_switch "my_func"
-test_submodule_switch () {
+# test_submodule_switch_func "my_func"
+test_submodule_switch_func () {
 	command="$1"
 	test_submodule_switch_common "$command"
 
@@ -587,12 +587,16 @@ test_submodule_switch () {
 	'
 }
 
+test_submodule_switch () {
+	test_submodule_switch_func "git $1"
+}
+
 # Same as test_submodule_switch(), except that throwing away local changes in
 # the superproject is allowed.
 test_submodule_forced_switch () {
 	command="$1"
 	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
-	test_submodule_switch_common "$command"
+	test_submodule_switch_common "git $command"
 
 	# When forced, a file in the superproject does not prevent creating a
 	# submodule of the same name.
diff --git a/t/t1013-read-tree-submodule.sh b/t/t1013-read-tree-submodule.sh
index 91a6fafcb4..b6df7444c0 100755
--- a/t/t1013-read-tree-submodule.sh
+++ b/t/t1013-read-tree-submodule.sh
@@ -12,8 +12,8 @@ test_submodule_switch_recursing_with_args "read-tree -u -m"
 
 test_submodule_forced_switch_recursing_with_args "read-tree -u --reset"
 
-test_submodule_switch "git read-tree -u -m"
+test_submodule_switch "read-tree -u -m"
 
-test_submodule_forced_switch "git read-tree -u --reset"
+test_submodule_forced_switch "read-tree -u --reset"
 
 test_done
diff --git a/t/t2013-checkout-submodule.sh b/t/t2013-checkout-submodule.sh
index 8f86b5f4b2..b2bdd1fcb4 100755
--- a/t/t2013-checkout-submodule.sh
+++ b/t/t2013-checkout-submodule.sh
@@ -68,8 +68,8 @@ test_submodule_switch_recursing_with_args "checkout"
 
 test_submodule_forced_switch_recursing_with_args "checkout -f"
 
-test_submodule_switch "git checkout"
+test_submodule_switch "checkout"
 
-test_submodule_forced_switch "git checkout -f"
+test_submodule_forced_switch "checkout -f"
 
 test_done
diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
index a2bba04ba9..788605ccc0 100755
--- a/t/t3426-rebase-submodule.sh
+++ b/t/t3426-rebase-submodule.sh
@@ -20,7 +20,7 @@ git_rebase () {
 	git rebase "$1"
 }
 
-test_submodule_switch "git_rebase"
+test_submodule_switch_func "git_rebase"
 
 git_rebase_interactive () {
 	git status -su >expect &&
@@ -38,7 +38,7 @@ git_rebase_interactive () {
 	git rebase -i "$1"
 }
 
-test_submodule_switch "git_rebase_interactive"
+test_submodule_switch_func "git_rebase_interactive"
 
 test_expect_success 'rebase interactive ignores modified submodules' '
 	test_when_finished "rm -rf super sub" &&
diff --git a/t/t3512-cherry-pick-submodule.sh b/t/t3512-cherry-pick-submodule.sh
index bd78287841..6ece1d8573 100755
--- a/t/t3512-cherry-pick-submodule.sh
+++ b/t/t3512-cherry-pick-submodule.sh
@@ -7,7 +7,7 @@ test_description='cherry-pick can handle submodules'
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "git cherry-pick"
+test_submodule_switch "cherry-pick"
 
 test_expect_success 'unrelated submodule/file conflict is ignored' '
 	test_create_repo sub &&
diff --git a/t/t3513-revert-submodule.sh b/t/t3513-revert-submodule.sh
index 5e39fcdb66..95a7f64471 100755
--- a/t/t3513-revert-submodule.sh
+++ b/t/t3513-revert-submodule.sh
@@ -26,6 +26,6 @@ git_revert () {
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
-test_submodule_switch "git_revert"
+test_submodule_switch_func "git_revert"
 
 test_done
diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
index b93d1d74da..6a7e801ca0 100755
--- a/t/t3906-stash-submodule.sh
+++ b/t/t3906-stash-submodule.sh
@@ -19,7 +19,7 @@ git_stash () {
 KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES=1
 KNOWN_FAILURE_CHERRY_PICK_SEES_EMPTY_COMMIT=1
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
-test_submodule_switch "git_stash"
+test_submodule_switch_func "git_stash"
 
 setup_basic () {
 	test_when_finished "rm -rf main sub" &&
diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
index a9bd40a6d0..b645e303a0 100755
--- a/t/t4137-apply-submodule.sh
+++ b/t/t4137-apply-submodule.sh
@@ -9,12 +9,12 @@ apply_index () {
 	git diff --ignore-submodules=dirty "..$1" | git apply --index -
 }
 
-test_submodule_switch "apply_index"
+test_submodule_switch_func "apply_index"
 
 apply_3way () {
 	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
 }
 
-test_submodule_switch "apply_3way"
+test_submodule_switch_func "apply_3way"
 
 test_done
diff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh
index 0ba8194403..1b179d5f45 100755
--- a/t/t4255-am-submodule.sh
+++ b/t/t4255-am-submodule.sh
@@ -9,14 +9,14 @@ am () {
 	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am -
 }
 
-test_submodule_switch "am"
+test_submodule_switch_func "am"
 
 am_3way () {
 	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am --3way -
 }
 
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "am_3way"
+test_submodule_switch_func "am_3way"
 
 test_expect_success 'setup diff.submodule' '
 	test_commit one &&
diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
index f916729a12..f911bf631e 100755
--- a/t/t5572-pull-submodule.sh
+++ b/t/t5572-pull-submodule.sh
@@ -17,21 +17,21 @@ git_pull () {
 }
 
 # pulls without conflicts
-test_submodule_switch "git_pull"
+test_submodule_switch_func "git_pull"
 
 git_pull_ff () {
 	reset_branch_to_HEAD "$1" &&
 	git pull --ff
 }
 
-test_submodule_switch "git_pull_ff"
+test_submodule_switch_func "git_pull_ff"
 
 git_pull_ff_only () {
 	reset_branch_to_HEAD "$1" &&
 	git pull --ff-only
 }
 
-test_submodule_switch "git_pull_ff_only"
+test_submodule_switch_func "git_pull_ff_only"
 
 git_pull_noff () {
 	reset_branch_to_HEAD "$1" &&
@@ -40,7 +40,7 @@ git_pull_noff () {
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "git_pull_noff"
+test_submodule_switch_func "git_pull_noff"
 
 test_expect_success 'pull --recurse-submodule setup' '
 	test_create_repo child &&
diff --git a/t/t6041-bisect-submodule.sh b/t/t6041-bisect-submodule.sh
index 62b8a2e7bb..0e0cdf638d 100755
--- a/t/t6041-bisect-submodule.sh
+++ b/t/t6041-bisect-submodule.sh
@@ -27,6 +27,6 @@ git_bisect () {
 	git bisect bad $BAD
 }
 
-test_submodule_switch "git_bisect"
+test_submodule_switch_func "git_bisect"
 
 test_done
diff --git a/t/t7112-reset-submodule.sh b/t/t7112-reset-submodule.sh
index a1cb9ff858..8741b665c9 100755
--- a/t/t7112-reset-submodule.sh
+++ b/t/t7112-reset-submodule.sh
@@ -13,10 +13,10 @@ test_submodule_switch_recursing_with_args "reset --keep"
 
 test_submodule_forced_switch_recursing_with_args "reset --hard"
 
-test_submodule_switch "git reset --keep"
+test_submodule_switch "reset --keep"
 
-test_submodule_switch "git reset --merge"
+test_submodule_switch "reset --merge"
 
-test_submodule_forced_switch "git reset --hard"
+test_submodule_forced_switch "reset --hard"
 
 test_done
diff --git a/t/t7613-merge-submodule.sh b/t/t7613-merge-submodule.sh
index d1e9fcc781..04bf4be7d7 100755
--- a/t/t7613-merge-submodule.sh
+++ b/t/t7613-merge-submodule.sh
@@ -6,14 +6,14 @@ test_description='merge can handle submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 # merges without conflicts
-test_submodule_switch "git merge"
+test_submodule_switch "merge"
 
-test_submodule_switch "git merge --ff"
+test_submodule_switch "merge --ff"
 
-test_submodule_switch "git merge --ff-only"
+test_submodule_switch "merge --ff-only"
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "git merge --no-ff"
+test_submodule_switch "merge --no-ff"
 
 test_done
-- 
2.27.0.132.g321788e831

[PATCH v4 4/4] lib-submodule-update: use callbacks in test_submodule_switch_common()

[Denton Liu]

When we run a test helper function in test_submodule_switch_common(), we
sometimes specify a whole helper function as the $command. When we do
this, in some test cases, we just mark the whole function with
`test_must_fail`. However, it's possible that the helper function might
fail earlier or later than expected due to an introduced bug. If this
happens, then the test case will still report as passing but it should
really be marked as failing since it didn't actually display the
intended behaviour.

Instead of invoking $command as one monolithic helper function, break it
up into three parts:

	1. $command which is always a git command.
	2. $before which is a callback function that runs just prior to
	   $command.
	3. $after which is a callback function that runs just after
	   $command.

If the command requires a filename argument, specify it as `\$arg` since
that variable will be set and the whole $command string will be eval'd.
Unfortunately, there is no way to get rid of the eval as some of the
commands that are passed (such as the `git pull` tests) require that no
additional arguments are passed so we must have some mechanism for the
caller to specify whether or not it wants the filename argument.

The $before and $after callback functions will be passed the filename as
the first arg. These callback functions are optional and, if missing,
will be replaced with `true`. Also, in the case where we have a
`test_must_fail` test, $after will not be executed, similar to how the
helper functions currently behave when the git command fails and exits
the &&-chain.

Finally, as an added bonus, `test_must_fail` will only run on $command
which is guaranteed to be a git command.

An alternate design was considered where $OVERWRITING_FAIL is set from
test_submodule_switch_common() and exposed to the helper function. This
approach was considered too difficult to understand due to the fact that
using a signalling magic environment variable might be too indirect.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh   | 117 ++++++++++++++++++++++++++++--------
 t/t3426-rebase-submodule.sh |  10 ++-
 t/t3513-revert-submodule.sh |  10 +--
 t/t3906-stash-submodule.sh  |  10 +--
 t/t4137-apply-submodule.sh  |  12 ++--
 t/t4255-am-submodule.sh     |  12 ++--
 t/t5572-pull-submodule.sh   |  28 ++-------
 t/t6041-bisect-submodule.sh |  10 +--
 8 files changed, 125 insertions(+), 84 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 7c3ba1be00..3401a59ad8 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -306,11 +306,21 @@ test_submodule_content () {
 # Removing a submodule containing a .git directory must fail even when forced
 # to protect the history!
 #
+# $1: The git command to be eval'd and tested. The submodule being operated on
+# will be available as $arg.
+#
+# $2: The function that will run before the git command. It will be passed the
+# submodule being operated on as the only argument. This argument is optional.
+#
+# $3: The function that will run after $1. It will be passed the submodule
+# being operated on as the only argument. This argument is optional. It will
+# not be run when testing a case where the git command is expected to fail.
 
 # Internal function; use test_submodule_switch_func(), test_submodule_switch(),
 # or test_submodule_forced_switch() instead.
 test_submodule_switch_common () {
-	command="$1"
+	command="$1" before="${2:-true}" after="${3:-true}"
+
 	######################### Appearing submodule #########################
 	# Switching to a commit letting a submodule appear creates empty dir ...
 	if test "$KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES" = 1
@@ -326,7 +336,10 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t add_sub1 origin/add_sub1 &&
-			$command add_sub1 &&
+			arg=add_sub1 &&
+			$before "$arg" &&
+			eval $command &&
+			$after "$arg" &&
 			test_superproject_content origin/add_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -341,7 +354,10 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			mkdir sub1 &&
 			git branch -t add_sub1 origin/add_sub1 &&
-			$command add_sub1 &&
+			arg=add_sub1 &&
+			$before "$arg" &&
+			eval $command &&
+			$after "$arg" &&
 			test_superproject_content origin/add_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -356,7 +372,10 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&
-			$command replace_file_with_sub1 &&
+			arg=replace_file_with_sub1 &&
+			$before "$arg" &&
+			eval $command &&
+			$after "$arg" &&
 			test_superproject_content origin/replace_file_with_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -380,7 +399,10 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&
-			$command replace_directory_with_sub1 &&
+			arg=replace_directory_with_sub1  &&
+			$before "$arg" &&
+			eval $command &&
+			$after "$arg" &&
 			test_superproject_content origin/replace_directory_with_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -402,7 +424,10 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t remove_sub1 origin/remove_sub1 &&
-			$command remove_sub1 &&
+			arg=remove_sub1 &&
+			$before "$arg" &&
+			eval $command &&
+			$after "$arg" &&
 			test_superproject_content origin/remove_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -415,7 +440,10 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t remove_sub1 origin/remove_sub1 &&
 			replace_gitfile_with_git_dir sub1 &&
-			$command remove_sub1 &&
+			arg=remove_sub1 &&
+			$before "$arg" &&
+			eval $command &&
+			$after "$arg" &&
 			test_superproject_content origin/remove_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -443,7 +471,9 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
-			test_must_fail $command replace_sub1_with_directory &&
+			arg=replace_sub1_with_directory &&
+			$before "$arg" &&
+			eval test_must_fail $command &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -456,7 +486,9 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_directory &&
+			arg=replace_sub1_with_directory &&
+			$before "$arg" &&
+			eval test_must_fail $command &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -470,7 +502,9 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
-			test_must_fail $command replace_sub1_with_file &&
+			arg=replace_sub1_with_file &&
+			$before "$arg" &&
+			eval test_must_fail $command &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -484,7 +518,9 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_file &&
+			arg=replace_sub1_with_file &&
+			$before "$arg" &&
+			eval test_must_fail $command &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -508,7 +544,10 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t modify_sub1 origin/modify_sub1 &&
-			$command modify_sub1 &&
+			arg=modify_sub1 &&
+			$before "$arg" &&
+			eval $command &&
+			$after "$arg" &&
 			test_superproject_content origin/modify_sub1 &&
 			test_submodule_content sub1 origin/add_sub1 &&
 			git submodule update &&
@@ -523,7 +562,10 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t invalid_sub1 origin/invalid_sub1 &&
-			$command invalid_sub1 &&
+			arg=invalid_sub1 &&
+			$before "$arg" &&
+			eval $command &&
+			$after "$arg" &&
 			test_superproject_content origin/invalid_sub1 &&
 			test_submodule_content sub1 origin/add_sub1 &&
 			test_must_fail git submodule update &&
@@ -538,7 +580,10 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t valid_sub1 origin/valid_sub1 &&
-			$command valid_sub1 &&
+			arg=valid_sub1 &&
+			$before "$arg" &&
+			eval $command &&
+			$after "$arg" &&
 			test_superproject_content origin/valid_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -559,17 +604,34 @@ test_submodule_switch_common () {
 # conditions, set the appropriate KNOWN_FAILURE_* variable used in the tests
 # below to 1.
 #
-# Use as follows:
+# $1: The git command to be eval'd and tested. The submodule being operated on
+# will be available as $arg. Do not include the leading "git".
 #
-# my_func () {
+# $2: The function that will run before the git command. It will be passed the
+# submodule being operated on as the only argument. This argument is optional.
+#
+# $3: The function that will run after $1. It will be passed the submodule
+# being operated on as the only argument. This argument is optional. It will
+# not be run when testing a case where the git command is expected to fail.
+#
+# The following example uses `git some-command` as an example command to be
+# tested. It updates the worktree and index to match a target, but not any
+# submodule directories.
+#
+# my_func_before () {
 #   target=$1
-#   # Do something here that updates the worktree and index to match target,
-#   # but not any submodule directories.
+#   # Prepare for git some-command to be run
 # }
-# test_submodule_switch_func "my_func"
+# my_func_after () {
+#   target=$1
+#   # Check the state after git some-command is run
+# }
+# test_submodule_switch_func "some-command \$arg" "my_func_before" "my_func_after"
 test_submodule_switch_func () {
-	command="$1"
-	test_submodule_switch_common "$command"
+	command="git $1"
+	before="$2"
+	after="$3"
+	test_submodule_switch_common "$command" "$before" "$after"
 
 	# An empty directory does not prevent the creation of a submodule of
 	# the same name, but a file does.
@@ -580,7 +642,9 @@ test_submodule_switch_func () {
 			cd submodule_update &&
 			git branch -t add_sub1 origin/add_sub1 &&
 			>sub1 &&
-			test_must_fail $command add_sub1 &&
+			arg=add_sub1 &&
+			$before "$arg" &&
+			eval test_must_fail $command &&
 			test_superproject_content origin/no_submodule &&
 			test_must_be_empty sub1
 		)
@@ -588,15 +652,15 @@ test_submodule_switch_func () {
 }
 
 test_submodule_switch () {
-	test_submodule_switch_func "git $1"
+	test_submodule_switch_func "$1 \$arg"
 }
 
 # Same as test_submodule_switch(), except that throwing away local changes in
 # the superproject is allowed.
 test_submodule_forced_switch () {
-	command="$1"
+	command="git $1 \$arg"
 	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
-	test_submodule_switch_common "git $command"
+	test_submodule_switch_common "$command"
 
 	# When forced, a file in the superproject does not prevent creating a
 	# submodule of the same name.
@@ -607,7 +671,8 @@ test_submodule_forced_switch () {
 			cd submodule_update &&
 			git branch -t add_sub1 origin/add_sub1 &&
 			>sub1 &&
-			$command add_sub1 &&
+			arg=add_sub1 &&
+			eval $command &&
 			test_superproject_content origin/add_sub1 &&
 			test_dir_is_empty sub1
 		)
diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
index 788605ccc0..d31e6487bd 100755
--- a/t/t3426-rebase-submodule.sh
+++ b/t/t3426-rebase-submodule.sh
@@ -16,11 +16,10 @@ git_rebase () {
 	git revert HEAD &&
 	git status -su >actual &&
 	ls -1pR * >>actual &&
-	test_cmp expect actual &&
-	git rebase "$1"
+	test_cmp expect actual
 }
 
-test_submodule_switch_func "git_rebase"
+test_submodule_switch_func "rebase \$arg" "git_rebase"
 
 git_rebase_interactive () {
 	git status -su >expect &&
@@ -34,11 +33,10 @@ git_rebase_interactive () {
 	ls -1pR * >>actual &&
 	test_cmp expect actual &&
 	set_fake_editor &&
-	echo "fake-editor.sh" >.git/info/exclude &&
-	git rebase -i "$1"
+	echo "fake-editor.sh" >.git/info/exclude
 }
 
-test_submodule_switch_func "git_rebase_interactive"
+test_submodule_switch_func "rebase -i \$arg" "git_rebase_interactive"
 
 test_expect_success 'rebase interactive ignores modified submodules' '
 	test_when_finished "rm -rf super sub" &&
diff --git a/t/t3513-revert-submodule.sh b/t/t3513-revert-submodule.sh
index 95a7f64471..994cdc40f0 100755
--- a/t/t3513-revert-submodule.sh
+++ b/t/t3513-revert-submodule.sh
@@ -11,11 +11,13 @@ test_description='revert can handle submodules'
 # first so we can restore the work tree test setup after doing the checkout
 # and revert.  We test here that the restored work tree content is identical
 # to that at the beginning. The last revert is then tested by the framework.
-git_revert () {
+git_revert_before () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
-	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
-	git checkout "$1" &&
+	tar cf "$TRASH_DIRECTORY/tmp.tar" *
+}
+
+git_revert_after () {
 	git revert HEAD &&
 	rm -rf * &&
 	tar xf "$TRASH_DIRECTORY/tmp.tar" &&
@@ -26,6 +28,6 @@ git_revert () {
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
-test_submodule_switch_func "git_revert"
+test_submodule_switch_func "checkout \$arg" "git_revert_before" "git_revert_after"
 
 test_done
diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
index 6a7e801ca0..358a625124 100755
--- a/t/t3906-stash-submodule.sh
+++ b/t/t3906-stash-submodule.sh
@@ -5,10 +5,12 @@ test_description='stash can handle submodules'
 . ./test-lib.sh
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
-git_stash () {
+git_stash_before () {
 	git status -su >expect &&
-	ls -1pR * >>expect &&
-	git read-tree -u -m "$1" &&
+	ls -1pR * >>expect
+}
+
+git_stash_after () {
 	git stash &&
 	git status -su >actual &&
 	ls -1pR * >>actual &&
@@ -19,7 +21,7 @@ git_stash () {
 KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES=1
 KNOWN_FAILURE_CHERRY_PICK_SEES_EMPTY_COMMIT=1
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
-test_submodule_switch_func "git_stash"
+test_submodule_switch_func "read-tree -u -m \$arg" "git_stash_before" "git_stash_after"
 
 setup_basic () {
 	test_when_finished "rm -rf main sub" &&
diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
index b645e303a0..fe5ebeaa41 100755
--- a/t/t4137-apply-submodule.sh
+++ b/t/t4137-apply-submodule.sh
@@ -5,16 +5,12 @@ test_description='git apply handling submodules'
 . ./test-lib.sh
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
-apply_index () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --index -
+create_diff () {
+	git diff --ignore-submodules=dirty "..$1" >diff
 }
 
-test_submodule_switch_func "apply_index"
+test_submodule_switch_func "apply --index diff" "create_diff"
 
-apply_3way () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
-}
-
-test_submodule_switch_func "apply_3way"
+test_submodule_switch_func "apply --3way diff" "create_diff"
 
 test_done
diff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh
index 1b179d5f45..5305280dfd 100755
--- a/t/t4255-am-submodule.sh
+++ b/t/t4255-am-submodule.sh
@@ -5,18 +5,14 @@ test_description='git am handling submodules'
 . ./test-lib.sh
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
-am () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am -
+create_patch () {
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch
 }
 
-test_submodule_switch_func "am"
-
-am_3way () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am --3way -
-}
+test_submodule_switch_func "am patch" "create_patch"
 
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch_func "am_3way"
+test_submodule_switch_func "am --3way patch" "create_patch"
 
 test_expect_success 'setup diff.submodule' '
 	test_commit one &&
diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
index f911bf631e..3b37853537 100755
--- a/t/t5572-pull-submodule.sh
+++ b/t/t5572-pull-submodule.sh
@@ -11,36 +11,16 @@ reset_branch_to_HEAD () {
 	git branch --set-upstream-to="origin/$1" "$1"
 }
 
-git_pull () {
-	reset_branch_to_HEAD "$1" &&
-	git pull
-}
-
 # pulls without conflicts
-test_submodule_switch_func "git_pull"
+test_submodule_switch_func "pull" "reset_branch_to_HEAD"
 
-git_pull_ff () {
-	reset_branch_to_HEAD "$1" &&
-	git pull --ff
-}
+test_submodule_switch_func "pull --ff" "reset_branch_to_HEAD"
 
-test_submodule_switch_func "git_pull_ff"
-
-git_pull_ff_only () {
-	reset_branch_to_HEAD "$1" &&
-	git pull --ff-only
-}
-
-test_submodule_switch_func "git_pull_ff_only"
-
-git_pull_noff () {
-	reset_branch_to_HEAD "$1" &&
-	git pull --no-ff
-}
+test_submodule_switch_func "pull --ff-only" "reset_branch_to_HEAD"
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch_func "git_pull_noff"
+test_submodule_switch_func "pull --no-ff" "reset_branch_to_HEAD"
 
 test_expect_success 'pull --recurse-submodule setup' '
 	test_create_repo child &&
diff --git a/t/t6041-bisect-submodule.sh b/t/t6041-bisect-submodule.sh
index 0e0cdf638d..b99e81d55d 100755
--- a/t/t6041-bisect-submodule.sh
+++ b/t/t6041-bisect-submodule.sh
@@ -5,12 +5,14 @@ test_description='bisect can handle submodules'
 . ./test-lib.sh
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
-git_bisect () {
+git_bisect_before () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
-	GOOD=$(git rev-parse --verify HEAD) &&
-	git checkout "$1" &&
+	GOOD=$(git rev-parse --verify HEAD)
+}
+
+git_bisect_after () {
 	echo "foo" >bar &&
 	git add bar &&
 	git commit -m "bisect bad" &&
@@ -27,6 +29,6 @@ git_bisect () {
 	git bisect bad $BAD
 }
 
-test_submodule_switch_func "git_bisect"
+test_submodule_switch_func "checkout \$arg" "git_bisect_before" "git_bisect_after"
 
 test_done
-- 
2.27.0.132.g321788e831

Re: [PATCH v4 4/4] lib-submodule-update: use callbacks in test_submodule_switch_common()

[Jeff King]

On Thu, Jun 18, 2020 at 04:49:14AM -0400, Denton Liu wrote:

> Instead of invoking $command as one monolithic helper function, break it
> up into three parts:
> 
> 	1. $command which is always a git command.
> 	2. $before which is a callback function that runs just prior to
> 	   $command.
> 	3. $after which is a callback function that runs just after
> 	   $command.

Thanks for reworking it in this way. I do think it's less confusing than
the original.

> If the command requires a filename argument, specify it as `\$arg` since
> that variable will be set and the whole $command string will be eval'd.
> Unfortunately, there is no way to get rid of the eval as some of the
> commands that are passed (such as the `git pull` tests) require that no
> additional arguments are passed so we must have some mechanism for the
> caller to specify whether or not it wants the filename argument.

We didn't need that "\$arg" thing before because we were passing whole
functions (which then decided to pass the arg or not). Can we just keep
doing that (using functions)?

That puts the onus on callers who want to ignore the arg to wrap any
commands in a function, but saves ones who _do_ want to use the arg from
having to do anything. So it's a tradeoff, but I think an extra function
wrapper is less magical than the "\$arg" eval (plus from skimming the
diff, I think most callers do use it).

So for example, this:

> +test_submodule_switch_func "apply --3way diff" "create_diff"

becomes:

  apply_diff () {
          git apply --3way diff
  }
  test_submodule_switch_func apply_diff create_diff

Obviously that requires dropping the implicit "git" from $command (but
I'd argue that makes things less magical, too).

> @@ -326,7 +336,10 @@ test_submodule_switch_common () {
>  		(
>  			cd submodule_update &&
>  			git branch -t add_sub1 origin/add_sub1 &&
> -			$command add_sub1 &&
> +			arg=add_sub1 &&
> +			$before "$arg" &&
> +			eval $command &&
> +			$after "$arg" &&

This block gets repeated a lot, and I wondered if a helper function
could make it better. But:

  1. We'd have to pass in a bunch of variables (or consider them as
     magic globals).

  2. Some callers want test_must_fail and other's don't.

So I think it would be something like:

  do_command () {
    $before "$1" &&
    "$2" $command "$1" &&
    $after "$1"
  }

  ...
  do_command add_sub1 &&
  ...

There's definitely still some magic there, but at least it's all
contained within the one script. I could go either way on it.

-Peff

[PATCH v5 0/4] t: replace incorrect test_must_fail usage (part 5)

[Denton Liu]

Hi Peff,

I've taken your suggestions and rerolled the last patch. Notably, I
combined $before and $command into a single function and then passed in
`test_must_fail` as the second argument if it's expected to fail. I also
decided to use a helper function to reduce the duplication of the
calling logic but it's still a little verbose because I opted to make
use of argument passing over global variables where possible.

One thing that I'm leaning towards is completely removing $after. We can
combine $after with $command like I did in the first few iterations of
this series and just write it as:

	git_test_thing () {
		# before...
		$2 git do-the-thing "$1" &&
		if test -n "$2"
		then
			return
		fi &&
		# after...
	}

but I'm also okay with leaving it as is.

Also, one problem that's present is that for tests that use
test_submodule_switch() and test_submodule_forced_switch(), $command
will be listed as git_test_func, which might make debugging more
difficult. I think I'd prefer the version we currently have queued for
this reason (and why I did the whole song and dance of $OVERWRITING_FAIL
in the first place), but if you think that this is fine, I'm fine with
this approach as well.

Thanks,

Denton
---

The overall scope of these patches is to replace inappropriate uses of
test_must_fail. IOW, we should only allow test_must_fail to run on `git`
and `test-tool`. Ultimately, we will conclude by making test_must_fail
error out on non-git commands. An advance view of the final series can
be found here[1].

This is the fifth part. It focuses on lib-submodule-update.sh and tests
that make use of it.

The first part can be found here[2]. The second part can be found
here[3]. The third part can be found here[4]. The fourth part can be
found here[5].

Changes since v1.2:

* In "lib-submodule-update: pass OVERWRITING_FAIL", use if-then return
  to reduce the amount of code churn

Changes since v2:

* Replace the OVERWRITING_FAIL approach with callback functions as
  suggested by Peff[6]

Changes since v3:

* Simply handling of empty $before and $after

* Add more comments on the usage of the helper functions

Changes since v4:

* Combine $before and $command back together

* Use a helper function to call the callbacks

* Replace eval with proper argument passing

[1]: (may be rebased at any time) https://github.com/Denton-L/git/tree/ready/cleanup-test-must-fail2
[2]: https://lore.kernel.org/git/cover.1576583819.git.liu.denton@gmail.com/
[3]: https://lore.kernel.org/git/cover.1577454401.git.liu.denton@gmail.com/
[4]: https://lore.kernel.org/git/cover.1585209554.git.liu.denton@gmail.com/
[5]: https://lore.kernel.org/git/cover.1587372771.git.liu.denton@gmail.com/
[6]: https://lore.kernel.org/git/20200521182928.GA1308647@coredump.intra.peff.net/

Denton Liu (4):
  lib-submodule-update: add space after function name
  lib-submodule-update: consolidate --recurse-submodules
  lib-submodule-update: prepend "git" to $command
  lib-submodule-update: use callbacks in test_submodule_switch_common()

 t/lib-submodule-update.sh        | 107 ++++++++++++++++++++++---------
 t/t1013-read-tree-submodule.sh   |   4 +-
 t/t2013-checkout-submodule.sh    |   4 +-
 t/t3426-rebase-submodule.sh      |   8 +--
 t/t3512-cherry-pick-submodule.sh |   2 +-
 t/t3513-revert-submodule.sh      |   7 +-
 t/t3906-stash-submodule.sh       |   7 +-
 t/t4137-apply-submodule.sh       |  10 +--
 t/t4255-am-submodule.sh          |  10 +--
 t/t5572-pull-submodule.sh        |  16 ++---
 t/t6041-bisect-submodule.sh      |   7 +-
 t/t7112-reset-submodule.sh       |   6 +-
 t/t7613-merge-submodule.sh       |   8 +--
 13 files changed, 127 insertions(+), 69 deletions(-)

Range-diff against v4:
1:  ba2f642e0f = 1:  ba2f642e0f lib-submodule-update: add space after function name
2:  16d0a3eb9a = 2:  16d0a3eb9a lib-submodule-update: consolidate --recurse-submodules
3:  09446be5b9 = 3:  09446be5b9 lib-submodule-update: prepend "git" to $command
4:  35d07117e6 ! 4:  250eee1dd3 lib-submodule-update: use callbacks in test_submodule_switch_common()
    @@ Commit message
         intended behaviour.
     
         Instead of invoking $command as one monolithic helper function, break it
    -    up into three parts:
    +    up into two parts:
     
    -            1. $command which is always a git command.
    -            2. $before which is a callback function that runs just prior to
    -               $command.
    -            3. $after which is a callback function that runs just after
    +            1. $command which is a function containing a git command.
    +            2. $after which is a callback function that runs just after
                    $command.
     
    -    If the command requires a filename argument, specify it as `\$arg` since
    -    that variable will be set and the whole $command string will be eval'd.
    -    Unfortunately, there is no way to get rid of the eval as some of the
    -    commands that are passed (such as the `git pull` tests) require that no
    -    additional arguments are passed so we must have some mechanism for the
    -    caller to specify whether or not it wants the filename argument.
    +    If the command requires a filename argument, it will be passed in as
    +    the first arg.
     
    -    The $before and $after callback functions will be passed the filename as
    -    the first arg. These callback functions are optional and, if missing,
    -    will be replaced with `true`. Also, in the case where we have a
    -    `test_must_fail` test, $after will not be executed, similar to how the
    -    helper functions currently behave when the git command fails and exits
    -    the &&-chain.
    +    The $after callback function will be passed the filename as the first
    +    arg. This callback function is optional and, if missing, will be
    +    replaced with `true`. Also, in the case where we have a `test_must_fail`
    +    test, $after will not be executed, similar to how the helper functions
    +    currently behave when the git command fails and exits the &&-chain.
     
    -    Finally, as an added bonus, `test_must_fail` will only run on $command
    -    which is guaranteed to be a git command.
    +    Finally, as an added bonus, `test_must_fail` will now only run on git
    +    commands.
     
         An alternate design was considered where $OVERWRITING_FAIL is set from
         test_submodule_switch_common() and exposed to the helper function. This
    @@ Commit message
         using a signalling magic environment variable might be too indirect.
     
      ## t/lib-submodule-update.sh ##
    +@@ t/lib-submodule-update.sh: test_submodule_content () {
    + 	)
    + }
    + 
    ++test_command () {
    ++	command="$1" after="$2" arg="$3" &&
    ++	$command "$arg" &&
    ++	$after "$arg"
    ++}
    ++
    ++test_command_fail () {
    ++	command="$1" arg="$2" &&
    ++	$command "$arg" test_must_fail
    ++}
    ++
    + # Test that the following transitions are correctly handled:
    + # - Updated submodule
    + # - New submodule
     @@ t/lib-submodule-update.sh: test_submodule_content () {
      # Removing a submodule containing a .git directory must fail even when forced
      # to protect the history!
      #
    -+# $1: The git command to be eval'd and tested. The submodule being operated on
    -+# will be available as $arg.
    ++# $1: The function which contains the git command. It will be passed two
    ++# arguments. $1 will be the submodule being operated on. $2 will be
    ++# "test_must_fail" if the git command is expected to fail, else the empty
    ++# string.
     +#
    -+# $2: The function that will run before the git command. It will be passed the
    -+# submodule being operated on as the only argument. This argument is optional.
    -+#
    -+# $3: The function that will run after $1. It will be passed the submodule
    ++# $2: The function that will run after $1. It will be passed the submodule
     +# being operated on as the only argument. This argument is optional. It will
     +# not be run when testing a case where the git command is expected to fail.
      
    @@ t/lib-submodule-update.sh: test_submodule_content () {
      # or test_submodule_forced_switch() instead.
      test_submodule_switch_common () {
     -	command="$1"
    -+	command="$1" before="${2:-true}" after="${3:-true}"
    ++	command="$1" after="${2:-true}"
     +
      	######################### Appearing submodule #########################
      	# Switching to a commit letting a submodule appear creates empty dir ...
    @@ t/lib-submodule-update.sh: test_submodule_switch_common () {
      			cd submodule_update &&
      			git branch -t add_sub1 origin/add_sub1 &&
     -			$command add_sub1 &&
    -+			arg=add_sub1 &&
    -+			$before "$arg" &&
    -+			eval $command &&
    -+			$after "$arg" &&
    ++			test_command "$command" "$after" add_sub1 &&
      			test_superproject_content origin/add_sub1 &&
      			test_dir_is_empty sub1 &&
      			git submodule update --init --recursive &&
    @@ t/lib-submodule-update.sh: test_submodule_switch_common () {
      			mkdir sub1 &&
      			git branch -t add_sub1 origin/add_sub1 &&
     -			$command add_sub1 &&
    -+			arg=add_sub1 &&
    -+			$before "$arg" &&
    -+			eval $command &&
    -+			$after "$arg" &&
    ++			test_command "$command" "$after" add_sub1 &&
      			test_superproject_content origin/add_sub1 &&
      			test_dir_is_empty sub1 &&
      			git submodule update --init --recursive &&
    @@ t/lib-submodule-update.sh: test_submodule_switch_common () {
      			cd submodule_update &&
      			git branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&
     -			$command replace_file_with_sub1 &&
    -+			arg=replace_file_with_sub1 &&
    -+			$before "$arg" &&
    -+			eval $command &&
    -+			$after "$arg" &&
    ++			test_command "$command" "$after" replace_file_with_sub1 &&
      			test_superproject_content origin/replace_file_with_sub1 &&
      			test_dir_is_empty sub1 &&
      			git submodule update --init --recursive &&
    @@ t/lib-submodule-update.sh: test_submodule_switch_common () {
      			cd submodule_update &&
      			git branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&
     -			$command replace_directory_with_sub1 &&
    -+			arg=replace_directory_with_sub1  &&
    -+			$before "$arg" &&
    -+			eval $command &&
    -+			$after "$arg" &&
    ++			test_command "$command" "$after" replace_directory_with_sub1 &&
      			test_superproject_content origin/replace_directory_with_sub1 &&
      			test_dir_is_empty sub1 &&
      			git submodule update --init --recursive &&
    @@ t/lib-submodule-update.sh: test_submodule_switch_common () {
      			cd submodule_update &&
      			git branch -t remove_sub1 origin/remove_sub1 &&
     -			$command remove_sub1 &&
    -+			arg=remove_sub1 &&
    -+			$before "$arg" &&
    -+			eval $command &&
    -+			$after "$arg" &&
    ++			test_command "$command" "$after" remove_sub1 &&
      			test_superproject_content origin/remove_sub1 &&
      			test_submodule_content sub1 origin/add_sub1
      		)
    @@ t/lib-submodule-update.sh: test_submodule_switch_common () {
      			git branch -t remove_sub1 origin/remove_sub1 &&
      			replace_gitfile_with_git_dir sub1 &&
     -			$command remove_sub1 &&
    -+			arg=remove_sub1 &&
    -+			$before "$arg" &&
    -+			eval $command &&
    -+			$after "$arg" &&
    ++			test_command "$command" "$after" remove_sub1 &&
      			test_superproject_content origin/remove_sub1 &&
      			test_git_directory_is_unchanged sub1 &&
      			test_submodule_content sub1 origin/add_sub1
    @@ t/lib-submodule-update.sh: test_submodule_switch_common () {
      			cd submodule_update &&
      			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
     -			test_must_fail $command replace_sub1_with_directory &&
    -+			arg=replace_sub1_with_directory &&
    -+			$before "$arg" &&
    -+			eval test_must_fail $command &&
    ++			test_command_fail "$command" replace_sub1_with_directory &&
      			test_superproject_content origin/add_sub1 &&
      			test_submodule_content sub1 origin/add_sub1
      		)
    @@ t/lib-submodule-update.sh: test_submodule_switch_common () {
      			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
      			replace_gitfile_with_git_dir sub1 &&
     -			test_must_fail $command replace_sub1_with_directory &&
    -+			arg=replace_sub1_with_directory &&
    -+			$before "$arg" &&
    -+			eval test_must_fail $command &&
    ++			test_command_fail "$command" replace_sub1_with_directory &&
      			test_superproject_content origin/add_sub1 &&
      			test_git_directory_is_unchanged sub1 &&
      			test_submodule_content sub1 origin/add_sub1
    @@ t/lib-submodule-update.sh: test_submodule_switch_common () {
      			cd submodule_update &&
      			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
     -			test_must_fail $command replace_sub1_with_file &&
    -+			arg=replace_sub1_with_file &&
    -+			$before "$arg" &&
    -+			eval test_must_fail $command &&
    ++			test_command_fail "$command" replace_sub1_with_file &&
      			test_superproject_content origin/add_sub1 &&
      			test_submodule_content sub1 origin/add_sub1
      		)
    @@ t/lib-submodule-update.sh: test_submodule_switch_common () {
      			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
      			replace_gitfile_with_git_dir sub1 &&
     -			test_must_fail $command replace_sub1_with_file &&
    -+			arg=replace_sub1_with_file &&
    -+			$before "$arg" &&
    -+			eval test_must_fail $command &&
    ++			test_command_fail "$command" replace_sub1_with_file &&
      			test_superproject_content origin/add_sub1 &&
      			test_git_directory_is_unchanged sub1 &&
      			test_submodule_content sub1 origin/add_sub1
    @@ t/lib-submodule-update.sh: test_submodule_switch_common () {
      			cd submodule_update &&
      			git branch -t modify_sub1 origin/modify_sub1 &&
     -			$command modify_sub1 &&
    -+			arg=modify_sub1 &&
    -+			$before "$arg" &&
    -+			eval $command &&
    -+			$after "$arg" &&
    ++			test_command "$command" "$after" modify_sub1 &&
      			test_superproject_content origin/modify_sub1 &&
      			test_submodule_content sub1 origin/add_sub1 &&
      			git submodule update &&
    @@ t/lib-submodule-update.sh: test_submodule_switch_common () {
      			cd submodule_update &&
      			git branch -t invalid_sub1 origin/invalid_sub1 &&
     -			$command invalid_sub1 &&
    -+			arg=invalid_sub1 &&
    -+			$before "$arg" &&
    -+			eval $command &&
    -+			$after "$arg" &&
    ++			test_command "$command" "$after" invalid_sub1 &&
      			test_superproject_content origin/invalid_sub1 &&
      			test_submodule_content sub1 origin/add_sub1 &&
      			test_must_fail git submodule update &&
    @@ t/lib-submodule-update.sh: test_submodule_switch_common () {
      			cd submodule_update &&
      			git branch -t valid_sub1 origin/valid_sub1 &&
     -			$command valid_sub1 &&
    -+			arg=valid_sub1 &&
    -+			$before "$arg" &&
    -+			eval $command &&
    -+			$after "$arg" &&
    ++			test_command "$command" "$after" valid_sub1 &&
      			test_superproject_content origin/valid_sub1 &&
      			test_dir_is_empty sub1 &&
      			git submodule update --init --recursive &&
    @@ t/lib-submodule-update.sh: test_submodule_switch_common () {
      # below to 1.
      #
     -# Use as follows:
    -+# $1: The git command to be eval'd and tested. The submodule being operated on
    -+# will be available as $arg. Do not include the leading "git".
    - #
    --# my_func () {
    -+# $2: The function that will run before the git command. It will be passed the
    -+# submodule being operated on as the only argument. This argument is optional.
    ++# $1: The function which contains the git command. It will be passed two
    ++# arguments. $1 will be the submodule being operated on. $2 will be
    ++# "test_must_fail" if the git command is expected to fail, else the empty
    ++# string.
     +#
    -+# $3: The function that will run after $1. It will be passed the submodule
    ++# $2: The function that will run after $1. It will be passed the submodule
     +# being operated on as the only argument. This argument is optional. It will
     +# not be run when testing a case where the git command is expected to fail.
     +#
     +# The following example uses `git some-command` as an example command to be
     +# tested. It updates the worktree and index to match a target, but not any
     +# submodule directories.
    -+#
    -+# my_func_before () {
    + #
    + # my_func () {
      #   target=$1
     -#   # Do something here that updates the worktree and index to match target,
     -#   # but not any submodule directories.
    -+#   # Prepare for git some-command to be run
    ++#   $2 git some-command $target
      # }
     -# test_submodule_switch_func "my_func"
     +# my_func_after () {
     +#   target=$1
     +#   # Check the state after git some-command is run
     +# }
    -+# test_submodule_switch_func "some-command \$arg" "my_func_before" "my_func_after"
    ++# test_submodule_switch_func "my_func" "my_func_after"
      test_submodule_switch_func () {
    --	command="$1"
    + 	command="$1"
     -	test_submodule_switch_common "$command"
    -+	command="git $1"
    -+	before="$2"
    -+	after="$3"
    -+	test_submodule_switch_common "$command" "$before" "$after"
    ++	after="$2"
    ++	test_submodule_switch_common "$command" "$after"
      
      	# An empty directory does not prevent the creation of a submodule of
      	# the same name, but a file does.
    @@ t/lib-submodule-update.sh: test_submodule_switch_func () {
      			git branch -t add_sub1 origin/add_sub1 &&
      			>sub1 &&
     -			test_must_fail $command add_sub1 &&
    -+			arg=add_sub1 &&
    -+			$before "$arg" &&
    -+			eval test_must_fail $command &&
    ++			test_command_fail "$command" add_sub1 &&
      			test_superproject_content origin/no_submodule &&
      			test_must_be_empty sub1
      		)
    -@@ t/lib-submodule-update.sh: test_submodule_switch_func () {
    + 	'
      }
      
    ++git_test_func () {
    ++	$2 git $gitcmd "$1"
    ++}
    ++
      test_submodule_switch () {
     -	test_submodule_switch_func "git $1"
    -+	test_submodule_switch_func "$1 \$arg"
    ++	gitcmd="$1"
    ++	test_submodule_switch_func "git_test_func"
      }
      
      # Same as test_submodule_switch(), except that throwing away local changes in
      # the superproject is allowed.
      test_submodule_forced_switch () {
     -	command="$1"
    -+	command="git $1 \$arg"
    ++	gitcmd="$1"
    ++	command="git_test_func"
      	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
     -	test_submodule_switch_common "git $command"
     +	test_submodule_switch_common "$command"
    @@ t/lib-submodule-update.sh: test_submodule_forced_switch () {
      			git branch -t add_sub1 origin/add_sub1 &&
      			>sub1 &&
     -			$command add_sub1 &&
    -+			arg=add_sub1 &&
    -+			eval $command &&
    ++			test_command "$command" true add_sub1 &&
      			test_superproject_content origin/add_sub1 &&
      			test_dir_is_empty sub1
      		)
     
      ## t/t3426-rebase-submodule.sh ##
     @@ t/t3426-rebase-submodule.sh: git_rebase () {
    - 	git revert HEAD &&
      	git status -su >actual &&
      	ls -1pR * >>actual &&
    --	test_cmp expect actual &&
    + 	test_cmp expect actual &&
     -	git rebase "$1"
    -+	test_cmp expect actual
    ++	$2 git rebase "$1"
      }
      
    --test_submodule_switch_func "git_rebase"
    -+test_submodule_switch_func "rebase \$arg" "git_rebase"
    - 
    - git_rebase_interactive () {
    - 	git status -su >expect &&
    + test_submodule_switch_func "git_rebase"
     @@ t/t3426-rebase-submodule.sh: git_rebase_interactive () {
    - 	ls -1pR * >>actual &&
      	test_cmp expect actual &&
      	set_fake_editor &&
    --	echo "fake-editor.sh" >.git/info/exclude &&
    + 	echo "fake-editor.sh" >.git/info/exclude &&
     -	git rebase -i "$1"
    -+	echo "fake-editor.sh" >.git/info/exclude
    ++	$2 git rebase -i "$1"
      }
      
    --test_submodule_switch_func "git_rebase_interactive"
    -+test_submodule_switch_func "rebase -i \$arg" "git_rebase_interactive"
    - 
    - test_expect_success 'rebase interactive ignores modified submodules' '
    - 	test_when_finished "rm -rf super sub" &&
    + test_submodule_switch_func "git_rebase_interactive"
     
      ## t/t3513-revert-submodule.sh ##
    -@@ t/t3513-revert-submodule.sh: test_description='revert can handle submodules'
    - # first so we can restore the work tree test setup after doing the checkout
    - # and revert.  We test here that the restored work tree content is identical
    - # to that at the beginning. The last revert is then tested by the framework.
    --git_revert () {
    -+git_revert_before () {
    +@@ t/t3513-revert-submodule.sh: git_revert () {
      	git status -su >expect &&
      	ls -1pR * >>expect &&
    --	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
    + 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
     -	git checkout "$1" &&
    -+	tar cf "$TRASH_DIRECTORY/tmp.tar" *
    ++	$2 git checkout "$1"
     +}
     +
     +git_revert_after () {
    @@ t/t3513-revert-submodule.sh: git_revert () {
      
      KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
     -test_submodule_switch_func "git_revert"
    -+test_submodule_switch_func "checkout \$arg" "git_revert_before" "git_revert_after"
    ++test_submodule_switch_func "git_revert" "git_revert_after"
      
      test_done
     
      ## t/t3906-stash-submodule.sh ##
     @@ t/t3906-stash-submodule.sh: test_description='stash can handle submodules'
    - . ./test-lib.sh
    - . "$TEST_DIRECTORY"/lib-submodule-update.sh
    - 
    --git_stash () {
    -+git_stash_before () {
    + git_stash () {
      	git status -su >expect &&
    --	ls -1pR * >>expect &&
    + 	ls -1pR * >>expect &&
     -	git read-tree -u -m "$1" &&
    -+	ls -1pR * >>expect
    ++	$2 git read-tree -u -m "$1"
     +}
     +
     +git_stash_after () {
    @@ t/t3906-stash-submodule.sh: git_stash () {
      KNOWN_FAILURE_CHERRY_PICK_SEES_EMPTY_COMMIT=1
      KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
     -test_submodule_switch_func "git_stash"
    -+test_submodule_switch_func "read-tree -u -m \$arg" "git_stash_before" "git_stash_after"
    ++test_submodule_switch_func "git_stash" "git_stash_after"
      
      setup_basic () {
      	test_when_finished "rm -rf main sub" &&
     
      ## t/t4137-apply-submodule.sh ##
     @@ t/t4137-apply-submodule.sh: test_description='git apply handling submodules'
    - . ./test-lib.sh
      . "$TEST_DIRECTORY"/lib-submodule-update.sh
      
    --apply_index () {
    + apply_index () {
     -	git diff --ignore-submodules=dirty "..$1" | git apply --index -
    -+create_diff () {
    -+	git diff --ignore-submodules=dirty "..$1" >diff
    ++	git diff --ignore-submodules=dirty "..$1" >diff &&
    ++	$2 git apply --index diff
      }
      
    --test_submodule_switch_func "apply_index"
    -+test_submodule_switch_func "apply --index diff" "create_diff"
    + test_submodule_switch_func "apply_index"
      
    --apply_3way () {
    + apply_3way () {
     -	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
    --}
    --
    --test_submodule_switch_func "apply_3way"
    -+test_submodule_switch_func "apply --3way diff" "create_diff"
    ++	git diff --ignore-submodules=dirty "..$1" >diff
    ++	$2 git apply --3way diff
    + }
      
    - test_done
    + test_submodule_switch_func "apply_3way"
     
      ## t/t4255-am-submodule.sh ##
     @@ t/t4255-am-submodule.sh: test_description='git am handling submodules'
    - . ./test-lib.sh
      . "$TEST_DIRECTORY"/lib-submodule-update.sh
      
    --am () {
    + am () {
     -	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am -
    -+create_patch () {
    -+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch
    ++	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
    ++	$2 git am patch
      }
      
    --test_submodule_switch_func "am"
    --
    --am_3way () {
    + test_submodule_switch_func "am"
    + 
    + am_3way () {
     -	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am --3way -
    --}
    -+test_submodule_switch_func "am patch" "create_patch"
    ++	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
    ++	$2 git am --3way patch
    + }
      
      KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
    --test_submodule_switch_func "am_3way"
    -+test_submodule_switch_func "am --3way patch" "create_patch"
    - 
    - test_expect_success 'setup diff.submodule' '
    - 	test_commit one &&
     
      ## t/t5572-pull-submodule.sh ##
     @@ t/t5572-pull-submodule.sh: reset_branch_to_HEAD () {
    - 	git branch --set-upstream-to="origin/$1" "$1"
    + 
    + git_pull () {
    + 	reset_branch_to_HEAD "$1" &&
    +-	git pull
    ++	$2 git pull
      }
      
    --git_pull () {
    --	reset_branch_to_HEAD "$1" &&
    --	git pull
    --}
    --
      # pulls without conflicts
    --test_submodule_switch_func "git_pull"
    -+test_submodule_switch_func "pull" "reset_branch_to_HEAD"
    +@@ t/t5572-pull-submodule.sh: test_submodule_switch_func "git_pull"
      
    --git_pull_ff () {
    --	reset_branch_to_HEAD "$1" &&
    + git_pull_ff () {
    + 	reset_branch_to_HEAD "$1" &&
     -	git pull --ff
    --}
    -+test_submodule_switch_func "pull --ff" "reset_branch_to_HEAD"
    ++	$2 git pull --ff
    + }
      
    --test_submodule_switch_func "git_pull_ff"
    --
    --git_pull_ff_only () {
    --	reset_branch_to_HEAD "$1" &&
    + test_submodule_switch_func "git_pull_ff"
    + 
    + git_pull_ff_only () {
    + 	reset_branch_to_HEAD "$1" &&
     -	git pull --ff-only
    --}
    --
    --test_submodule_switch_func "git_pull_ff_only"
    --
    --git_pull_noff () {
    --	reset_branch_to_HEAD "$1" &&
    ++	$2 git pull --ff-only
    + }
    + 
    + test_submodule_switch_func "git_pull_ff_only"
    + 
    + git_pull_noff () {
    + 	reset_branch_to_HEAD "$1" &&
     -	git pull --no-ff
    --}
    -+test_submodule_switch_func "pull --ff-only" "reset_branch_to_HEAD"
    ++	$2 git pull --no-ff
    + }
      
      KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
    - KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
    --test_submodule_switch_func "git_pull_noff"
    -+test_submodule_switch_func "pull --no-ff" "reset_branch_to_HEAD"
    - 
    - test_expect_success 'pull --recurse-submodule setup' '
    - 	test_create_repo child &&
     
      ## t/t6041-bisect-submodule.sh ##
    -@@ t/t6041-bisect-submodule.sh: test_description='bisect can handle submodules'
    - . ./test-lib.sh
    - . "$TEST_DIRECTORY"/lib-submodule-update.sh
    - 
    --git_bisect () {
    -+git_bisect_before () {
    - 	git status -su >expect &&
    +@@ t/t6041-bisect-submodule.sh: git_bisect () {
      	ls -1pR * >>expect &&
      	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
    --	GOOD=$(git rev-parse --verify HEAD) &&
    + 	GOOD=$(git rev-parse --verify HEAD) &&
     -	git checkout "$1" &&
    -+	GOOD=$(git rev-parse --verify HEAD)
    ++	$2 git checkout "$1"
     +}
     +
     +git_bisect_after () {
    @@ t/t6041-bisect-submodule.sh: git_bisect () {
      }
      
     -test_submodule_switch_func "git_bisect"
    -+test_submodule_switch_func "checkout \$arg" "git_bisect_before" "git_bisect_after"
    ++test_submodule_switch_func "git_bisect" "git_bisect_after"
      
      test_done
-- 
2.27.0.132.g321788e831

[PATCH v5 1/4] lib-submodule-update: add space after function name

[Denton Liu]

In the shell scripts in this codebase, the usual style is to include a
space between the function name and the (). Add these missing spaces to
conform to the usual style of the code.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 1dd17fc03e..122554acd6 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -183,7 +183,7 @@ test_git_directory_is_unchanged () {
 	)
 }
 
-test_git_directory_exists() {
+test_git_directory_exists () {
 	test -e ".git/modules/$1" &&
 	if test -f sub1/.git
 	then
@@ -309,7 +309,7 @@ test_submodule_content () {
 
 # Internal function; use test_submodule_switch() or
 # test_submodule_forced_switch() instead.
-test_submodule_switch_common() {
+test_submodule_switch_common () {
 	command="$1"
 	######################### Appearing submodule #########################
 	# Switching to a commit letting a submodule appear creates empty dir ...
@@ -629,7 +629,7 @@ test_submodule_forced_switch () {
 
 # Internal function; use test_submodule_switch_recursing_with_args() or
 # test_submodule_forced_switch_recursing_with_args() instead.
-test_submodule_recursing_with_args_common() {
+test_submodule_recursing_with_args_common () {
 	command="$1"
 
 	######################### Appearing submodule #########################
-- 
2.27.0.132.g321788e831

[PATCH v5 2/4] lib-submodule-update: consolidate --recurse-submodules

[Denton Liu]

Both test_submodule_switch_recursing_with_args() and
test_submodule_forced_switch_recursing_with_args() call the internal
function test_submodule_recursing_with_args_common() with the final
argument of `--recurse-submodules`. Consolidate this duplication by
appending the argument in test_submodule_recursing_with_args_common().

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 122554acd6..bb36287803 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -630,7 +630,7 @@ test_submodule_forced_switch () {
 # Internal function; use test_submodule_switch_recursing_with_args() or
 # test_submodule_forced_switch_recursing_with_args() instead.
 test_submodule_recursing_with_args_common () {
-	command="$1"
+	command="$1 --recurse-submodules"
 
 	######################### Appearing submodule #########################
 	# Switching to a commit letting a submodule appear checks it out ...
@@ -809,7 +809,7 @@ test_submodule_recursing_with_args_common () {
 # test_submodule_switch_recursing_with_args "$GIT_COMMAND"
 test_submodule_switch_recursing_with_args () {
 	cmd_args="$1"
-	command="git $cmd_args --recurse-submodules"
+	command="git $cmd_args"
 	test_submodule_recursing_with_args_common "$command"
 
 	RESULTDS=success
@@ -927,7 +927,7 @@ test_submodule_switch_recursing_with_args () {
 # away local changes in the superproject is allowed.
 test_submodule_forced_switch_recursing_with_args () {
 	cmd_args="$1"
-	command="git $cmd_args --recurse-submodules"
+	command="git $cmd_args"
 	test_submodule_recursing_with_args_common "$command"
 
 	RESULT=success
-- 
2.27.0.132.g321788e831

[PATCH v5 3/4] lib-submodule-update: prepend "git" to $command

[Denton Liu]

Since all invocations of test_submodule_forced_switch() are git
commands, automatically prepend "git" before invoking
test_submodule_switch_common().

Similarly, many invocations of test_submodule_switch() are also git
commands so automatically prepend "git" before invoking
test_submodule_switch_common() as well.

Finally, for invocations of test_submodule_switch() that invoke a custom
function, rename the old function to test_submodule_switch_func().

This is necessary because in a future commit, we will be adding some
logic that needs to distinguish between an invocation of a plain git
comamnd and an invocation of a test helper function.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh        | 14 +++++++++-----
 t/t1013-read-tree-submodule.sh   |  4 ++--
 t/t2013-checkout-submodule.sh    |  4 ++--
 t/t3426-rebase-submodule.sh      |  4 ++--
 t/t3512-cherry-pick-submodule.sh |  2 +-
 t/t3513-revert-submodule.sh      |  2 +-
 t/t3906-stash-submodule.sh       |  2 +-
 t/t4137-apply-submodule.sh       |  4 ++--
 t/t4255-am-submodule.sh          |  4 ++--
 t/t5572-pull-submodule.sh        |  8 ++++----
 t/t6041-bisect-submodule.sh      |  2 +-
 t/t7112-reset-submodule.sh       |  6 +++---
 t/t7613-merge-submodule.sh       |  8 ++++----
 13 files changed, 34 insertions(+), 30 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index bb36287803..7c3ba1be00 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -307,8 +307,8 @@ test_submodule_content () {
 # to protect the history!
 #
 
-# Internal function; use test_submodule_switch() or
-# test_submodule_forced_switch() instead.
+# Internal function; use test_submodule_switch_func(), test_submodule_switch(),
+# or test_submodule_forced_switch() instead.
 test_submodule_switch_common () {
 	command="$1"
 	######################### Appearing submodule #########################
@@ -566,8 +566,8 @@ test_submodule_switch_common () {
 #   # Do something here that updates the worktree and index to match target,
 #   # but not any submodule directories.
 # }
-# test_submodule_switch "my_func"
-test_submodule_switch () {
+# test_submodule_switch_func "my_func"
+test_submodule_switch_func () {
 	command="$1"
 	test_submodule_switch_common "$command"
 
@@ -587,12 +587,16 @@ test_submodule_switch () {
 	'
 }
 
+test_submodule_switch () {
+	test_submodule_switch_func "git $1"
+}
+
 # Same as test_submodule_switch(), except that throwing away local changes in
 # the superproject is allowed.
 test_submodule_forced_switch () {
 	command="$1"
 	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
-	test_submodule_switch_common "$command"
+	test_submodule_switch_common "git $command"
 
 	# When forced, a file in the superproject does not prevent creating a
 	# submodule of the same name.
diff --git a/t/t1013-read-tree-submodule.sh b/t/t1013-read-tree-submodule.sh
index 91a6fafcb4..b6df7444c0 100755
--- a/t/t1013-read-tree-submodule.sh
+++ b/t/t1013-read-tree-submodule.sh
@@ -12,8 +12,8 @@ test_submodule_switch_recursing_with_args "read-tree -u -m"
 
 test_submodule_forced_switch_recursing_with_args "read-tree -u --reset"
 
-test_submodule_switch "git read-tree -u -m"
+test_submodule_switch "read-tree -u -m"
 
-test_submodule_forced_switch "git read-tree -u --reset"
+test_submodule_forced_switch "read-tree -u --reset"
 
 test_done
diff --git a/t/t2013-checkout-submodule.sh b/t/t2013-checkout-submodule.sh
index 8f86b5f4b2..b2bdd1fcb4 100755
--- a/t/t2013-checkout-submodule.sh
+++ b/t/t2013-checkout-submodule.sh
@@ -68,8 +68,8 @@ test_submodule_switch_recursing_with_args "checkout"
 
 test_submodule_forced_switch_recursing_with_args "checkout -f"
 
-test_submodule_switch "git checkout"
+test_submodule_switch "checkout"
 
-test_submodule_forced_switch "git checkout -f"
+test_submodule_forced_switch "checkout -f"
 
 test_done
diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
index a2bba04ba9..788605ccc0 100755
--- a/t/t3426-rebase-submodule.sh
+++ b/t/t3426-rebase-submodule.sh
@@ -20,7 +20,7 @@ git_rebase () {
 	git rebase "$1"
 }
 
-test_submodule_switch "git_rebase"
+test_submodule_switch_func "git_rebase"
 
 git_rebase_interactive () {
 	git status -su >expect &&
@@ -38,7 +38,7 @@ git_rebase_interactive () {
 	git rebase -i "$1"
 }
 
-test_submodule_switch "git_rebase_interactive"
+test_submodule_switch_func "git_rebase_interactive"
 
 test_expect_success 'rebase interactive ignores modified submodules' '
 	test_when_finished "rm -rf super sub" &&
diff --git a/t/t3512-cherry-pick-submodule.sh b/t/t3512-cherry-pick-submodule.sh
index bd78287841..6ece1d8573 100755
--- a/t/t3512-cherry-pick-submodule.sh
+++ b/t/t3512-cherry-pick-submodule.sh
@@ -7,7 +7,7 @@ test_description='cherry-pick can handle submodules'
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "git cherry-pick"
+test_submodule_switch "cherry-pick"
 
 test_expect_success 'unrelated submodule/file conflict is ignored' '
 	test_create_repo sub &&
diff --git a/t/t3513-revert-submodule.sh b/t/t3513-revert-submodule.sh
index 5e39fcdb66..95a7f64471 100755
--- a/t/t3513-revert-submodule.sh
+++ b/t/t3513-revert-submodule.sh
@@ -26,6 +26,6 @@ git_revert () {
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
-test_submodule_switch "git_revert"
+test_submodule_switch_func "git_revert"
 
 test_done
diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
index b93d1d74da..6a7e801ca0 100755
--- a/t/t3906-stash-submodule.sh
+++ b/t/t3906-stash-submodule.sh
@@ -19,7 +19,7 @@ git_stash () {
 KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES=1
 KNOWN_FAILURE_CHERRY_PICK_SEES_EMPTY_COMMIT=1
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
-test_submodule_switch "git_stash"
+test_submodule_switch_func "git_stash"
 
 setup_basic () {
 	test_when_finished "rm -rf main sub" &&
diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
index a9bd40a6d0..b645e303a0 100755
--- a/t/t4137-apply-submodule.sh
+++ b/t/t4137-apply-submodule.sh
@@ -9,12 +9,12 @@ apply_index () {
 	git diff --ignore-submodules=dirty "..$1" | git apply --index -
 }
 
-test_submodule_switch "apply_index"
+test_submodule_switch_func "apply_index"
 
 apply_3way () {
 	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
 }
 
-test_submodule_switch "apply_3way"
+test_submodule_switch_func "apply_3way"
 
 test_done
diff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh
index 0ba8194403..1b179d5f45 100755
--- a/t/t4255-am-submodule.sh
+++ b/t/t4255-am-submodule.sh
@@ -9,14 +9,14 @@ am () {
 	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am -
 }
 
-test_submodule_switch "am"
+test_submodule_switch_func "am"
 
 am_3way () {
 	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am --3way -
 }
 
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "am_3way"
+test_submodule_switch_func "am_3way"
 
 test_expect_success 'setup diff.submodule' '
 	test_commit one &&
diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
index f916729a12..f911bf631e 100755
--- a/t/t5572-pull-submodule.sh
+++ b/t/t5572-pull-submodule.sh
@@ -17,21 +17,21 @@ git_pull () {
 }
 
 # pulls without conflicts
-test_submodule_switch "git_pull"
+test_submodule_switch_func "git_pull"
 
 git_pull_ff () {
 	reset_branch_to_HEAD "$1" &&
 	git pull --ff
 }
 
-test_submodule_switch "git_pull_ff"
+test_submodule_switch_func "git_pull_ff"
 
 git_pull_ff_only () {
 	reset_branch_to_HEAD "$1" &&
 	git pull --ff-only
 }
 
-test_submodule_switch "git_pull_ff_only"
+test_submodule_switch_func "git_pull_ff_only"
 
 git_pull_noff () {
 	reset_branch_to_HEAD "$1" &&
@@ -40,7 +40,7 @@ git_pull_noff () {
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "git_pull_noff"
+test_submodule_switch_func "git_pull_noff"
 
 test_expect_success 'pull --recurse-submodule setup' '
 	test_create_repo child &&
diff --git a/t/t6041-bisect-submodule.sh b/t/t6041-bisect-submodule.sh
index 62b8a2e7bb..0e0cdf638d 100755
--- a/t/t6041-bisect-submodule.sh
+++ b/t/t6041-bisect-submodule.sh
@@ -27,6 +27,6 @@ git_bisect () {
 	git bisect bad $BAD
 }
 
-test_submodule_switch "git_bisect"
+test_submodule_switch_func "git_bisect"
 
 test_done
diff --git a/t/t7112-reset-submodule.sh b/t/t7112-reset-submodule.sh
index a1cb9ff858..8741b665c9 100755
--- a/t/t7112-reset-submodule.sh
+++ b/t/t7112-reset-submodule.sh
@@ -13,10 +13,10 @@ test_submodule_switch_recursing_with_args "reset --keep"
 
 test_submodule_forced_switch_recursing_with_args "reset --hard"
 
-test_submodule_switch "git reset --keep"
+test_submodule_switch "reset --keep"
 
-test_submodule_switch "git reset --merge"
+test_submodule_switch "reset --merge"
 
-test_submodule_forced_switch "git reset --hard"
+test_submodule_forced_switch "reset --hard"
 
 test_done
diff --git a/t/t7613-merge-submodule.sh b/t/t7613-merge-submodule.sh
index d1e9fcc781..04bf4be7d7 100755
--- a/t/t7613-merge-submodule.sh
+++ b/t/t7613-merge-submodule.sh
@@ -6,14 +6,14 @@ test_description='merge can handle submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 # merges without conflicts
-test_submodule_switch "git merge"
+test_submodule_switch "merge"
 
-test_submodule_switch "git merge --ff"
+test_submodule_switch "merge --ff"
 
-test_submodule_switch "git merge --ff-only"
+test_submodule_switch "merge --ff-only"
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
-test_submodule_switch "git merge --no-ff"
+test_submodule_switch "merge --no-ff"
 
 test_done
-- 
2.27.0.132.g321788e831

[PATCH v5 4/4] lib-submodule-update: use callbacks in test_submodule_switch_common()

[Denton Liu]

When we run a test helper function in test_submodule_switch_common(), we
sometimes specify a whole helper function as the $command. When we do
this, in some test cases, we just mark the whole function with
`test_must_fail`. However, it's possible that the helper function might
fail earlier or later than expected due to an introduced bug. If this
happens, then the test case will still report as passing but it should
really be marked as failing since it didn't actually display the
intended behaviour.

Instead of invoking $command as one monolithic helper function, break it
up into two parts:

	1. $command which is a function containing a git command.
	2. $after which is a callback function that runs just after
	   $command.

If the command requires a filename argument, it will be passed in as
the first arg.

The $after callback function will be passed the filename as the first
arg. This callback function is optional and, if missing, will be
replaced with `true`. Also, in the case where we have a `test_must_fail`
test, $after will not be executed, similar to how the helper functions
currently behave when the git command fails and exits the &&-chain.

Finally, as an added bonus, `test_must_fail` will now only run on git
commands.

An alternate design was considered where $OVERWRITING_FAIL is set from
test_submodule_switch_common() and exposed to the helper function. This
approach was considered too difficult to understand due to the fact that
using a signalling magic environment variable might be too indirect.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh   | 89 +++++++++++++++++++++++++++----------
 t/t3426-rebase-submodule.sh |  4 +-
 t/t3513-revert-submodule.sh |  7 ++-
 t/t3906-stash-submodule.sh  |  7 ++-
 t/t4137-apply-submodule.sh  |  6 ++-
 t/t4255-am-submodule.sh     |  6 ++-
 t/t5572-pull-submodule.sh   |  8 ++--
 t/t6041-bisect-submodule.sh |  7 ++-
 8 files changed, 94 insertions(+), 40 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 7c3ba1be00..104090b489 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -290,6 +290,17 @@ test_submodule_content () {
 	)
 }
 
+test_command () {
+	command="$1" after="$2" arg="$3" &&
+	$command "$arg" &&
+	$after "$arg"
+}
+
+test_command_fail () {
+	command="$1" arg="$2" &&
+	$command "$arg" test_must_fail
+}
+
 # Test that the following transitions are correctly handled:
 # - Updated submodule
 # - New submodule
@@ -306,11 +317,20 @@ test_submodule_content () {
 # Removing a submodule containing a .git directory must fail even when forced
 # to protect the history!
 #
+# $1: The function which contains the git command. It will be passed two
+# arguments. $1 will be the submodule being operated on. $2 will be
+# "test_must_fail" if the git command is expected to fail, else the empty
+# string.
+#
+# $2: The function that will run after $1. It will be passed the submodule
+# being operated on as the only argument. This argument is optional. It will
+# not be run when testing a case where the git command is expected to fail.
 
 # Internal function; use test_submodule_switch_func(), test_submodule_switch(),
 # or test_submodule_forced_switch() instead.
 test_submodule_switch_common () {
-	command="$1"
+	command="$1" after="${2:-true}"
+
 	######################### Appearing submodule #########################
 	# Switching to a commit letting a submodule appear creates empty dir ...
 	if test "$KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES" = 1
@@ -326,7 +346,7 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t add_sub1 origin/add_sub1 &&
-			$command add_sub1 &&
+			test_command "$command" "$after" add_sub1 &&
 			test_superproject_content origin/add_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -341,7 +361,7 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			mkdir sub1 &&
 			git branch -t add_sub1 origin/add_sub1 &&
-			$command add_sub1 &&
+			test_command "$command" "$after" add_sub1 &&
 			test_superproject_content origin/add_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -356,7 +376,7 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&
-			$command replace_file_with_sub1 &&
+			test_command "$command" "$after" replace_file_with_sub1 &&
 			test_superproject_content origin/replace_file_with_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -380,7 +400,7 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&
-			$command replace_directory_with_sub1 &&
+			test_command "$command" "$after" replace_directory_with_sub1 &&
 			test_superproject_content origin/replace_directory_with_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -402,7 +422,7 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t remove_sub1 origin/remove_sub1 &&
-			$command remove_sub1 &&
+			test_command "$command" "$after" remove_sub1 &&
 			test_superproject_content origin/remove_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -415,7 +435,7 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t remove_sub1 origin/remove_sub1 &&
 			replace_gitfile_with_git_dir sub1 &&
-			$command remove_sub1 &&
+			test_command "$command" "$after" remove_sub1 &&
 			test_superproject_content origin/remove_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -443,7 +463,7 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
-			test_must_fail $command replace_sub1_with_directory &&
+			test_command_fail "$command" replace_sub1_with_directory &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -456,7 +476,7 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_directory &&
+			test_command_fail "$command" replace_sub1_with_directory &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -470,7 +490,7 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
-			test_must_fail $command replace_sub1_with_file &&
+			test_command_fail "$command" replace_sub1_with_file &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -484,7 +504,7 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_file &&
+			test_command_fail "$command" replace_sub1_with_file &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -508,7 +528,7 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t modify_sub1 origin/modify_sub1 &&
-			$command modify_sub1 &&
+			test_command "$command" "$after" modify_sub1 &&
 			test_superproject_content origin/modify_sub1 &&
 			test_submodule_content sub1 origin/add_sub1 &&
 			git submodule update &&
@@ -523,7 +543,7 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t invalid_sub1 origin/invalid_sub1 &&
-			$command invalid_sub1 &&
+			test_command "$command" "$after" invalid_sub1 &&
 			test_superproject_content origin/invalid_sub1 &&
 			test_submodule_content sub1 origin/add_sub1 &&
 			test_must_fail git submodule update &&
@@ -538,7 +558,7 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t valid_sub1 origin/valid_sub1 &&
-			$command valid_sub1 &&
+			test_command "$command" "$after" valid_sub1 &&
 			test_superproject_content origin/valid_sub1 &&
 			test_dir_is_empty sub1 &&
 			git submodule update --init --recursive &&
@@ -559,17 +579,32 @@ test_submodule_switch_common () {
 # conditions, set the appropriate KNOWN_FAILURE_* variable used in the tests
 # below to 1.
 #
-# Use as follows:
+# $1: The function which contains the git command. It will be passed two
+# arguments. $1 will be the submodule being operated on. $2 will be
+# "test_must_fail" if the git command is expected to fail, else the empty
+# string.
+#
+# $2: The function that will run after $1. It will be passed the submodule
+# being operated on as the only argument. This argument is optional. It will
+# not be run when testing a case where the git command is expected to fail.
+#
+# The following example uses `git some-command` as an example command to be
+# tested. It updates the worktree and index to match a target, but not any
+# submodule directories.
 #
 # my_func () {
 #   target=$1
-#   # Do something here that updates the worktree and index to match target,
-#   # but not any submodule directories.
+#   $2 git some-command $target
 # }
-# test_submodule_switch_func "my_func"
+# my_func_after () {
+#   target=$1
+#   # Check the state after git some-command is run
+# }
+# test_submodule_switch_func "my_func" "my_func_after"
 test_submodule_switch_func () {
 	command="$1"
-	test_submodule_switch_common "$command"
+	after="$2"
+	test_submodule_switch_common "$command" "$after"
 
 	# An empty directory does not prevent the creation of a submodule of
 	# the same name, but a file does.
@@ -580,23 +615,29 @@ test_submodule_switch_func () {
 			cd submodule_update &&
 			git branch -t add_sub1 origin/add_sub1 &&
 			>sub1 &&
-			test_must_fail $command add_sub1 &&
+			test_command_fail "$command" add_sub1 &&
 			test_superproject_content origin/no_submodule &&
 			test_must_be_empty sub1
 		)
 	'
 }
 
+git_test_func () {
+	$2 git $gitcmd "$1"
+}
+
 test_submodule_switch () {
-	test_submodule_switch_func "git $1"
+	gitcmd="$1"
+	test_submodule_switch_func "git_test_func"
 }
 
 # Same as test_submodule_switch(), except that throwing away local changes in
 # the superproject is allowed.
 test_submodule_forced_switch () {
-	command="$1"
+	gitcmd="$1"
+	command="git_test_func"
 	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
-	test_submodule_switch_common "git $command"
+	test_submodule_switch_common "$command"
 
 	# When forced, a file in the superproject does not prevent creating a
 	# submodule of the same name.
@@ -607,7 +648,7 @@ test_submodule_forced_switch () {
 			cd submodule_update &&
 			git branch -t add_sub1 origin/add_sub1 &&
 			>sub1 &&
-			$command add_sub1 &&
+			test_command "$command" true add_sub1 &&
 			test_superproject_content origin/add_sub1 &&
 			test_dir_is_empty sub1
 		)
diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
index 788605ccc0..dd5daa53d3 100755
--- a/t/t3426-rebase-submodule.sh
+++ b/t/t3426-rebase-submodule.sh
@@ -17,7 +17,7 @@ git_rebase () {
 	git status -su >actual &&
 	ls -1pR * >>actual &&
 	test_cmp expect actual &&
-	git rebase "$1"
+	$2 git rebase "$1"
 }
 
 test_submodule_switch_func "git_rebase"
@@ -35,7 +35,7 @@ git_rebase_interactive () {
 	test_cmp expect actual &&
 	set_fake_editor &&
 	echo "fake-editor.sh" >.git/info/exclude &&
-	git rebase -i "$1"
+	$2 git rebase -i "$1"
 }
 
 test_submodule_switch_func "git_rebase_interactive"
diff --git a/t/t3513-revert-submodule.sh b/t/t3513-revert-submodule.sh
index 95a7f64471..7ac06731bd 100755
--- a/t/t3513-revert-submodule.sh
+++ b/t/t3513-revert-submodule.sh
@@ -15,7 +15,10 @@ git_revert () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
-	git checkout "$1" &&
+	$2 git checkout "$1"
+}
+
+git_revert_after () {
 	git revert HEAD &&
 	rm -rf * &&
 	tar xf "$TRASH_DIRECTORY/tmp.tar" &&
@@ -26,6 +29,6 @@ git_revert () {
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
-test_submodule_switch_func "git_revert"
+test_submodule_switch_func "git_revert" "git_revert_after"
 
 test_done
diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
index 6a7e801ca0..30bdc113cd 100755
--- a/t/t3906-stash-submodule.sh
+++ b/t/t3906-stash-submodule.sh
@@ -8,7 +8,10 @@ test_description='stash can handle submodules'
 git_stash () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
-	git read-tree -u -m "$1" &&
+	$2 git read-tree -u -m "$1"
+}
+
+git_stash_after () {
 	git stash &&
 	git status -su >actual &&
 	ls -1pR * >>actual &&
@@ -19,7 +22,7 @@ git_stash () {
 KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES=1
 KNOWN_FAILURE_CHERRY_PICK_SEES_EMPTY_COMMIT=1
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
-test_submodule_switch_func "git_stash"
+test_submodule_switch_func "git_stash" "git_stash_after"
 
 setup_basic () {
 	test_when_finished "rm -rf main sub" &&
diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
index b645e303a0..5477d48ffd 100755
--- a/t/t4137-apply-submodule.sh
+++ b/t/t4137-apply-submodule.sh
@@ -6,13 +6,15 @@ test_description='git apply handling submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 apply_index () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --index -
+	git diff --ignore-submodules=dirty "..$1" >diff &&
+	$2 git apply --index diff
 }
 
 test_submodule_switch_func "apply_index"
 
 apply_3way () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
+	git diff --ignore-submodules=dirty "..$1" >diff
+	$2 git apply --3way diff
 }
 
 test_submodule_switch_func "apply_3way"
diff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh
index 1b179d5f45..8403124ac0 100755
--- a/t/t4255-am-submodule.sh
+++ b/t/t4255-am-submodule.sh
@@ -6,13 +6,15 @@ test_description='git am handling submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 am () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am -
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
+	$2 git am patch
 }
 
 test_submodule_switch_func "am"
 
 am_3way () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am --3way -
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
+	$2 git am --3way patch
 }
 
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
index f911bf631e..cfa40b9251 100755
--- a/t/t5572-pull-submodule.sh
+++ b/t/t5572-pull-submodule.sh
@@ -13,7 +13,7 @@ reset_branch_to_HEAD () {
 
 git_pull () {
 	reset_branch_to_HEAD "$1" &&
-	git pull
+	$2 git pull
 }
 
 # pulls without conflicts
@@ -21,21 +21,21 @@ test_submodule_switch_func "git_pull"
 
 git_pull_ff () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --ff
+	$2 git pull --ff
 }
 
 test_submodule_switch_func "git_pull_ff"
 
 git_pull_ff_only () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --ff-only
+	$2 git pull --ff-only
 }
 
 test_submodule_switch_func "git_pull_ff_only"
 
 git_pull_noff () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --no-ff
+	$2 git pull --no-ff
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
diff --git a/t/t6041-bisect-submodule.sh b/t/t6041-bisect-submodule.sh
index 0e0cdf638d..e213876fcd 100755
--- a/t/t6041-bisect-submodule.sh
+++ b/t/t6041-bisect-submodule.sh
@@ -10,7 +10,10 @@ git_bisect () {
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
 	GOOD=$(git rev-parse --verify HEAD) &&
-	git checkout "$1" &&
+	$2 git checkout "$1"
+}
+
+git_bisect_after () {
 	echo "foo" >bar &&
 	git add bar &&
 	git commit -m "bisect bad" &&
@@ -27,6 +30,6 @@ git_bisect () {
 	git bisect bad $BAD
 }
 
-test_submodule_switch_func "git_bisect"
+test_submodule_switch_func "git_bisect" "git_bisect_after"
 
 test_done
-- 
2.27.0.132.g321788e831

Re: [PATCH v5 0/4] t: replace incorrect test_must_fail usage (part 5)

[Jeff King]

On Tue, Jun 23, 2020 at 06:28:01AM -0400, Denton Liu wrote:

> I've taken your suggestions and rerolled the last patch. Notably, I
> combined $before and $command into a single function and then passed in
> `test_must_fail` as the second argument if it's expected to fail. I also
> decided to use a helper function to reduce the duplication of the
> calling logic but it's still a little verbose because I opted to make
> use of argument passing over global variables where possible.
> 
> One thing that I'm leaning towards is completely removing $after. We can
> combine $after with $command like I did in the first few iterations of
> this series and just write it as:
> 
> 	git_test_thing () {
> 		# before...
> 		$2 git do-the-thing "$1" &&
> 		if test -n "$2"
> 		then
> 			return
> 		fi &&
> 		# after...
> 	}
> 
> but I'm also okay with leaving it as is.

Yeah, as soon as I saw "combined $before and $command", then I wondered
why we needed to split anything (since clearly we're passing
responsibility to handle must_fail to the callback function).

So I think we should either have the full before/command/after split, or
have a single callback function which understands that it has to respect
"$2" as a prefix).

I'm OK with either. I do like both of them better than the magic
$OVERWRITING_FAIL thing, but this series seems to be taking a lot more
energy than I think its outcome is worth. Let's resolve it somehow and
work on something more exciting. ;)

> Also, one problem that's present is that for tests that use
> test_submodule_switch() and test_submodule_forced_switch(), $command
> will be listed as git_test_func, which might make debugging more
> difficult. I think I'd prefer the version we currently have queued for
> this reason (and why I did the whole song and dance of $OVERWRITING_FAIL
> in the first place), but if you think that this is fine, I'm fine with
> this approach as well.

I agree it's not ideal, but I don't think it's too bad. There are lots
of tests that have output you can't just cut-and-paste due to code
generation, helper functions, etc.

-Peff

[PATCH v5.1] lib-submodule-update: pass 'test_must_fail' as an argument

[Denton Liu]

When we run a test helper function in test_submodule_switch_common(), we
sometimes specify a whole helper function as the $command. When we do
this, in some test cases, we just mark the whole function with
`test_must_fail`. However, it's possible that the helper function might
fail earlier or later than expected due to an introduced bug. If this
happens, then the test case will still report as passing but it should
really be marked as failing since it didn't actually display the
intended behaviour.

Instead of invoking `test_must_fail $command`, pass the string
"test_must_fail" as the second argument in case where the git command is
expected to fail.

When $command is a helper function, the parent function calling
test_submodule_switch_common() is test_submodule_switch_func(). For all
test_submodule_switch_func() invocations, increase the granularity of
the argument test helper function by prefixing the git invocation which is
meant to fail with the second argument like this:

	$2 git checkout "$1"

In the other cases, test_submodule_switch() and
test_submodule_forced_switch(), instead of passing in the git command
directly, wrap it using the git_test_func() and pass the git arguments
using the global variable $gitcmd. Unfortunately, since closures aren't
a thing in shell scripts, the global variable is necessary. Another
unfortunate result is that the "git_test_func" will used as the test
case name when $command is printed but it's worth it for the cleaner
code.

Finally, as an added bonus, `test_must_fail` will now only run on git
commands.

An alternate design was considered where the global variable,
$OVERWRITING_FAIL, is set from test_submodule_switch_common() and
exposed to the helper function. This would allow $command to be set to a
more sensible value. However this approach was considered too difficult
to understand due to the fact that using a signalling magical global
variable is too indirect and it's best to keep the magic confined to
within one file.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
This patch replaces v5 4/4. Hopefully, this'll be the last iteration...

 t/lib-submodule-update.sh   | 49 +++++++++++++++++++++++++++----------
 t/t3426-rebase-submodule.sh |  4 +--
 t/t3513-revert-submodule.sh |  6 ++++-
 t/t3906-stash-submodule.sh  |  6 ++++-
 t/t4137-apply-submodule.sh  |  6 +++--
 t/t4255-am-submodule.sh     |  6 +++--
 t/t5572-pull-submodule.sh   |  8 +++---
 t/t6041-bisect-submodule.sh |  6 ++++-
 8 files changed, 65 insertions(+), 26 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 7c3ba1be00..53c5374ea1 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -303,8 +303,12 @@ test_submodule_content () {
 # update" is run. And even then that command doesn't delete the work tree of
 # a removed submodule.
 #
+# The first argument of the callback function will be the name of the submodule.
+#
 # Removing a submodule containing a .git directory must fail even when forced
-# to protect the history!
+# to protect the history! If we are testing this case, the second argument of
+# the callback function will be 'test_must_fail', else it will be the empty
+# string.
 #
 
 # Internal function; use test_submodule_switch_func(), test_submodule_switch(),
@@ -443,7 +447,7 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
-			test_must_fail $command replace_sub1_with_directory &&
+			$command replace_sub1_with_directory test_must_fail &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -456,7 +460,7 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_directory &&
+			$command replace_sub1_with_directory test_must_fail &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -470,7 +474,7 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
-			test_must_fail $command replace_sub1_with_file &&
+			$command replace_sub1_with_file test_must_fail &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -484,7 +488,7 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_file &&
+			$command replace_sub1_with_file test_must_fail &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -559,12 +563,25 @@ test_submodule_switch_common () {
 # conditions, set the appropriate KNOWN_FAILURE_* variable used in the tests
 # below to 1.
 #
-# Use as follows:
+# The first argument of the callback function will be the name of the submodule.
+#
+# Removing a submodule containing a .git directory must fail even when forced
+# to protect the history! If we are testing this case, the second argument of
+# the callback function will be 'test_must_fail', else it will be the empty
+# string.
+#
+# The following example uses `git some-command` as an example command to be
+# tested. It updates the worktree and index to match a target, but not any
+# submodule directories.
 #
 # my_func () {
-#   target=$1
-#   # Do something here that updates the worktree and index to match target,
-#   # but not any submodule directories.
+#   ...prepare for `git some-command` to be run...
+#   $2 git some-command "$1" &&
+#   if test -n "$2"
+#   then
+#     return
+#   fi &&
+#   ...check the state after git some-command is run...
 # }
 # test_submodule_switch_func "my_func"
 test_submodule_switch_func () {
@@ -580,23 +597,29 @@ test_submodule_switch_func () {
 			cd submodule_update &&
 			git branch -t add_sub1 origin/add_sub1 &&
 			>sub1 &&
-			test_must_fail $command add_sub1 &&
+			$command add_sub1 test_must_fail &&
 			test_superproject_content origin/no_submodule &&
 			test_must_be_empty sub1
 		)
 	'
 }
 
+git_test_func () {
+	$2 git $gitcmd "$1"
+}
+
 test_submodule_switch () {
-	test_submodule_switch_func "git $1"
+	gitcmd="$1"
+	test_submodule_switch_func "git_test_func"
 }
 
 # Same as test_submodule_switch(), except that throwing away local changes in
 # the superproject is allowed.
 test_submodule_forced_switch () {
-	command="$1"
+	gitcmd="$1"
+	command="git_test_func"
 	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
-	test_submodule_switch_common "git $command"
+	test_submodule_switch_common "$command"
 
 	# When forced, a file in the superproject does not prevent creating a
 	# submodule of the same name.
diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
index 788605ccc0..dd5daa53d3 100755
--- a/t/t3426-rebase-submodule.sh
+++ b/t/t3426-rebase-submodule.sh
@@ -17,7 +17,7 @@ git_rebase () {
 	git status -su >actual &&
 	ls -1pR * >>actual &&
 	test_cmp expect actual &&
-	git rebase "$1"
+	$2 git rebase "$1"
 }
 
 test_submodule_switch_func "git_rebase"
@@ -35,7 +35,7 @@ git_rebase_interactive () {
 	test_cmp expect actual &&
 	set_fake_editor &&
 	echo "fake-editor.sh" >.git/info/exclude &&
-	git rebase -i "$1"
+	$2 git rebase -i "$1"
 }
 
 test_submodule_switch_func "git_rebase_interactive"
diff --git a/t/t3513-revert-submodule.sh b/t/t3513-revert-submodule.sh
index 95a7f64471..f98e3301b9 100755
--- a/t/t3513-revert-submodule.sh
+++ b/t/t3513-revert-submodule.sh
@@ -15,7 +15,11 @@ git_revert () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
-	git checkout "$1" &&
+	$2 git checkout "$1" &&
+	if test -n "$2"
+	then
+		return
+	fi &&
 	git revert HEAD &&
 	rm -rf * &&
 	tar xf "$TRASH_DIRECTORY/tmp.tar" &&
diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
index 6a7e801ca0..61db4a9886 100755
--- a/t/t3906-stash-submodule.sh
+++ b/t/t3906-stash-submodule.sh
@@ -8,7 +8,11 @@ test_description='stash can handle submodules'
 git_stash () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
-	git read-tree -u -m "$1" &&
+	$2 git read-tree -u -m "$1" &&
+	if test -n "$2"
+	then
+		return
+	fi &&
 	git stash &&
 	git status -su >actual &&
 	ls -1pR * >>actual &&
diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
index b645e303a0..5477d48ffd 100755
--- a/t/t4137-apply-submodule.sh
+++ b/t/t4137-apply-submodule.sh
@@ -6,13 +6,15 @@ test_description='git apply handling submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 apply_index () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --index -
+	git diff --ignore-submodules=dirty "..$1" >diff &&
+	$2 git apply --index diff
 }
 
 test_submodule_switch_func "apply_index"
 
 apply_3way () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
+	git diff --ignore-submodules=dirty "..$1" >diff
+	$2 git apply --3way diff
 }
 
 test_submodule_switch_func "apply_3way"
diff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh
index 1b179d5f45..8403124ac0 100755
--- a/t/t4255-am-submodule.sh
+++ b/t/t4255-am-submodule.sh
@@ -6,13 +6,15 @@ test_description='git am handling submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 am () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am -
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
+	$2 git am patch
 }
 
 test_submodule_switch_func "am"
 
 am_3way () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am --3way -
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
+	$2 git am --3way patch
 }
 
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
index f911bf631e..cfa40b9251 100755
--- a/t/t5572-pull-submodule.sh
+++ b/t/t5572-pull-submodule.sh
@@ -13,7 +13,7 @@ reset_branch_to_HEAD () {
 
 git_pull () {
 	reset_branch_to_HEAD "$1" &&
-	git pull
+	$2 git pull
 }
 
 # pulls without conflicts
@@ -21,21 +21,21 @@ test_submodule_switch_func "git_pull"
 
 git_pull_ff () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --ff
+	$2 git pull --ff
 }
 
 test_submodule_switch_func "git_pull_ff"
 
 git_pull_ff_only () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --ff-only
+	$2 git pull --ff-only
 }
 
 test_submodule_switch_func "git_pull_ff_only"
 
 git_pull_noff () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --no-ff
+	$2 git pull --no-ff
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
diff --git a/t/t6041-bisect-submodule.sh b/t/t6041-bisect-submodule.sh
index 0e0cdf638d..619f181032 100755
--- a/t/t6041-bisect-submodule.sh
+++ b/t/t6041-bisect-submodule.sh
@@ -10,7 +10,11 @@ git_bisect () {
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
 	GOOD=$(git rev-parse --verify HEAD) &&
-	git checkout "$1" &&
+	$2 git checkout "$1" &&
+	if test -n "$2"
+	then
+		return
+	fi &&
 	echo "foo" >bar &&
 	git add bar &&
 	git commit -m "bisect bad" &&
-- 
2.27.0.307.g7979e895e7

Re: [PATCH v5.1] lib-submodule-update: pass 'test_must_fail' as an argument

[Junio C Hamano]

Denton Liu <liu.denton@gmail.com> writes:

> When $command is a helper function, the parent function calling
> test_submodule_switch_common() is test_submodule_switch_func(). For all
> test_submodule_switch_func() invocations, increase the granularity of
> the argument test helper function by prefixing the git invocation which is
> meant to fail with the second argument like this:
>
> 	$2 git checkout "$1"
>
> ...
> Finally, as an added bonus, `test_must_fail` will now only run on git
> commands.

> This patch replaces v5 4/4. Hopefully, this'll be the last iteration...

So three copies of v5.1 are identical replacement for 4/4, they seem.

> -			test_must_fail $command replace_sub1_with_directory &&
> +			$command replace_sub1_with_directory test_must_fail &&

> +# The following example uses `git some-command` as an example command to be
> +# tested. It updates the worktree and index to match a target, but not any
> +# submodule directories.
>  #
>  # my_func () {
> +#   ...prepare for `git some-command` to be run...
> +#   $2 git some-command "$1" &&
> +#   if test -n "$2"
> +#   then
> +#     return
> +#   fi &&

So, in practice $2 MUST BE either test_must_fail OR an empty
string.  Feeding the my_func anything other than these two values is
a BUG.  I wonder if we can somehow make sure to reduce mental burden
by readers.  Perhaps

> +git_test_func () {

	may_only_be_test_must_fail "$2"

> +	$2 git $gitcmd "$1"
> +}

where

	may_only_be_test_must_fail () {
		test -z "$1" || test "$1" = test_must_fail || die
	}

or something.  The idea is to make sure that this extra helper can
be named in such a way that it can serve as a documention to help
readers of "git_test_func ()" and others about "$2".

> diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
> index 788605ccc0..dd5daa53d3 100755
> --- a/t/t3426-rebase-submodule.sh
> +++ b/t/t3426-rebase-submodule.sh
> @@ -17,7 +17,7 @@ git_rebase () {
>  	git status -su >actual &&
>  	ls -1pR * >>actual &&
>  	test_cmp expect actual &&
> -	git rebase "$1"
> +	$2 git rebase "$1"

Likewise here.

Even without such a change this round reads much better than the
previous ones.

> An alternate design was considered where the global variable,
> $OVERWRITING_FAIL, is set from test_submodule_switch_common() and
> exposed to the helper function.  This would allow $command to be
> set to a more sensible value. However ...

By the way, I do not think this paragraph adds much value to the
message.  It wasn't clear how OVERWRITING_FAIL wanted to achieve its
goal back in its own commit, and without any actual code but with
only the four lines to go by, there is no chance that you can
convince anybody how $command could have been "more sensible value".

Thanks.

Re: [PATCH v5.1] lib-submodule-update: pass 'test_must_fail' as an argument

[Đoàn Trần Công Danh]

On 2020-06-23 16:21:20-0400, Denton Liu <liu.denton@gmail.com> wrote:
> diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
> index b645e303a0..5477d48ffd 100755
> --- a/t/t4137-apply-submodule.sh
> +++ b/t/t4137-apply-submodule.sh
> @@ -6,13 +6,15 @@ test_description='git apply handling submodules'
>  . "$TEST_DIRECTORY"/lib-submodule-update.sh
>  
>  apply_index () {
> -	git diff --ignore-submodules=dirty "..$1" | git apply --index -
> +	git diff --ignore-submodules=dirty "..$1" >diff &&
> +	$2 git apply --index diff
>  }
>  
>  test_submodule_switch_func "apply_index"
>  
>  apply_3way () {
> -	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
> +	git diff --ignore-submodules=dirty "..$1" >diff
> +	$2 git apply --3way diff

It seems like you forgot to chain commands with '&&' here.

>  }
>  
>  test_submodule_switch_func "apply_3way"

-- 
Danh

[PATCH v5.2] lib-submodule-update: pass 'test_must_fail' as an argument

[Denton Liu]

When we run a test helper function in test_submodule_switch_common(), we
sometimes specify a whole helper function as the $command. When we do
this, in some test cases, we just mark the whole function with
`test_must_fail`. However, it's possible that the helper function might
fail earlier or later than expected due to an introduced bug. If this
happens, then the test case will still report as passing but it should
really be marked as failing since it didn't actually display the
intended behaviour.

Instead of invoking `test_must_fail $command`, pass the string
"test_must_fail" as the second argument in case where the git command is
expected to fail.

When $command is a helper function, the parent function calling
test_submodule_switch_common() is test_submodule_switch_func(). For all
test_submodule_switch_func() invocations, increase the granularity of
the argument test helper function by prefixing the git invocation which is
meant to fail with the second argument like this:

	$2 git checkout "$1"

In the other cases, test_submodule_switch() and
test_submodule_forced_switch(), instead of passing in the git command
directly, wrap it using the git_test_func() and pass the git arguments
using the global variable $gitcmd. Unfortunately, since closures aren't
a thing in shell scripts, the global variable is necessary. Another
unfortunate result is that the "git_test_func" will used as the test
case name when $command is printed but it's worth it for the cleaner
code.

Finally, as an added bonus, `test_must_fail` will now only run on git
commands.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
Changes since v5.1:

* Fix broken &&-chain

* Add may_only_be_test_must_fail() checks

* Remove $OVERWRITING_FAIL from the commit message

 t/lib-submodule-update.sh   | 55 ++++++++++++++++++++++++++++---------
 t/t3426-rebase-submodule.sh |  6 ++--
 t/t3513-revert-submodule.sh |  7 ++++-
 t/t3906-stash-submodule.sh  |  7 ++++-
 t/t4137-apply-submodule.sh  |  8 ++++--
 t/t4255-am-submodule.sh     |  8 ++++--
 t/t5572-pull-submodule.sh   | 12 +++++---
 t/t6041-bisect-submodule.sh |  7 ++++-
 8 files changed, 84 insertions(+), 26 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 7c3ba1be00..8e9f310bcb 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -303,8 +303,12 @@ test_submodule_content () {
 # update" is run. And even then that command doesn't delete the work tree of
 # a removed submodule.
 #
+# The first argument of the callback function will be the name of the submodule.
+#
 # Removing a submodule containing a .git directory must fail even when forced
-# to protect the history!
+# to protect the history! If we are testing this case, the second argument of
+# the callback function will be 'test_must_fail', else it will be the empty
+# string.
 #
 
 # Internal function; use test_submodule_switch_func(), test_submodule_switch(),
@@ -443,7 +447,7 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
-			test_must_fail $command replace_sub1_with_directory &&
+			$command replace_sub1_with_directory test_must_fail &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -456,7 +460,7 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_directory &&
+			$command replace_sub1_with_directory test_must_fail &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -470,7 +474,7 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
-			test_must_fail $command replace_sub1_with_file &&
+			$command replace_sub1_with_file test_must_fail &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -484,7 +488,7 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_file &&
+			$command replace_sub1_with_file test_must_fail &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -559,12 +563,25 @@ test_submodule_switch_common () {
 # conditions, set the appropriate KNOWN_FAILURE_* variable used in the tests
 # below to 1.
 #
-# Use as follows:
+# The first argument of the callback function will be the name of the submodule.
+#
+# Removing a submodule containing a .git directory must fail even when forced
+# to protect the history! If we are testing this case, the second argument of
+# the callback function will be 'test_must_fail', else it will be the empty
+# string.
+#
+# The following example uses `git some-command` as an example command to be
+# tested. It updates the worktree and index to match a target, but not any
+# submodule directories.
 #
 # my_func () {
-#   target=$1
-#   # Do something here that updates the worktree and index to match target,
-#   # but not any submodule directories.
+#   ...prepare for `git some-command` to be run...
+#   $2 git some-command "$1" &&
+#   if test -n "$2"
+#   then
+#     return
+#   fi &&
+#   ...check the state after git some-command is run...
 # }
 # test_submodule_switch_func "my_func"
 test_submodule_switch_func () {
@@ -580,23 +597,35 @@ test_submodule_switch_func () {
 			cd submodule_update &&
 			git branch -t add_sub1 origin/add_sub1 &&
 			>sub1 &&
-			test_must_fail $command add_sub1 &&
+			$command add_sub1 test_must_fail &&
 			test_superproject_content origin/no_submodule &&
 			test_must_be_empty sub1
 		)
 	'
 }
 
+# Ensures that the that the arg either contains "test_must_fail" or is empty.
+may_only_be_test_must_fail () {
+	test -z "$1" || test "$1" = test_must_fail || die
+}
+
+git_test_func () {
+	may_only_be_test_must_fail "$2" &&
+	$2 git $gitcmd "$1"
+}
+
 test_submodule_switch () {
-	test_submodule_switch_func "git $1"
+	gitcmd="$1"
+	test_submodule_switch_func "git_test_func"
 }
 
 # Same as test_submodule_switch(), except that throwing away local changes in
 # the superproject is allowed.
 test_submodule_forced_switch () {
-	command="$1"
+	gitcmd="$1"
+	command="git_test_func"
 	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
-	test_submodule_switch_common "git $command"
+	test_submodule_switch_common "$command"
 
 	# When forced, a file in the superproject does not prevent creating a
 	# submodule of the same name.
diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
index 788605ccc0..0ad3a07bf4 100755
--- a/t/t3426-rebase-submodule.sh
+++ b/t/t3426-rebase-submodule.sh
@@ -17,7 +17,8 @@ git_rebase () {
 	git status -su >actual &&
 	ls -1pR * >>actual &&
 	test_cmp expect actual &&
-	git rebase "$1"
+	may_only_be_test_must_fail "$2" &&
+	$2 git rebase "$1"
 }
 
 test_submodule_switch_func "git_rebase"
@@ -35,7 +36,8 @@ git_rebase_interactive () {
 	test_cmp expect actual &&
 	set_fake_editor &&
 	echo "fake-editor.sh" >.git/info/exclude &&
-	git rebase -i "$1"
+	may_only_be_test_must_fail "$2" &&
+	$2 git rebase -i "$1"
 }
 
 test_submodule_switch_func "git_rebase_interactive"
diff --git a/t/t3513-revert-submodule.sh b/t/t3513-revert-submodule.sh
index 95a7f64471..a759f12cbb 100755
--- a/t/t3513-revert-submodule.sh
+++ b/t/t3513-revert-submodule.sh
@@ -15,7 +15,12 @@ git_revert () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
-	git checkout "$1" &&
+	may_only_be_test_must_fail "$2" &&
+	$2 git checkout "$1" &&
+	if test -n "$2"
+	then
+		return
+	fi &&
 	git revert HEAD &&
 	rm -rf * &&
 	tar xf "$TRASH_DIRECTORY/tmp.tar" &&
diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
index 6a7e801ca0..a52e53dd2d 100755
--- a/t/t3906-stash-submodule.sh
+++ b/t/t3906-stash-submodule.sh
@@ -8,7 +8,12 @@ test_description='stash can handle submodules'
 git_stash () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
-	git read-tree -u -m "$1" &&
+	may_only_be_test_must_fail "$2" &&
+	$2 git read-tree -u -m "$1" &&
+	if test -n "$2"
+	then
+		return
+	fi &&
 	git stash &&
 	git status -su >actual &&
 	ls -1pR * >>actual &&
diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
index b645e303a0..07d5262537 100755
--- a/t/t4137-apply-submodule.sh
+++ b/t/t4137-apply-submodule.sh
@@ -6,13 +6,17 @@ test_description='git apply handling submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 apply_index () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --index -
+	git diff --ignore-submodules=dirty "..$1" >diff &&
+	may_only_be_test_must_fail "$2" &&
+	$2 git apply --index diff
 }
 
 test_submodule_switch_func "apply_index"
 
 apply_3way () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
+	git diff --ignore-submodules=dirty "..$1" >diff &&
+	may_only_be_test_must_fail "$2" &&
+	$2 git apply --3way diff
 }
 
 test_submodule_switch_func "apply_3way"
diff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh
index 1b179d5f45..a7ba08f728 100755
--- a/t/t4255-am-submodule.sh
+++ b/t/t4255-am-submodule.sh
@@ -6,13 +6,17 @@ test_description='git am handling submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 am () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am -
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
+	may_only_be_test_must_fail "$2" &&
+	$2 git am patch
 }
 
 test_submodule_switch_func "am"
 
 am_3way () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am --3way -
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
+	may_only_be_test_must_fail "$2" &&
+	$2 git am --3way patch
 }
 
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
index f911bf631e..1d75e3b12b 100755
--- a/t/t5572-pull-submodule.sh
+++ b/t/t5572-pull-submodule.sh
@@ -13,7 +13,8 @@ reset_branch_to_HEAD () {
 
 git_pull () {
 	reset_branch_to_HEAD "$1" &&
-	git pull
+	may_only_be_test_must_fail "$2" &&
+	$2 git pull
 }
 
 # pulls without conflicts
@@ -21,21 +22,24 @@ test_submodule_switch_func "git_pull"
 
 git_pull_ff () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --ff
+	may_only_be_test_must_fail "$2" &&
+	$2 git pull --ff
 }
 
 test_submodule_switch_func "git_pull_ff"
 
 git_pull_ff_only () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --ff-only
+	may_only_be_test_must_fail "$2" &&
+	$2 git pull --ff-only
 }
 
 test_submodule_switch_func "git_pull_ff_only"
 
 git_pull_noff () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --no-ff
+	may_only_be_test_must_fail "$2" &&
+	$2 git pull --no-ff
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
diff --git a/t/t6041-bisect-submodule.sh b/t/t6041-bisect-submodule.sh
index 0e0cdf638d..df1eff0fb8 100755
--- a/t/t6041-bisect-submodule.sh
+++ b/t/t6041-bisect-submodule.sh
@@ -10,7 +10,12 @@ git_bisect () {
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
 	GOOD=$(git rev-parse --verify HEAD) &&
-	git checkout "$1" &&
+	may_only_be_test_must_fail "$2" &&
+	$2 git checkout "$1" &&
+	if test -n "$2"
+	then
+		return
+	fi &&
 	echo "foo" >bar &&
 	git add bar &&
 	git commit -m "bisect bad" &&
-- 
2.27.0.307.g7979e895e7

Re: [PATCH v5.2] lib-submodule-update: pass 'test_must_fail' as an argument

[Junio C Hamano]

Denton Liu <liu.denton@gmail.com> writes:

> +# Ensures that the that the arg either contains "test_must_fail" or is empty.
> +may_only_be_test_must_fail () {
> +	test -z "$1" || test "$1" = test_must_fail || die

No error message?  I guess it is OK as this is to only catch a bug
in the test script, not a bug in Git being tested.

> @@ -17,7 +17,8 @@ git_rebase () {
>  	git status -su >actual &&
>  	ls -1pR * >>actual &&
>  	test_cmp expect actual &&
> -	git rebase "$1"
> +	may_only_be_test_must_fail "$2" &&
> +	$2 git rebase "$1"
>  }

I would have expected the sanity check for incoming arguments would
be done at the very beginning, but I guess it is OK (the same
comment applies to all the others).

> @@ -15,7 +15,12 @@ git_revert () {
>  	git status -su >expect &&
>  	ls -1pR * >>expect &&
>  	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
> -	git checkout "$1" &&
> +	may_only_be_test_must_fail "$2" &&
> +	$2 git checkout "$1" &&
> +	if test -n "$2"
> +	then
> +		return
> +	fi &&

The addition to "git_rebase" does not need this "if must-fail was
given, return early" because it was at the end of the helper, but
here we have some post procedure that relies on the checkout to
be successful, so after correctly failing the checkout we return.

It is somewhat tricky, but OK.  The same for the next one.

> diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
> index 6a7e801ca0..a52e53dd2d 100755
> --- a/t/t3906-stash-submodule.sh
> +++ b/t/t3906-stash-submodule.sh
> @@ -8,7 +8,12 @@ test_description='stash can handle submodules'
>  git_stash () {
>  	git status -su >expect &&
>  	ls -1pR * >>expect &&
> -	git read-tree -u -m "$1" &&
> +	may_only_be_test_must_fail "$2" &&
> +	$2 git read-tree -u -m "$1" &&
> +	if test -n "$2"
> +	then
> +		return
> +	fi &&
>  	git stash &&
>  	git status -su >actual &&
>  	ls -1pR * >>actual &&

Looking good.  Will replace.

Thanks.

[PATCH v5.1] lib-submodule-update: pass 'test_must_fail' as an argument

[Denton Liu]

When we run a test helper function in test_submodule_switch_common(), we
sometimes specify a whole helper function as the $command. When we do
this, in some test cases, we just mark the whole function with
`test_must_fail`. However, it's possible that the helper function might
fail earlier or later than expected due to an introduced bug. If this
happens, then the test case will still report as passing but it should
really be marked as failing since it didn't actually display the
intended behaviour.

Instead of invoking `test_must_fail $command`, pass the string
"test_must_fail" as the second argument in case where the git command is
expected to fail.

When $command is a helper function, the parent function calling
test_submodule_switch_common() is test_submodule_switch_func(). For all
test_submodule_switch_func() invocations, increase the granularity of
the argument test helper function by prefixing the git invocation which is
meant to fail with the second argument like this:

	$2 git checkout "$1"

In the other cases, test_submodule_switch() and
test_submodule_forced_switch(), instead of passing in the git command
directly, wrap it using the git_test_func() and pass the git arguments
using the global variable $gitcmd. Unfortunately, since closures aren't
a thing in shell scripts, the global variable is necessary. Another
unfortunate result is that the "git_test_func" will used as the test
case name when $command is printed but it's worth it for the cleaner
code.

Finally, as an added bonus, `test_must_fail` will now only run on git
commands.

An alternate design was considered where the global variable,
$OVERWRITING_FAIL, is set from test_submodule_switch_common() and
exposed to the helper function. This would allow $command to be set to a
more sensible value. However this approach was considered too difficult
to understand due to the fact that using a signalling magical global
variable is too indirect and it's best to keep the magic confined to
within one file.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
[I dropped the CC list because I've already sent out two copies of this
patch but, for some reason, it isn't hitting the list. I don't want to
spam anyone while I'm experimenting.]

This patch replaces v5 4/4. Hopefully, this'll be the last iteration...

 t/lib-submodule-update.sh   | 49 +++++++++++++++++++++++++++----------
 t/t3426-rebase-submodule.sh |  4 +--
 t/t3513-revert-submodule.sh |  6 ++++-
 t/t3906-stash-submodule.sh  |  6 ++++-
 t/t4137-apply-submodule.sh  |  6 +++--
 t/t4255-am-submodule.sh     |  6 +++--
 t/t5572-pull-submodule.sh   |  8 +++---
 t/t6041-bisect-submodule.sh |  6 ++++-
 8 files changed, 65 insertions(+), 26 deletions(-)

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 7c3ba1be00..53c5374ea1 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -303,8 +303,12 @@ test_submodule_content () {
 # update" is run. And even then that command doesn't delete the work tree of
 # a removed submodule.
 #
+# The first argument of the callback function will be the name of the submodule.
+#
 # Removing a submodule containing a .git directory must fail even when forced
-# to protect the history!
+# to protect the history! If we are testing this case, the second argument of
+# the callback function will be 'test_must_fail', else it will be the empty
+# string.
 #
 
 # Internal function; use test_submodule_switch_func(), test_submodule_switch(),
@@ -443,7 +447,7 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
-			test_must_fail $command replace_sub1_with_directory &&
+			$command replace_sub1_with_directory test_must_fail &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -456,7 +460,7 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_directory &&
+			$command replace_sub1_with_directory test_must_fail &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -470,7 +474,7 @@ test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
-			test_must_fail $command replace_sub1_with_file &&
+			$command replace_sub1_with_file test_must_fail &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -484,7 +488,7 @@ test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_file &&
+			$command replace_sub1_with_file test_must_fail &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -559,12 +563,25 @@ test_submodule_switch_common () {
 # conditions, set the appropriate KNOWN_FAILURE_* variable used in the tests
 # below to 1.
 #
-# Use as follows:
+# The first argument of the callback function will be the name of the submodule.
+#
+# Removing a submodule containing a .git directory must fail even when forced
+# to protect the history! If we are testing this case, the second argument of
+# the callback function will be 'test_must_fail', else it will be the empty
+# string.
+#
+# The following example uses `git some-command` as an example command to be
+# tested. It updates the worktree and index to match a target, but not any
+# submodule directories.
 #
 # my_func () {
-#   target=$1
-#   # Do something here that updates the worktree and index to match target,
-#   # but not any submodule directories.
+#   ...prepare for `git some-command` to be run...
+#   $2 git some-command "$1" &&
+#   if test -n "$2"
+#   then
+#     return
+#   fi &&
+#   ...check the state after git some-command is run...
 # }
 # test_submodule_switch_func "my_func"
 test_submodule_switch_func () {
@@ -580,23 +597,29 @@ test_submodule_switch_func () {
 			cd submodule_update &&
 			git branch -t add_sub1 origin/add_sub1 &&
 			>sub1 &&
-			test_must_fail $command add_sub1 &&
+			$command add_sub1 test_must_fail &&
 			test_superproject_content origin/no_submodule &&
 			test_must_be_empty sub1
 		)
 	'
 }
 
+git_test_func () {
+	$2 git $gitcmd "$1"
+}
+
 test_submodule_switch () {
-	test_submodule_switch_func "git $1"
+	gitcmd="$1"
+	test_submodule_switch_func "git_test_func"
 }
 
 # Same as test_submodule_switch(), except that throwing away local changes in
 # the superproject is allowed.
 test_submodule_forced_switch () {
-	command="$1"
+	gitcmd="$1"
+	command="git_test_func"
 	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
-	test_submodule_switch_common "git $command"
+	test_submodule_switch_common "$command"
 
 	# When forced, a file in the superproject does not prevent creating a
 	# submodule of the same name.
diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
index 788605ccc0..dd5daa53d3 100755
--- a/t/t3426-rebase-submodule.sh
+++ b/t/t3426-rebase-submodule.sh
@@ -17,7 +17,7 @@ git_rebase () {
 	git status -su >actual &&
 	ls -1pR * >>actual &&
 	test_cmp expect actual &&
-	git rebase "$1"
+	$2 git rebase "$1"
 }
 
 test_submodule_switch_func "git_rebase"
@@ -35,7 +35,7 @@ git_rebase_interactive () {
 	test_cmp expect actual &&
 	set_fake_editor &&
 	echo "fake-editor.sh" >.git/info/exclude &&
-	git rebase -i "$1"
+	$2 git rebase -i "$1"
 }
 
 test_submodule_switch_func "git_rebase_interactive"
diff --git a/t/t3513-revert-submodule.sh b/t/t3513-revert-submodule.sh
index 95a7f64471..f98e3301b9 100755
--- a/t/t3513-revert-submodule.sh
+++ b/t/t3513-revert-submodule.sh
@@ -15,7 +15,11 @@ git_revert () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
-	git checkout "$1" &&
+	$2 git checkout "$1" &&
+	if test -n "$2"
+	then
+		return
+	fi &&
 	git revert HEAD &&
 	rm -rf * &&
 	tar xf "$TRASH_DIRECTORY/tmp.tar" &&
diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
index 6a7e801ca0..61db4a9886 100755
--- a/t/t3906-stash-submodule.sh
+++ b/t/t3906-stash-submodule.sh
@@ -8,7 +8,11 @@ test_description='stash can handle submodules'
 git_stash () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
-	git read-tree -u -m "$1" &&
+	$2 git read-tree -u -m "$1" &&
+	if test -n "$2"
+	then
+		return
+	fi &&
 	git stash &&
 	git status -su >actual &&
 	ls -1pR * >>actual &&
diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
index b645e303a0..5477d48ffd 100755
--- a/t/t4137-apply-submodule.sh
+++ b/t/t4137-apply-submodule.sh
@@ -6,13 +6,15 @@ test_description='git apply handling submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 apply_index () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --index -
+	git diff --ignore-submodules=dirty "..$1" >diff &&
+	$2 git apply --index diff
 }
 
 test_submodule_switch_func "apply_index"
 
 apply_3way () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
+	git diff --ignore-submodules=dirty "..$1" >diff
+	$2 git apply --3way diff
 }
 
 test_submodule_switch_func "apply_3way"
diff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh
index 1b179d5f45..8403124ac0 100755
--- a/t/t4255-am-submodule.sh
+++ b/t/t4255-am-submodule.sh
@@ -6,13 +6,15 @@ test_description='git am handling submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 am () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am -
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
+	$2 git am patch
 }
 
 test_submodule_switch_func "am"
 
 am_3way () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am --3way -
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
+	$2 git am --3way patch
 }
 
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
index f911bf631e..cfa40b9251 100755
--- a/t/t5572-pull-submodule.sh
+++ b/t/t5572-pull-submodule.sh
@@ -13,7 +13,7 @@ reset_branch_to_HEAD () {
 
 git_pull () {
 	reset_branch_to_HEAD "$1" &&
-	git pull
+	$2 git pull
 }
 
 # pulls without conflicts
@@ -21,21 +21,21 @@ test_submodule_switch_func "git_pull"
 
 git_pull_ff () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --ff
+	$2 git pull --ff
 }
 
 test_submodule_switch_func "git_pull_ff"
 
 git_pull_ff_only () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --ff-only
+	$2 git pull --ff-only
 }
 
 test_submodule_switch_func "git_pull_ff_only"
 
 git_pull_noff () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --no-ff
+	$2 git pull --no-ff
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
diff --git a/t/t6041-bisect-submodule.sh b/t/t6041-bisect-submodule.sh
index 0e0cdf638d..619f181032 100755
--- a/t/t6041-bisect-submodule.sh
+++ b/t/t6041-bisect-submodule.sh
@@ -10,7 +10,11 @@ git_bisect () {
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
 	GOOD=$(git rev-parse --verify HEAD) &&
-	git checkout "$1" &&
+	$2 git checkout "$1" &&
+	if test -n "$2"
+	then
+		return
+	fi &&
 	echo "foo" >bar &&
 	git add bar &&
 	git commit -m "bisect bad" &&
-- 
2.27.0.307.g7979e895e7