git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / code / Atom feed
* [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh
@ 2021-06-14 10:48 Ævar Arnfjörð Bjarmason
  2021-06-14 10:48 ` [PATCH 1/8] test-lib tests: move "run_sub_test" to a new lib-subtest.sh Ævar Arnfjörð Bjarmason
                   ` (10 more replies)
  0 siblings, 11 replies; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:48 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Felipe Contreras, Adam Spiers,
	Thomas Rast, Ilya Bobyr, Ævar Arnfjörð Bjarmason

This refactors the testing of test-lib.sh itself in t0000 into a
lib-subtest.sh, fixing duplicate setup, bugs and various shell nits
along the way.

This was waiting on my now-merged 2019256717d (Merge branch
'ab/test-lib-updates', 2021-06-14).

There's a conflict here with Felipe's proposed
http://lore.kernel.org/git/20210609170520.67014-1-felipe.contreras@gmail.com,
but it's just a matter of moving the fix around and solving a trivial
conflict. I thought it was better to submit this now anyway, while
this code is fresh in people's minds.

I'm hoping Felipe will take some variant of my
http://lore.kernel.org/git/8735tk22if.fsf@evledraar.gmail.com and
perhaps review this series & base his patch on top of these changes,
but alternatively they can go in independently with Junio solving the
conflict.

Ævar Arnfjörð Bjarmason (8):
  test-lib tests: move "run_sub_test" to a new lib-subtest.sh
  test-lib tests: split up "write and run" into two functions
  test-lib tests: stop using a subshell in write_sub_test_lib_test()
  test-lib tests: don't provide a description for the sub-tests
  test-lib tests: get rid of copy/pasted mock test code
  test-lib tests: avoid subshell for "test_cmp" for readability
  test-lib tests: refactor common part of check_sub_test_lib_test*()
  test-lib tests: assert 1 exit code, not non-zero

 t/lib-subtest.sh | 105 ++++++++++++
 t/t0000-basic.sh | 419 +++++++++++++----------------------------------
 2 files changed, 219 insertions(+), 305 deletions(-)
 create mode 100644 t/lib-subtest.sh

-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [PATCH 1/8] test-lib tests: move "run_sub_test" to a new lib-subtest.sh
  2021-06-14 10:48 [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:48 ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:48 ` [PATCH 2/8] test-lib tests: split up "write and run" into two functions Ævar Arnfjörð Bjarmason
                   ` (9 subsequent siblings)
  10 siblings, 0 replies; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:48 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Felipe Contreras, Adam Spiers,
	Thomas Rast, Ilya Bobyr, Ævar Arnfjörð Bjarmason

Move the "check_sub_test_lib_test()" and its sister functions to a new
lib-subtest.sh.

In the future (not in this series) I'd like to test test-lib's output
in a more targeted and smaller test, and I'll need these functions to
do that.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/lib-subtest.sh | 64 ++++++++++++++++++++++++++++++++++++++++++++++
 t/t0000-basic.sh | 66 +-----------------------------------------------
 2 files changed, 65 insertions(+), 65 deletions(-)
 create mode 100644 t/lib-subtest.sh

diff --git a/t/lib-subtest.sh b/t/lib-subtest.sh
new file mode 100644
index 00000000000..0c613e00da3
--- /dev/null
+++ b/t/lib-subtest.sh
@@ -0,0 +1,64 @@
+_run_sub_test_lib_test_common () {
+	neg="$1" name="$2" descr="$3" # stdin is the body of the test code
+	shift 3
+	mkdir "$name" &&
+	(
+		# Pretend we're not running under a test harness, whether we
+		# are or not. The test-lib output depends on the setting of
+		# this variable, so we need a stable setting under which to run
+		# the sub-test.
+		sane_unset HARNESS_ACTIVE &&
+		cd "$name" &&
+		write_script "$name.sh" "$TEST_SHELL_PATH" <<-EOF &&
+		test_description='$descr (run in sub test-lib)
+
+		This is run in a sub test-lib so that we do not get incorrect
+		passing metrics
+		'
+
+		# Point to the t/test-lib.sh, which isn't in ../ as usual
+		. "\$TEST_DIRECTORY"/test-lib.sh
+		EOF
+		cat >>"$name.sh" &&
+		export TEST_DIRECTORY &&
+		TEST_OUTPUT_DIRECTORY=$(pwd) &&
+		export TEST_OUTPUT_DIRECTORY &&
+		sane_unset GIT_TEST_FAIL_PREREQS &&
+		if test -z "$neg"
+		then
+			./"$name.sh" "$@" >out 2>err
+		else
+			! ./"$name.sh" "$@" >out 2>err
+		fi
+	)
+}
+
+run_sub_test_lib_test () {
+	_run_sub_test_lib_test_common '' "$@"
+}
+
+run_sub_test_lib_test_err () {
+	_run_sub_test_lib_test_common '!' "$@"
+}
+
+check_sub_test_lib_test () {
+	name="$1" # stdin is the expected output from the test
+	(
+		cd "$name" &&
+		test_must_be_empty err &&
+		sed -e 's/^> //' -e 's/Z$//' >expect &&
+		test_cmp expect out
+	)
+}
+
+check_sub_test_lib_test_err () {
+	name="$1" # stdin is the expected output from the test
+	# expected error output is in descriptor 3
+	(
+		cd "$name" &&
+		sed -e 's/^> //' -e 's/Z$//' >expect.out &&
+		test_cmp expect.out out &&
+		sed -e 's/^> //' -e 's/Z$//' <&3 >expect.err &&
+		test_cmp expect.err err
+	)
+}
diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index 2c6e34b9478..72809477645 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -19,6 +19,7 @@ modification *should* take notice and update the test vectors here.
 '
 
 . ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-subtest.sh
 
 try_local_xy () {
 	local x="local" y="alsolocal" &&
@@ -66,71 +67,6 @@ test_expect_success 'success is reported like this' '
 	:
 '
 
-_run_sub_test_lib_test_common () {
-	neg="$1" name="$2" descr="$3" # stdin is the body of the test code
-	shift 3
-	mkdir "$name" &&
-	(
-		# Pretend we're not running under a test harness, whether we
-		# are or not. The test-lib output depends on the setting of
-		# this variable, so we need a stable setting under which to run
-		# the sub-test.
-		sane_unset HARNESS_ACTIVE &&
-		cd "$name" &&
-		write_script "$name.sh" "$TEST_SHELL_PATH" <<-EOF &&
-		test_description='$descr (run in sub test-lib)
-
-		This is run in a sub test-lib so that we do not get incorrect
-		passing metrics
-		'
-
-		# Point to the t/test-lib.sh, which isn't in ../ as usual
-		. "\$TEST_DIRECTORY"/test-lib.sh
-		EOF
-		cat >>"$name.sh" &&
-		export TEST_DIRECTORY &&
-		TEST_OUTPUT_DIRECTORY=$(pwd) &&
-		export TEST_OUTPUT_DIRECTORY &&
-		sane_unset GIT_TEST_FAIL_PREREQS &&
-		if test -z "$neg"
-		then
-			./"$name.sh" "$@" >out 2>err
-		else
-			! ./"$name.sh" "$@" >out 2>err
-		fi
-	)
-}
-
-run_sub_test_lib_test () {
-	_run_sub_test_lib_test_common '' "$@"
-}
-
-run_sub_test_lib_test_err () {
-	_run_sub_test_lib_test_common '!' "$@"
-}
-
-check_sub_test_lib_test () {
-	name="$1" # stdin is the expected output from the test
-	(
-		cd "$name" &&
-		test_must_be_empty err &&
-		sed -e 's/^> //' -e 's/Z$//' >expect &&
-		test_cmp expect out
-	)
-}
-
-check_sub_test_lib_test_err () {
-	name="$1" # stdin is the expected output from the test
-	# expected error output is in descriptor 3
-	(
-		cd "$name" &&
-		sed -e 's/^> //' -e 's/Z$//' >expect.out &&
-		test_cmp expect.out out &&
-		sed -e 's/^> //' -e 's/Z$//' <&3 >expect.err &&
-		test_cmp expect.err err
-	)
-}
-
 test_expect_success 'pretend we have a fully passing test suite' '
 	run_sub_test_lib_test full-pass "3 passing tests" <<-\EOF &&
 	for i in 1 2 3
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [PATCH 2/8] test-lib tests: split up "write and run" into two functions
  2021-06-14 10:48 [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh Ævar Arnfjörð Bjarmason
  2021-06-14 10:48 ` [PATCH 1/8] test-lib tests: move "run_sub_test" to a new lib-subtest.sh Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:48 ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:48 ` [PATCH 3/8] test-lib tests: stop using a subshell in write_sub_test_lib_test() Ævar Arnfjörð Bjarmason
                   ` (8 subsequent siblings)
  10 siblings, 0 replies; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:48 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Felipe Contreras, Adam Spiers,
	Thomas Rast, Ilya Bobyr, Ævar Arnfjörð Bjarmason

Refactor the function to write and run tests of the test-lib.sh output
into two functions.

When this was added back in 565b6fa87bb (tests: refactor mechanics of
testing in a sub test-lib, 2012-12-16) there was no reason to do this,
but since we started supporting test arguments in
517cd55fd51 (test-lib: self-test that --verbose works, 2013-06-23)
we've started to write out duplicate tests simply to test different
arguments, now we'll be able to re-use them.

This change doesn't consolidate any of those tests yet, it just makes
it possible to do so. All the changes in t0000-basic.sh are a simple
search-replacement.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/lib-subtest.sh | 39 +++++++++++++++++++------
 t/t0000-basic.sh | 74 ++++++++++++++++++++++++------------------------
 2 files changed, 67 insertions(+), 46 deletions(-)

diff --git a/t/lib-subtest.sh b/t/lib-subtest.sh
index 0c613e00da3..0ece41c0e5f 100644
--- a/t/lib-subtest.sh
+++ b/t/lib-subtest.sh
@@ -1,13 +1,7 @@
-_run_sub_test_lib_test_common () {
-	neg="$1" name="$2" descr="$3" # stdin is the body of the test code
-	shift 3
+write_sub_test_lib_test () {
+	name="$1" descr="$2" # stdin is the body of the test code
 	mkdir "$name" &&
 	(
-		# Pretend we're not running under a test harness, whether we
-		# are or not. The test-lib output depends on the setting of
-		# this variable, so we need a stable setting under which to run
-		# the sub-test.
-		sane_unset HARNESS_ACTIVE &&
 		cd "$name" &&
 		write_script "$name.sh" "$TEST_SHELL_PATH" <<-EOF &&
 		test_description='$descr (run in sub test-lib)
@@ -19,7 +13,22 @@ _run_sub_test_lib_test_common () {
 		# Point to the t/test-lib.sh, which isn't in ../ as usual
 		. "\$TEST_DIRECTORY"/test-lib.sh
 		EOF
-		cat >>"$name.sh" &&
+		cat >>"$name.sh"
+	)
+}
+
+_run_sub_test_lib_test_common () {
+	neg="$1" name="$2" descr="$3" # stdin is the body of the test code
+	shift 3
+	(
+		cd "$name" &&
+
+		# Pretend we're not running under a test harness, whether we
+		# are or not. The test-lib output depends on the setting of
+		# this variable, so we need a stable setting under which to run
+		# the sub-test.
+		sane_unset HARNESS_ACTIVE &&
+
 		export TEST_DIRECTORY &&
 		TEST_OUTPUT_DIRECTORY=$(pwd) &&
 		export TEST_OUTPUT_DIRECTORY &&
@@ -33,6 +42,18 @@ _run_sub_test_lib_test_common () {
 	)
 }
 
+write_and_run_sub_test_lib_test () {
+	name="$1" descr="$2" # stdin is the body of the test code
+	write_sub_test_lib_test "$@" || return 1
+	_run_sub_test_lib_test_common '' "$@"
+}
+
+write_and_run_sub_test_lib_test_err () {
+	name="$1" descr="$2" # stdin is the body of the test code
+	write_sub_test_lib_test "$@" || return 1
+	_run_sub_test_lib_test_common '!' "$@"
+}
+
 run_sub_test_lib_test () {
 	_run_sub_test_lib_test_common '' "$@"
 }
diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index 72809477645..7f6714a5fc5 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -68,7 +68,7 @@ test_expect_success 'success is reported like this' '
 '
 
 test_expect_success 'pretend we have a fully passing test suite' '
-	run_sub_test_lib_test full-pass "3 passing tests" <<-\EOF &&
+	write_and_run_sub_test_lib_test full-pass "3 passing tests" <<-\EOF &&
 	for i in 1 2 3
 	do
 		test_expect_success "passing test #$i" "true"
@@ -85,7 +85,7 @@ test_expect_success 'pretend we have a fully passing test suite' '
 '
 
 test_expect_success 'pretend we have a partially passing test suite' '
-	run_sub_test_lib_test_err \
+	write_and_run_sub_test_lib_test_err \
 		partial-pass "2/3 tests passing" <<-\EOF &&
 	test_expect_success "passing test #1" "true"
 	test_expect_success "failing test #2" "false"
@@ -103,7 +103,7 @@ test_expect_success 'pretend we have a partially passing test suite' '
 '
 
 test_expect_success 'pretend we have a known breakage' '
-	run_sub_test_lib_test failing-todo "A failing TODO test" <<-\EOF &&
+	write_and_run_sub_test_lib_test failing-todo "A failing TODO test" <<-\EOF &&
 	test_expect_success "passing test" "true"
 	test_expect_failure "pretend we have a known breakage" "false"
 	test_done
@@ -118,7 +118,7 @@ test_expect_success 'pretend we have a known breakage' '
 '
 
 test_expect_success 'pretend we have fixed a known breakage' '
-	run_sub_test_lib_test passing-todo "A passing TODO test" <<-\EOF &&
+	write_and_run_sub_test_lib_test passing-todo "A passing TODO test" <<-\EOF &&
 	test_expect_failure "pretend we have fixed a known breakage" "true"
 	test_done
 	EOF
@@ -130,7 +130,7 @@ test_expect_success 'pretend we have fixed a known breakage' '
 '
 
 test_expect_success 'pretend we have fixed one of two known breakages (run in sub test-lib)' '
-	run_sub_test_lib_test partially-passing-todos \
+	write_and_run_sub_test_lib_test partially-passing-todos \
 		"2 TODO tests, one passing" <<-\EOF &&
 	test_expect_failure "pretend we have a known breakage" "false"
 	test_expect_success "pretend we have a passing test" "true"
@@ -149,7 +149,7 @@ test_expect_success 'pretend we have fixed one of two known breakages (run in su
 '
 
 test_expect_success 'pretend we have a pass, fail, and known breakage' '
-	run_sub_test_lib_test_err \
+	write_and_run_sub_test_lib_test_err \
 		mixed-results1 "mixed results #1" <<-\EOF &&
 	test_expect_success "passing test" "true"
 	test_expect_success "failing test" "false"
@@ -168,7 +168,7 @@ test_expect_success 'pretend we have a pass, fail, and known breakage' '
 '
 
 test_expect_success 'pretend we have a mix of all possible results' '
-	run_sub_test_lib_test_err \
+	write_and_run_sub_test_lib_test_err \
 		mixed-results2 "mixed results #2" <<-\EOF &&
 	test_expect_success "passing test" "true"
 	test_expect_success "passing test" "true"
@@ -204,7 +204,7 @@ test_expect_success 'pretend we have a mix of all possible results' '
 '
 
 test_expect_success 'test --verbose' '
-	run_sub_test_lib_test_err \
+	write_and_run_sub_test_lib_test_err \
 		t1234-verbose "test verbose" --verbose <<-\EOF &&
 	test_expect_success "passing test" true
 	test_expect_success "test with output" "echo foo"
@@ -231,7 +231,7 @@ test_expect_success 'test --verbose' '
 '
 
 test_expect_success 'test --verbose-only' '
-	run_sub_test_lib_test_err \
+	write_and_run_sub_test_lib_test_err \
 		t2345-verbose-only-2 "test verbose-only=2" \
 		--verbose-only=2 <<-\EOF &&
 	test_expect_success "passing test" true
@@ -256,7 +256,7 @@ test_expect_success 'test --verbose-only' '
 test_expect_success 'GIT_SKIP_TESTS' '
 	(
 		GIT_SKIP_TESTS="git.2" && export GIT_SKIP_TESTS &&
-		run_sub_test_lib_test git-skip-tests-basic \
+		write_and_run_sub_test_lib_test git-skip-tests-basic \
 			"GIT_SKIP_TESTS" <<-\EOF &&
 		for i in 1 2 3
 		do
@@ -277,7 +277,7 @@ test_expect_success 'GIT_SKIP_TESTS' '
 test_expect_success 'GIT_SKIP_TESTS several tests' '
 	(
 		GIT_SKIP_TESTS="git.2 git.5" && export GIT_SKIP_TESTS &&
-		run_sub_test_lib_test git-skip-tests-several \
+		write_and_run_sub_test_lib_test git-skip-tests-several \
 			"GIT_SKIP_TESTS several tests" <<-\EOF &&
 		for i in 1 2 3 4 5 6
 		do
@@ -301,7 +301,7 @@ test_expect_success 'GIT_SKIP_TESTS several tests' '
 test_expect_success 'GIT_SKIP_TESTS sh pattern' '
 	(
 		GIT_SKIP_TESTS="git.[2-5]" && export GIT_SKIP_TESTS &&
-		run_sub_test_lib_test git-skip-tests-sh-pattern \
+		write_and_run_sub_test_lib_test git-skip-tests-sh-pattern \
 			"GIT_SKIP_TESTS sh pattern" <<-\EOF &&
 		for i in 1 2 3 4 5 6
 		do
@@ -325,7 +325,7 @@ test_expect_success 'GIT_SKIP_TESTS sh pattern' '
 test_expect_success 'GIT_SKIP_TESTS entire suite' '
 	(
 		GIT_SKIP_TESTS="git" && export GIT_SKIP_TESTS &&
-		run_sub_test_lib_test git-skip-tests-entire-suite \
+		write_and_run_sub_test_lib_test git-skip-tests-entire-suite \
 			"GIT_SKIP_TESTS entire suite" <<-\EOF &&
 		for i in 1 2 3
 		do
@@ -342,7 +342,7 @@ test_expect_success 'GIT_SKIP_TESTS entire suite' '
 test_expect_success 'GIT_SKIP_TESTS does not skip unmatched suite' '
 	(
 		GIT_SKIP_TESTS="notgit" && export GIT_SKIP_TESTS &&
-		run_sub_test_lib_test git-skip-tests-unmatched-suite \
+		write_and_run_sub_test_lib_test git-skip-tests-unmatched-suite \
 			"GIT_SKIP_TESTS does not skip unmatched suite" <<-\EOF &&
 		for i in 1 2 3
 		do
@@ -361,7 +361,7 @@ test_expect_success 'GIT_SKIP_TESTS does not skip unmatched suite' '
 '
 
 test_expect_success '--run basic' '
-	run_sub_test_lib_test run-basic \
+	write_and_run_sub_test_lib_test run-basic \
 		"--run basic" --run="1,3,5" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -382,7 +382,7 @@ test_expect_success '--run basic' '
 '
 
 test_expect_success '--run with a range' '
-	run_sub_test_lib_test run-range \
+	write_and_run_sub_test_lib_test run-range \
 		"--run with a range" --run="1-3" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -403,7 +403,7 @@ test_expect_success '--run with a range' '
 '
 
 test_expect_success '--run with two ranges' '
-	run_sub_test_lib_test run-two-ranges \
+	write_and_run_sub_test_lib_test run-two-ranges \
 		"--run with two ranges" --run="1-2,5-6" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -424,7 +424,7 @@ test_expect_success '--run with two ranges' '
 '
 
 test_expect_success '--run with a left open range' '
-	run_sub_test_lib_test run-left-open-range \
+	write_and_run_sub_test_lib_test run-left-open-range \
 		"--run with a left open range" --run="-3" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -445,7 +445,7 @@ test_expect_success '--run with a left open range' '
 '
 
 test_expect_success '--run with a right open range' '
-	run_sub_test_lib_test run-right-open-range \
+	write_and_run_sub_test_lib_test run-right-open-range \
 		"--run with a right open range" --run="4-" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -466,7 +466,7 @@ test_expect_success '--run with a right open range' '
 '
 
 test_expect_success '--run with basic negation' '
-	run_sub_test_lib_test run-basic-neg \
+	write_and_run_sub_test_lib_test run-basic-neg \
 		"--run with basic negation" --run="!3" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -487,7 +487,7 @@ test_expect_success '--run with basic negation' '
 '
 
 test_expect_success '--run with two negations' '
-	run_sub_test_lib_test run-two-neg \
+	write_and_run_sub_test_lib_test run-two-neg \
 		"--run with two negations" --run="!3,!6" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -508,7 +508,7 @@ test_expect_success '--run with two negations' '
 '
 
 test_expect_success '--run a range and negation' '
-	run_sub_test_lib_test run-range-and-neg \
+	write_and_run_sub_test_lib_test run-range-and-neg \
 		"--run a range and negation" --run="-4,!2" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -529,7 +529,7 @@ test_expect_success '--run a range and negation' '
 '
 
 test_expect_success '--run range negation' '
-	run_sub_test_lib_test run-range-neg \
+	write_and_run_sub_test_lib_test run-range-neg \
 		"--run range negation" --run="!1-3" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -550,7 +550,7 @@ test_expect_success '--run range negation' '
 '
 
 test_expect_success '--run include, exclude and include' '
-	run_sub_test_lib_test run-inc-neg-inc \
+	write_and_run_sub_test_lib_test run-inc-neg-inc \
 		"--run include, exclude and include" \
 		--run="1-5,!1-3,2" <<-\EOF &&
 	for i in 1 2 3 4 5 6
@@ -572,7 +572,7 @@ test_expect_success '--run include, exclude and include' '
 '
 
 test_expect_success '--run include, exclude and include, comma separated' '
-	run_sub_test_lib_test run-inc-neg-inc-comma \
+	write_and_run_sub_test_lib_test run-inc-neg-inc-comma \
 		"--run include, exclude and include, comma separated" \
 		--run=1-5,!1-3,2 <<-\EOF &&
 	for i in 1 2 3 4 5 6
@@ -594,7 +594,7 @@ test_expect_success '--run include, exclude and include, comma separated' '
 '
 
 test_expect_success '--run exclude and include' '
-	run_sub_test_lib_test run-neg-inc \
+	write_and_run_sub_test_lib_test run-neg-inc \
 		"--run exclude and include" \
 		--run="!3-,5" <<-\EOF &&
 	for i in 1 2 3 4 5 6
@@ -616,7 +616,7 @@ test_expect_success '--run exclude and include' '
 '
 
 test_expect_success '--run empty selectors' '
-	run_sub_test_lib_test run-empty-sel \
+	write_and_run_sub_test_lib_test run-empty-sel \
 		"--run empty selectors" \
 		--run="1,,3,,,5" <<-\EOF &&
 	for i in 1 2 3 4 5 6
@@ -638,7 +638,7 @@ test_expect_success '--run empty selectors' '
 '
 
 test_expect_success '--run substring selector' '
-	run_sub_test_lib_test run-substring-selector \
+	write_and_run_sub_test_lib_test run-substring-selector \
 		"--run empty selectors" \
 		--run="relevant" <<-\EOF &&
 	test_expect_success "relevant test" "true"
@@ -662,7 +662,7 @@ test_expect_success '--run substring selector' '
 '
 
 test_expect_success '--run keyword selection' '
-	run_sub_test_lib_test_err run-inv-range-start \
+	write_and_run_sub_test_lib_test_err run-inv-range-start \
 		"--run invalid range start" \
 		--run="a-5" <<-\EOF &&
 	test_expect_success "passing test #1" "true"
@@ -677,7 +677,7 @@ test_expect_success '--run keyword selection' '
 '
 
 test_expect_success '--run invalid range end' '
-	run_sub_test_lib_test_err run-inv-range-end \
+	write_and_run_sub_test_lib_test_err run-inv-range-end \
 		"--run invalid range end" \
 		--run="1-z" <<-\EOF &&
 	test_expect_success "passing test #1" "true"
@@ -692,7 +692,7 @@ test_expect_success '--run invalid range end' '
 '
 
 test_expect_success 'tests respect prerequisites' '
-	run_sub_test_lib_test prereqs "tests respect prereqs" <<-\EOF &&
+	write_and_run_sub_test_lib_test prereqs "tests respect prereqs" <<-\EOF &&
 
 	test_set_prereq HAVEIT
 	test_expect_success HAVEIT "prereq is satisfied" "true"
@@ -722,7 +722,7 @@ test_expect_success 'tests respect prerequisites' '
 '
 
 test_expect_success 'tests respect lazy prerequisites' '
-	run_sub_test_lib_test lazy-prereqs "respect lazy prereqs" <<-\EOF &&
+	write_and_run_sub_test_lib_test lazy-prereqs "respect lazy prereqs" <<-\EOF &&
 
 	test_lazy_prereq LAZY_TRUE true
 	test_expect_success LAZY_TRUE "lazy prereq is satisifed" "true"
@@ -746,7 +746,7 @@ test_expect_success 'tests respect lazy prerequisites' '
 '
 
 test_expect_success 'nested lazy prerequisites' '
-	run_sub_test_lib_test nested-lazy "nested lazy prereqs" <<-\EOF &&
+	write_and_run_sub_test_lib_test nested-lazy "nested lazy prereqs" <<-\EOF &&
 
 	test_lazy_prereq NESTED_INNER "
 		>inner &&
@@ -772,7 +772,7 @@ test_expect_success 'nested lazy prerequisites' '
 '
 
 test_expect_success 'lazy prereqs do not turn off tracing' '
-	run_sub_test_lib_test lazy-prereq-and-tracing \
+	write_and_run_sub_test_lib_test lazy-prereq-and-tracing \
 		"lazy prereqs and -x" -v -x <<-\EOF &&
 	test_lazy_prereq LAZY true
 
@@ -785,7 +785,7 @@ test_expect_success 'lazy prereqs do not turn off tracing' '
 '
 
 test_expect_success 'tests clean up after themselves' '
-	run_sub_test_lib_test cleanup "test with cleanup" <<-\EOF &&
+	write_and_run_sub_test_lib_test cleanup "test with cleanup" <<-\EOF &&
 	clean=no
 	test_expect_success "do cleanup" "
 		test_when_finished clean=yes
@@ -805,7 +805,7 @@ test_expect_success 'tests clean up after themselves' '
 '
 
 test_expect_success 'tests clean up even on failures' '
-	run_sub_test_lib_test_err \
+	write_and_run_sub_test_lib_test_err \
 		failing-cleanup "Failing tests with cleanup commands" <<-\EOF &&
 	test_expect_success "tests clean up even after a failure" "
 		touch clean-after-failure &&
@@ -834,7 +834,7 @@ test_expect_success 'tests clean up even on failures' '
 '
 
 test_expect_success 'test_atexit is run' '
-	run_sub_test_lib_test_err \
+	write_and_run_sub_test_lib_test_err \
 		atexit-cleanup "Run atexit commands" -i <<-\EOF &&
 	test_expect_success "tests clean up even after a failure" "
 		> ../../clean-atexit &&
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [PATCH 3/8] test-lib tests: stop using a subshell in write_sub_test_lib_test()
  2021-06-14 10:48 [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh Ævar Arnfjörð Bjarmason
  2021-06-14 10:48 ` [PATCH 1/8] test-lib tests: move "run_sub_test" to a new lib-subtest.sh Ævar Arnfjörð Bjarmason
  2021-06-14 10:48 ` [PATCH 2/8] test-lib tests: split up "write and run" into two functions Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:48 ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:48 ` [PATCH 4/8] test-lib tests: don't provide a description for the sub-tests Ævar Arnfjörð Bjarmason
                   ` (7 subsequent siblings)
  10 siblings, 0 replies; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:48 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Felipe Contreras, Adam Spiers,
	Thomas Rast, Ilya Bobyr, Ævar Arnfjörð Bjarmason

Now that this function doesn't handle running the test anymore we can
do away with the sub-shell, which was used to scope an "unset" and
"export" shell variables.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/lib-subtest.sh | 21 +++++++++------------
 1 file changed, 9 insertions(+), 12 deletions(-)

diff --git a/t/lib-subtest.sh b/t/lib-subtest.sh
index 0ece41c0e5f..465f8f373e9 100644
--- a/t/lib-subtest.sh
+++ b/t/lib-subtest.sh
@@ -1,20 +1,17 @@
 write_sub_test_lib_test () {
 	name="$1" descr="$2" # stdin is the body of the test code
 	mkdir "$name" &&
-	(
-		cd "$name" &&
-		write_script "$name.sh" "$TEST_SHELL_PATH" <<-EOF &&
-		test_description='$descr (run in sub test-lib)
+	write_script "$name/$name.sh" "$TEST_SHELL_PATH" <<-EOF &&
+	test_description='$descr (run in sub test-lib)
 
-		This is run in a sub test-lib so that we do not get incorrect
-		passing metrics
-		'
+	This is run in a sub test-lib so that we do not get incorrect
+	passing metrics
+	'
 
-		# Point to the t/test-lib.sh, which isn't in ../ as usual
-		. "\$TEST_DIRECTORY"/test-lib.sh
-		EOF
-		cat >>"$name.sh"
-	)
+	# Point to the t/test-lib.sh, which isn't in ../ as usual
+	. "\$TEST_DIRECTORY"/test-lib.sh
+	EOF
+	cat >>"$name/$name.sh"
 }
 
 _run_sub_test_lib_test_common () {
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [PATCH 4/8] test-lib tests: don't provide a description for the sub-tests
  2021-06-14 10:48 [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh Ævar Arnfjörð Bjarmason
                   ` (2 preceding siblings ...)
  2021-06-14 10:48 ` [PATCH 3/8] test-lib tests: stop using a subshell in write_sub_test_lib_test() Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:48 ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:48 ` [PATCH 5/8] test-lib tests: get rid of copy/pasted mock test code Ævar Arnfjörð Bjarmason
                   ` (6 subsequent siblings)
  10 siblings, 0 replies; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:48 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Felipe Contreras, Adam Spiers,
	Thomas Rast, Ilya Bobyr, Ævar Arnfjörð Bjarmason

Change the $test_description provided for the generated subtests to be
constant, since the only purpose of having it is that test-lib.sh will
barf if it isn't supplied.

The other purpose of having it was to effectively split up the test
description between the argument to test_expect_success and the
argument to "write_and_run_sub_test_lib_test". Let's only use one of
the two.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/lib-subtest.sh |  12 ++--
 t/t0000-basic.sh | 147 +++++++++++++++++++++--------------------------
 2 files changed, 71 insertions(+), 88 deletions(-)

diff --git a/t/lib-subtest.sh b/t/lib-subtest.sh
index 465f8f373e9..4a50e380506 100644
--- a/t/lib-subtest.sh
+++ b/t/lib-subtest.sh
@@ -1,12 +1,8 @@
 write_sub_test_lib_test () {
-	name="$1" descr="$2" # stdin is the body of the test code
+	name="$1" # stdin is the body of the test code
 	mkdir "$name" &&
 	write_script "$name/$name.sh" "$TEST_SHELL_PATH" <<-EOF &&
-	test_description='$descr (run in sub test-lib)
-
-	This is run in a sub test-lib so that we do not get incorrect
-	passing metrics
-	'
+	test_description='A test of test-lib.sh itself'
 
 	# Point to the t/test-lib.sh, which isn't in ../ as usual
 	. "\$TEST_DIRECTORY"/test-lib.sh
@@ -15,8 +11,8 @@ write_sub_test_lib_test () {
 }
 
 _run_sub_test_lib_test_common () {
-	neg="$1" name="$2" descr="$3" # stdin is the body of the test code
-	shift 3
+	neg="$1" name="$2" # stdin is the body of the test code
+	shift 2
 	(
 		cd "$name" &&
 
diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index 7f6714a5fc5..b93a8ced7f6 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -67,8 +67,8 @@ test_expect_success 'success is reported like this' '
 	:
 '
 
-test_expect_success 'pretend we have a fully passing test suite' '
-	write_and_run_sub_test_lib_test full-pass "3 passing tests" <<-\EOF &&
+test_expect_success 'subtest: 3 passing tests' '
+	write_and_run_sub_test_lib_test full-pass <<-\EOF &&
 	for i in 1 2 3
 	do
 		test_expect_success "passing test #$i" "true"
@@ -84,9 +84,8 @@ test_expect_success 'pretend we have a fully passing test suite' '
 	EOF
 '
 
-test_expect_success 'pretend we have a partially passing test suite' '
-	write_and_run_sub_test_lib_test_err \
-		partial-pass "2/3 tests passing" <<-\EOF &&
+test_expect_success 'subtest: 2/3 tests passing' '
+	write_and_run_sub_test_lib_test_err partial-pass <<-\EOF &&
 	test_expect_success "passing test #1" "true"
 	test_expect_success "failing test #2" "false"
 	test_expect_success "passing test #3" "true"
@@ -102,8 +101,8 @@ test_expect_success 'pretend we have a partially passing test suite' '
 	EOF
 '
 
-test_expect_success 'pretend we have a known breakage' '
-	write_and_run_sub_test_lib_test failing-todo "A failing TODO test" <<-\EOF &&
+test_expect_success 'subtest: a failing TODO test' '
+	write_and_run_sub_test_lib_test failing-todo <<-\EOF &&
 	test_expect_success "passing test" "true"
 	test_expect_failure "pretend we have a known breakage" "false"
 	test_done
@@ -117,8 +116,8 @@ test_expect_success 'pretend we have a known breakage' '
 	EOF
 '
 
-test_expect_success 'pretend we have fixed a known breakage' '
-	write_and_run_sub_test_lib_test passing-todo "A passing TODO test" <<-\EOF &&
+test_expect_success 'subtest: a passing TODO test' '
+	write_and_run_sub_test_lib_test passing-todo <<-\EOF &&
 	test_expect_failure "pretend we have fixed a known breakage" "true"
 	test_done
 	EOF
@@ -129,9 +128,8 @@ test_expect_success 'pretend we have fixed a known breakage' '
 	EOF
 '
 
-test_expect_success 'pretend we have fixed one of two known breakages (run in sub test-lib)' '
-	write_and_run_sub_test_lib_test partially-passing-todos \
-		"2 TODO tests, one passing" <<-\EOF &&
+test_expect_success 'subtest: 2 TODO tests, one passin' '
+	write_and_run_sub_test_lib_test partially-passing-todos <<-\EOF &&
 	test_expect_failure "pretend we have a known breakage" "false"
 	test_expect_success "pretend we have a passing test" "true"
 	test_expect_failure "pretend we have fixed another known breakage" "true"
@@ -148,9 +146,8 @@ test_expect_success 'pretend we have fixed one of two known breakages (run in su
 	EOF
 '
 
-test_expect_success 'pretend we have a pass, fail, and known breakage' '
-	write_and_run_sub_test_lib_test_err \
-		mixed-results1 "mixed results #1" <<-\EOF &&
+test_expect_success 'subtest: mixed results: pass, failure and a TODO test' '
+	write_and_run_sub_test_lib_test_err mixed-results1 <<-\EOF &&
 	test_expect_success "passing test" "true"
 	test_expect_success "failing test" "false"
 	test_expect_failure "pretend we have a known breakage" "false"
@@ -167,9 +164,8 @@ test_expect_success 'pretend we have a pass, fail, and known breakage' '
 	EOF
 '
 
-test_expect_success 'pretend we have a mix of all possible results' '
-	write_and_run_sub_test_lib_test_err \
-		mixed-results2 "mixed results #2" <<-\EOF &&
+test_expect_success 'subtest: mixed results: a mixture of all possible results' '
+	write_and_run_sub_test_lib_test_err mixed-results2 <<-\EOF &&
 	test_expect_success "passing test" "true"
 	test_expect_success "passing test" "true"
 	test_expect_success "passing test" "true"
@@ -203,9 +199,8 @@ test_expect_success 'pretend we have a mix of all possible results' '
 	EOF
 '
 
-test_expect_success 'test --verbose' '
-	write_and_run_sub_test_lib_test_err \
-		t1234-verbose "test verbose" --verbose <<-\EOF &&
+test_expect_success 'subtest: --verbose option' '
+	write_and_run_sub_test_lib_test_err t1234-verbose --verbose <<-\EOF &&
 	test_expect_success "passing test" true
 	test_expect_success "test with output" "echo foo"
 	test_expect_success "failing test" false
@@ -230,9 +225,9 @@ test_expect_success 'test --verbose' '
 	EOF
 '
 
-test_expect_success 'test --verbose-only' '
+test_expect_success 'subtest: --verbose-only option' '
 	write_and_run_sub_test_lib_test_err \
-		t2345-verbose-only-2 "test verbose-only=2" \
+		t2345-verbose-only-2 \
 		--verbose-only=2 <<-\EOF &&
 	test_expect_success "passing test" true
 	test_expect_success "test with output" "echo foo"
@@ -253,11 +248,11 @@ test_expect_success 'test --verbose-only' '
 	EOF
 '
 
-test_expect_success 'GIT_SKIP_TESTS' '
+test_expect_success 'subtest: skip one with GIT_SKIP_TESTS' '
 	(
 		GIT_SKIP_TESTS="git.2" && export GIT_SKIP_TESTS &&
 		write_and_run_sub_test_lib_test git-skip-tests-basic \
-			"GIT_SKIP_TESTS" <<-\EOF &&
+			<<-\EOF &&
 		for i in 1 2 3
 		do
 			test_expect_success "passing test #$i" "true"
@@ -274,11 +269,11 @@ test_expect_success 'GIT_SKIP_TESTS' '
 	)
 '
 
-test_expect_success 'GIT_SKIP_TESTS several tests' '
+test_expect_success 'subtest: skip several with GIT_SKIP_TESTS' '
 	(
 		GIT_SKIP_TESTS="git.2 git.5" && export GIT_SKIP_TESTS &&
 		write_and_run_sub_test_lib_test git-skip-tests-several \
-			"GIT_SKIP_TESTS several tests" <<-\EOF &&
+			<<-\EOF &&
 		for i in 1 2 3 4 5 6
 		do
 			test_expect_success "passing test #$i" "true"
@@ -298,11 +293,11 @@ test_expect_success 'GIT_SKIP_TESTS several tests' '
 	)
 '
 
-test_expect_success 'GIT_SKIP_TESTS sh pattern' '
+test_expect_success 'subtest: sh pattern skipping with GIT_SKIP_TESTS' '
 	(
 		GIT_SKIP_TESTS="git.[2-5]" && export GIT_SKIP_TESTS &&
 		write_and_run_sub_test_lib_test git-skip-tests-sh-pattern \
-			"GIT_SKIP_TESTS sh pattern" <<-\EOF &&
+			<<-\EOF &&
 		for i in 1 2 3 4 5 6
 		do
 			test_expect_success "passing test #$i" "true"
@@ -322,11 +317,11 @@ test_expect_success 'GIT_SKIP_TESTS sh pattern' '
 	)
 '
 
-test_expect_success 'GIT_SKIP_TESTS entire suite' '
+test_expect_success 'subtest: skip entire test suite with GIT_SKIP_TESTS' '
 	(
 		GIT_SKIP_TESTS="git" && export GIT_SKIP_TESTS &&
 		write_and_run_sub_test_lib_test git-skip-tests-entire-suite \
-			"GIT_SKIP_TESTS entire suite" <<-\EOF &&
+			<<-\EOF &&
 		for i in 1 2 3
 		do
 			test_expect_success "passing test #$i" "true"
@@ -339,11 +334,11 @@ test_expect_success 'GIT_SKIP_TESTS entire suite' '
 	)
 '
 
-test_expect_success 'GIT_SKIP_TESTS does not skip unmatched suite' '
+test_expect_success 'subtest: GIT_SKIP_TESTS does not skip unmatched suite' '
 	(
 		GIT_SKIP_TESTS="notgit" && export GIT_SKIP_TESTS &&
 		write_and_run_sub_test_lib_test git-skip-tests-unmatched-suite \
-			"GIT_SKIP_TESTS does not skip unmatched suite" <<-\EOF &&
+			<<-\EOF &&
 		for i in 1 2 3
 		do
 			test_expect_success "passing test #$i" "true"
@@ -360,9 +355,8 @@ test_expect_success 'GIT_SKIP_TESTS does not skip unmatched suite' '
 	)
 '
 
-test_expect_success '--run basic' '
-	write_and_run_sub_test_lib_test run-basic \
-		"--run basic" --run="1,3,5" <<-\EOF &&
+test_expect_success 'subtest: --run basic' '
+	write_and_run_sub_test_lib_test run-basic --run="1,3,5" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
 		test_expect_success "passing test #$i" "true"
@@ -381,9 +375,9 @@ test_expect_success '--run basic' '
 	EOF
 '
 
-test_expect_success '--run with a range' '
+test_expect_success 'subtest: --run with a range' '
 	write_and_run_sub_test_lib_test run-range \
-		"--run with a range" --run="1-3" <<-\EOF &&
+		--run="1-3" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
 		test_expect_success "passing test #$i" "true"
@@ -402,9 +396,9 @@ test_expect_success '--run with a range' '
 	EOF
 '
 
-test_expect_success '--run with two ranges' '
+test_expect_success 'subtest: --run with two ranges' '
 	write_and_run_sub_test_lib_test run-two-ranges \
-		"--run with two ranges" --run="1-2,5-6" <<-\EOF &&
+		--run="1-2,5-6" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
 		test_expect_success "passing test #$i" "true"
@@ -423,9 +417,9 @@ test_expect_success '--run with two ranges' '
 	EOF
 '
 
-test_expect_success '--run with a left open range' '
+test_expect_success 'subtest: --run with a left open range' '
 	write_and_run_sub_test_lib_test run-left-open-range \
-		"--run with a left open range" --run="-3" <<-\EOF &&
+		--run="-3" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
 		test_expect_success "passing test #$i" "true"
@@ -444,9 +438,9 @@ test_expect_success '--run with a left open range' '
 	EOF
 '
 
-test_expect_success '--run with a right open range' '
+test_expect_success 'subtest: --run with a right open range' '
 	write_and_run_sub_test_lib_test run-right-open-range \
-		"--run with a right open range" --run="4-" <<-\EOF &&
+		--run="4-" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
 		test_expect_success "passing test #$i" "true"
@@ -465,9 +459,9 @@ test_expect_success '--run with a right open range' '
 	EOF
 '
 
-test_expect_success '--run with basic negation' '
+test_expect_success 'subtest: --run with basic negation' '
 	write_and_run_sub_test_lib_test run-basic-neg \
-		"--run with basic negation" --run="!3" <<-\EOF &&
+		--run="!3" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
 		test_expect_success "passing test #$i" "true"
@@ -486,9 +480,9 @@ test_expect_success '--run with basic negation' '
 	EOF
 '
 
-test_expect_success '--run with two negations' '
+test_expect_success 'subtest: --run with two negations' '
 	write_and_run_sub_test_lib_test run-two-neg \
-		"--run with two negations" --run="!3,!6" <<-\EOF &&
+		--run="!3,!6" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
 		test_expect_success "passing test #$i" "true"
@@ -507,9 +501,9 @@ test_expect_success '--run with two negations' '
 	EOF
 '
 
-test_expect_success '--run a range and negation' '
+test_expect_success 'subtest: --run a range and negation' '
 	write_and_run_sub_test_lib_test run-range-and-neg \
-		"--run a range and negation" --run="-4,!2" <<-\EOF &&
+		--run="-4,!2" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
 		test_expect_success "passing test #$i" "true"
@@ -528,9 +522,9 @@ test_expect_success '--run a range and negation' '
 	EOF
 '
 
-test_expect_success '--run range negation' '
+test_expect_success 'subtest: --run range negation' '
 	write_and_run_sub_test_lib_test run-range-neg \
-		"--run range negation" --run="!1-3" <<-\EOF &&
+		--run="!1-3" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
 		test_expect_success "passing test #$i" "true"
@@ -549,9 +543,8 @@ test_expect_success '--run range negation' '
 	EOF
 '
 
-test_expect_success '--run include, exclude and include' '
+test_expect_success 'subtest: --run include, exclude and include' '
 	write_and_run_sub_test_lib_test run-inc-neg-inc \
-		"--run include, exclude and include" \
 		--run="1-5,!1-3,2" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -571,9 +564,8 @@ test_expect_success '--run include, exclude and include' '
 	EOF
 '
 
-test_expect_success '--run include, exclude and include, comma separated' '
+test_expect_success 'subtest: --run include, exclude and include, comma separated' '
 	write_and_run_sub_test_lib_test run-inc-neg-inc-comma \
-		"--run include, exclude and include, comma separated" \
 		--run=1-5,!1-3,2 <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -593,9 +585,8 @@ test_expect_success '--run include, exclude and include, comma separated' '
 	EOF
 '
 
-test_expect_success '--run exclude and include' '
+test_expect_success 'subtest: --run exclude and include' '
 	write_and_run_sub_test_lib_test run-neg-inc \
-		"--run exclude and include" \
 		--run="!3-,5" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -615,9 +606,8 @@ test_expect_success '--run exclude and include' '
 	EOF
 '
 
-test_expect_success '--run empty selectors' '
+test_expect_success 'subtest: --run empty selectors' '
 	write_and_run_sub_test_lib_test run-empty-sel \
-		"--run empty selectors" \
 		--run="1,,3,,,5" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -637,9 +627,8 @@ test_expect_success '--run empty selectors' '
 	EOF
 '
 
-test_expect_success '--run substring selector' '
+test_expect_success 'subtest: --run substring selector' '
 	write_and_run_sub_test_lib_test run-substring-selector \
-		"--run empty selectors" \
 		--run="relevant" <<-\EOF &&
 	test_expect_success "relevant test" "true"
 	for i in 1 2 3 4 5 6
@@ -661,9 +650,8 @@ test_expect_success '--run substring selector' '
 	EOF
 '
 
-test_expect_success '--run keyword selection' '
+test_expect_success 'subtest: --run keyword selection' '
 	write_and_run_sub_test_lib_test_err run-inv-range-start \
-		"--run invalid range start" \
 		--run="a-5" <<-\EOF &&
 	test_expect_success "passing test #1" "true"
 	test_done
@@ -676,9 +664,8 @@ test_expect_success '--run keyword selection' '
 	EOF_ERR
 '
 
-test_expect_success '--run invalid range end' '
+test_expect_success 'subtest: --run invalid range end' '
 	write_and_run_sub_test_lib_test_err run-inv-range-end \
-		"--run invalid range end" \
 		--run="1-z" <<-\EOF &&
 	test_expect_success "passing test #1" "true"
 	test_done
@@ -691,8 +678,8 @@ test_expect_success '--run invalid range end' '
 	EOF_ERR
 '
 
-test_expect_success 'tests respect prerequisites' '
-	write_and_run_sub_test_lib_test prereqs "tests respect prereqs" <<-\EOF &&
+test_expect_success 'subtest: tests respect prerequisites' '
+	write_and_run_sub_test_lib_test prereqs <<-\EOF &&
 
 	test_set_prereq HAVEIT
 	test_expect_success HAVEIT "prereq is satisfied" "true"
@@ -721,8 +708,8 @@ test_expect_success 'tests respect prerequisites' '
 	EOF
 '
 
-test_expect_success 'tests respect lazy prerequisites' '
-	write_and_run_sub_test_lib_test lazy-prereqs "respect lazy prereqs" <<-\EOF &&
+test_expect_success 'subtest: tests respect lazy prerequisites' '
+	write_and_run_sub_test_lib_test lazy-prereqs <<-\EOF &&
 
 	test_lazy_prereq LAZY_TRUE true
 	test_expect_success LAZY_TRUE "lazy prereq is satisifed" "true"
@@ -745,8 +732,8 @@ test_expect_success 'tests respect lazy prerequisites' '
 	EOF
 '
 
-test_expect_success 'nested lazy prerequisites' '
-	write_and_run_sub_test_lib_test nested-lazy "nested lazy prereqs" <<-\EOF &&
+test_expect_success 'subtest: nested lazy prerequisites' '
+	write_and_run_sub_test_lib_test nested-lazy <<-\EOF &&
 
 	test_lazy_prereq NESTED_INNER "
 		>inner &&
@@ -771,9 +758,9 @@ test_expect_success 'nested lazy prerequisites' '
 	EOF
 '
 
-test_expect_success 'lazy prereqs do not turn off tracing' '
+test_expect_success 'subtest: lazy prereqs do not turn off tracing' '
 	write_and_run_sub_test_lib_test lazy-prereq-and-tracing \
-		"lazy prereqs and -x" -v -x <<-\EOF &&
+		-v -x <<-\EOF &&
 	test_lazy_prereq LAZY true
 
 	test_expect_success lazy "test_have_prereq LAZY && echo trace"
@@ -784,8 +771,8 @@ test_expect_success 'lazy prereqs do not turn off tracing' '
 	grep "echo trace" lazy-prereq-and-tracing/err
 '
 
-test_expect_success 'tests clean up after themselves' '
-	write_and_run_sub_test_lib_test cleanup "test with cleanup" <<-\EOF &&
+test_expect_success 'subtest: tests clean up after themselves' '
+	write_and_run_sub_test_lib_test cleanup <<-\EOF &&
 	clean=no
 	test_expect_success "do cleanup" "
 		test_when_finished clean=yes
@@ -804,9 +791,9 @@ test_expect_success 'tests clean up after themselves' '
 	EOF
 '
 
-test_expect_success 'tests clean up even on failures' '
+test_expect_success 'subtest: tests clean up even on failures' '
 	write_and_run_sub_test_lib_test_err \
-		failing-cleanup "Failing tests with cleanup commands" <<-\EOF &&
+		failing-cleanup <<-\EOF &&
 	test_expect_success "tests clean up even after a failure" "
 		touch clean-after-failure &&
 		test_when_finished rm clean-after-failure &&
@@ -833,9 +820,9 @@ test_expect_success 'tests clean up even on failures' '
 	EOF
 '
 
-test_expect_success 'test_atexit is run' '
+test_expect_success 'subtest: test_atexit is run' '
 	write_and_run_sub_test_lib_test_err \
-		atexit-cleanup "Run atexit commands" -i <<-\EOF &&
+		atexit-cleanup -i <<-\EOF &&
 	test_expect_success "tests clean up even after a failure" "
 		> ../../clean-atexit &&
 		test_atexit rm ../../clean-atexit &&
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [PATCH 5/8] test-lib tests: get rid of copy/pasted mock test code
  2021-06-14 10:48 [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh Ævar Arnfjörð Bjarmason
                   ` (3 preceding siblings ...)
  2021-06-14 10:48 ` [PATCH 4/8] test-lib tests: don't provide a description for the sub-tests Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:48 ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:48 ` [PATCH 6/8] test-lib tests: avoid subshell for "test_cmp" for readability Ævar Arnfjörð Bjarmason
                   ` (5 subsequent siblings)
  10 siblings, 0 replies; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:48 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Felipe Contreras, Adam Spiers,
	Thomas Rast, Ilya Bobyr, Ævar Arnfjörð Bjarmason

Now that we've split up the write_sub_test_lib_test*() and
run_sub_test_lib_test*() functions let's fix those tests in
t0000-basic.sh that were verbosely copy/pasting earlier tests.

I'm (ab)using writing a tag object under a ref-name that's
content-addressable from the content of the test script. If we can
update-ref that it's unique, if not we've got a duplicate. The tag
object stores the name of the earlier test for reporting the error.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/lib-subtest.sh |  34 ++++++-
 t/t0000-basic.sh | 224 ++++++++++++-----------------------------------
 2 files changed, 88 insertions(+), 170 deletions(-)

diff --git a/t/lib-subtest.sh b/t/lib-subtest.sh
index 4a50e380506..4834db6c4ca 100644
--- a/t/lib-subtest.sh
+++ b/t/lib-subtest.sh
@@ -1,3 +1,34 @@
+_assert_unique_sub_test () {
+	name=$1 &&
+
+	# Alert about the copy/paste programming
+	hash=$(git hash-object -w "$name") &&
+	cat >tag.sig <<-EOF &&
+	object $hash
+	type blob
+	tag $hash
+	tagger . <> 0 +0000
+
+	duplicate script detected!
+
+	This test script was already written as:
+
+	$name
+
+	You can just re-use its test code with your own
+	run_sub_test_lib_test*()
+	EOF
+
+	tag=$(git mktag <tag.sig) &&
+	if ! git update-ref refs/tags/blob-$hash $tag $(test_oid zero) 2>/dev/null
+	then
+		msg=$(git for-each-ref refs/tags/blob-$hash \
+			--format='%(contents)' refs/tags/blob-$hash)
+		error "on write of $name: $msg"
+		return 1
+	fi
+}
+
 write_sub_test_lib_test () {
 	name="$1" # stdin is the body of the test code
 	mkdir "$name" &&
@@ -7,7 +38,8 @@ write_sub_test_lib_test () {
 	# Point to the t/test-lib.sh, which isn't in ../ as usual
 	. "\$TEST_DIRECTORY"/test-lib.sh
 	EOF
-	cat >>"$name/$name.sh"
+	cat >>"$name/$name.sh" &&
+	_assert_unique_sub_test "$name/$name.sh"
 }
 
 _run_sub_test_lib_test_common () {
diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index b93a8ced7f6..886329d1f8e 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -226,18 +226,13 @@ test_expect_success 'subtest: --verbose option' '
 '
 
 test_expect_success 'subtest: --verbose-only option' '
-	write_and_run_sub_test_lib_test_err \
-		t2345-verbose-only-2 \
-		--verbose-only=2 <<-\EOF &&
-	test_expect_success "passing test" true
-	test_expect_success "test with output" "echo foo"
-	test_expect_success "failing test" false
-	test_done
-	EOF
-	check_sub_test_lib_test t2345-verbose-only-2 <<-\EOF
+	run_sub_test_lib_test_err \
+		t1234-verbose \
+		--verbose-only=2 &&
+	check_sub_test_lib_test t1234-verbose <<-\EOF
 	> ok 1 - passing test
 	> Z
-	> expecting success of 2345.2 '\''test with output'\'': echo foo
+	> expecting success of 1234.2 '\''test with output'\'': echo foo
 	> foo
 	> ok 2 - test with output
 	> Z
@@ -250,16 +245,9 @@ test_expect_success 'subtest: --verbose-only option' '
 
 test_expect_success 'subtest: skip one with GIT_SKIP_TESTS' '
 	(
-		GIT_SKIP_TESTS="git.2" && export GIT_SKIP_TESTS &&
-		write_and_run_sub_test_lib_test git-skip-tests-basic \
-			<<-\EOF &&
-		for i in 1 2 3
-		do
-			test_expect_success "passing test #$i" "true"
-		done
-		test_done
-		EOF
-		check_sub_test_lib_test git-skip-tests-basic <<-\EOF
+		GIT_SKIP_TESTS="full.2" && export GIT_SKIP_TESTS &&
+		run_sub_test_lib_test full-pass &&
+		check_sub_test_lib_test full-pass <<-\EOF
 		> ok 1 - passing test #1
 		> ok 2 # skip passing test #2 (GIT_SKIP_TESTS)
 		> ok 3 - passing test #3
@@ -296,15 +284,8 @@ test_expect_success 'subtest: skip several with GIT_SKIP_TESTS' '
 test_expect_success 'subtest: sh pattern skipping with GIT_SKIP_TESTS' '
 	(
 		GIT_SKIP_TESTS="git.[2-5]" && export GIT_SKIP_TESTS &&
-		write_and_run_sub_test_lib_test git-skip-tests-sh-pattern \
-			<<-\EOF &&
-		for i in 1 2 3 4 5 6
-		do
-			test_expect_success "passing test #$i" "true"
-		done
-		test_done
-		EOF
-		check_sub_test_lib_test git-skip-tests-sh-pattern <<-\EOF
+		run_sub_test_lib_test git-skip-tests-several &&
+		check_sub_test_lib_test git-skip-tests-several <<-\EOF
 		> ok 1 - passing test #1
 		> ok 2 # skip passing test #2 (GIT_SKIP_TESTS)
 		> ok 3 # skip passing test #3 (GIT_SKIP_TESTS)
@@ -320,15 +301,8 @@ test_expect_success 'subtest: sh pattern skipping with GIT_SKIP_TESTS' '
 test_expect_success 'subtest: skip entire test suite with GIT_SKIP_TESTS' '
 	(
 		GIT_SKIP_TESTS="git" && export GIT_SKIP_TESTS &&
-		write_and_run_sub_test_lib_test git-skip-tests-entire-suite \
-			<<-\EOF &&
-		for i in 1 2 3
-		do
-			test_expect_success "passing test #$i" "true"
-		done
-		test_done
-		EOF
-		check_sub_test_lib_test git-skip-tests-entire-suite <<-\EOF
+		run_sub_test_lib_test git-skip-tests-several &&
+		check_sub_test_lib_test git-skip-tests-several <<-\EOF
 		> 1..0 # SKIP skip all tests in git
 		EOF
 	)
@@ -337,15 +311,8 @@ test_expect_success 'subtest: skip entire test suite with GIT_SKIP_TESTS' '
 test_expect_success 'subtest: GIT_SKIP_TESTS does not skip unmatched suite' '
 	(
 		GIT_SKIP_TESTS="notgit" && export GIT_SKIP_TESTS &&
-		write_and_run_sub_test_lib_test git-skip-tests-unmatched-suite \
-			<<-\EOF &&
-		for i in 1 2 3
-		do
-			test_expect_success "passing test #$i" "true"
-		done
-		test_done
-		EOF
-		check_sub_test_lib_test git-skip-tests-unmatched-suite <<-\EOF
+		run_sub_test_lib_test full-pass &&
+		check_sub_test_lib_test full-pass <<-\EOF
 		> ok 1 - passing test #1
 		> ok 2 - passing test #2
 		> ok 3 - passing test #3
@@ -356,14 +323,8 @@ test_expect_success 'subtest: GIT_SKIP_TESTS does not skip unmatched suite' '
 '
 
 test_expect_success 'subtest: --run basic' '
-	write_and_run_sub_test_lib_test run-basic --run="1,3,5" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-basic <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several --run="1,3,5" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 - passing test #1
 	> ok 2 # skip passing test #2 (--run)
 	> ok 3 - passing test #3
@@ -376,15 +337,9 @@ test_expect_success 'subtest: --run basic' '
 '
 
 test_expect_success 'subtest: --run with a range' '
-	write_and_run_sub_test_lib_test run-range \
-		--run="1-3" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-range <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="1-3" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 - passing test #1
 	> ok 2 - passing test #2
 	> ok 3 - passing test #3
@@ -397,15 +352,9 @@ test_expect_success 'subtest: --run with a range' '
 '
 
 test_expect_success 'subtest: --run with two ranges' '
-	write_and_run_sub_test_lib_test run-two-ranges \
-		--run="1-2,5-6" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-two-ranges <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="1-2,5-6" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 - passing test #1
 	> ok 2 - passing test #2
 	> ok 3 # skip passing test #3 (--run)
@@ -418,15 +367,9 @@ test_expect_success 'subtest: --run with two ranges' '
 '
 
 test_expect_success 'subtest: --run with a left open range' '
-	write_and_run_sub_test_lib_test run-left-open-range \
-		--run="-3" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-left-open-range <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="-3" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 - passing test #1
 	> ok 2 - passing test #2
 	> ok 3 - passing test #3
@@ -439,15 +382,9 @@ test_expect_success 'subtest: --run with a left open range' '
 '
 
 test_expect_success 'subtest: --run with a right open range' '
-	write_and_run_sub_test_lib_test run-right-open-range \
-		--run="4-" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-right-open-range <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="4-" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 # skip passing test #1 (--run)
 	> ok 2 # skip passing test #2 (--run)
 	> ok 3 # skip passing test #3 (--run)
@@ -460,15 +397,9 @@ test_expect_success 'subtest: --run with a right open range' '
 '
 
 test_expect_success 'subtest: --run with basic negation' '
-	write_and_run_sub_test_lib_test run-basic-neg \
-		--run="!3" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-basic-neg <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="!3" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 - passing test #1
 	> ok 2 - passing test #2
 	> ok 3 # skip passing test #3 (--run)
@@ -481,15 +412,9 @@ test_expect_success 'subtest: --run with basic negation' '
 '
 
 test_expect_success 'subtest: --run with two negations' '
-	write_and_run_sub_test_lib_test run-two-neg \
-		--run="!3,!6" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-two-neg <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="!3,!6" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 - passing test #1
 	> ok 2 - passing test #2
 	> ok 3 # skip passing test #3 (--run)
@@ -502,15 +427,9 @@ test_expect_success 'subtest: --run with two negations' '
 '
 
 test_expect_success 'subtest: --run a range and negation' '
-	write_and_run_sub_test_lib_test run-range-and-neg \
-		--run="-4,!2" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-range-and-neg <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="-4,!2" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 - passing test #1
 	> ok 2 # skip passing test #2 (--run)
 	> ok 3 - passing test #3
@@ -523,15 +442,9 @@ test_expect_success 'subtest: --run a range and negation' '
 '
 
 test_expect_success 'subtest: --run range negation' '
-	write_and_run_sub_test_lib_test run-range-neg \
-		--run="!1-3" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-range-neg <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="!1-3" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 # skip passing test #1 (--run)
 	> ok 2 # skip passing test #2 (--run)
 	> ok 3 # skip passing test #3 (--run)
@@ -544,15 +457,9 @@ test_expect_success 'subtest: --run range negation' '
 '
 
 test_expect_success 'subtest: --run include, exclude and include' '
-	write_and_run_sub_test_lib_test run-inc-neg-inc \
-		--run="1-5,!1-3,2" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-inc-neg-inc <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="1-5,!1-3,2" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 # skip passing test #1 (--run)
 	> ok 2 - passing test #2
 	> ok 3 # skip passing test #3 (--run)
@@ -565,15 +472,9 @@ test_expect_success 'subtest: --run include, exclude and include' '
 '
 
 test_expect_success 'subtest: --run include, exclude and include, comma separated' '
-	write_and_run_sub_test_lib_test run-inc-neg-inc-comma \
-		--run=1-5,!1-3,2 <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-inc-neg-inc-comma <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run=1-5,!1-3,2 &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 # skip passing test #1 (--run)
 	> ok 2 - passing test #2
 	> ok 3 # skip passing test #3 (--run)
@@ -586,15 +487,9 @@ test_expect_success 'subtest: --run include, exclude and include, comma separate
 '
 
 test_expect_success 'subtest: --run exclude and include' '
-	write_and_run_sub_test_lib_test run-neg-inc \
-		--run="!3-,5" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-neg-inc <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="!3-,5" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 - passing test #1
 	> ok 2 - passing test #2
 	> ok 3 # skip passing test #3 (--run)
@@ -607,15 +502,9 @@ test_expect_success 'subtest: --run exclude and include' '
 '
 
 test_expect_success 'subtest: --run empty selectors' '
-	write_and_run_sub_test_lib_test run-empty-sel \
-		--run="1,,3,,,5" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-empty-sel <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="1,,3,,,5" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 - passing test #1
 	> ok 2 # skip passing test #2 (--run)
 	> ok 3 - passing test #3
@@ -665,12 +554,9 @@ test_expect_success 'subtest: --run keyword selection' '
 '
 
 test_expect_success 'subtest: --run invalid range end' '
-	write_and_run_sub_test_lib_test_err run-inv-range-end \
-		--run="1-z" <<-\EOF &&
-	test_expect_success "passing test #1" "true"
-	test_done
-	EOF
-	check_sub_test_lib_test_err run-inv-range-end \
+	run_sub_test_lib_test_err run-inv-range-start \
+		--run="1-z" &&
+	check_sub_test_lib_test_err run-inv-range-start \
 		<<-\EOF_OUT 3<<-EOF_ERR
 	> FATAL: Unexpected exit with code 1
 	EOF_OUT
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [PATCH 6/8] test-lib tests: avoid subshell for "test_cmp" for readability
  2021-06-14 10:48 [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh Ævar Arnfjörð Bjarmason
                   ` (4 preceding siblings ...)
  2021-06-14 10:48 ` [PATCH 5/8] test-lib tests: get rid of copy/pasted mock test code Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:48 ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:48 ` [PATCH 7/8] test-lib tests: refactor common part of check_sub_test_lib_test*() Ævar Arnfjörð Bjarmason
                   ` (4 subsequent siblings)
  10 siblings, 0 replies; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:48 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Felipe Contreras, Adam Spiers,
	Thomas Rast, Ilya Bobyr, Ævar Arnfjörð Bjarmason

The use of a sub-shell for running the test_cmp of stdout/stderr for
the test author was introduced in this form in 565b6fa87bb (tests:
refactor mechanics of testing in a sub test-lib, 2012-12-16), but from
looking at the history that seemed to have diligently copied my
original ad-hoc implementation in 7b905119703 (t/t0000-basic.sh: Run
the passing TODO test inside its own test-lib, 2010-08-19).

There's no reason to use a subshell here, we try to avoid it in
general. It also improves readability, if the test fails we print out
the relative path in the trash directory that needs to be looked
at.

Before that was mostly obscured, since the "write_sub_test_lib_test"
will pick the directory for you from the test name.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/lib-subtest.sh | 20 +++++++-------------
 1 file changed, 7 insertions(+), 13 deletions(-)

diff --git a/t/lib-subtest.sh b/t/lib-subtest.sh
index 4834db6c4ca..8f4935faf43 100644
--- a/t/lib-subtest.sh
+++ b/t/lib-subtest.sh
@@ -89,22 +89,16 @@ run_sub_test_lib_test_err () {
 
 check_sub_test_lib_test () {
 	name="$1" # stdin is the expected output from the test
-	(
-		cd "$name" &&
-		test_must_be_empty err &&
-		sed -e 's/^> //' -e 's/Z$//' >expect &&
-		test_cmp expect out
-	)
+	test_must_be_empty "$name"/err &&
+	sed -e 's/^> //' -e 's/Z$//' >"$name"/expect &&
+	test_cmp "$name/"expect "$name"/out
 }
 
 check_sub_test_lib_test_err () {
 	name="$1" # stdin is the expected output from the test
 	# expected error output is in descriptor 3
-	(
-		cd "$name" &&
-		sed -e 's/^> //' -e 's/Z$//' >expect.out &&
-		test_cmp expect.out out &&
-		sed -e 's/^> //' -e 's/Z$//' <&3 >expect.err &&
-		test_cmp expect.err err
-	)
+	sed -e 's/^> //' -e 's/Z$//' >"$name"/expect.out &&
+	test_cmp "$name"/expect.out "$name"/out &&
+	sed -e 's/^> //' -e 's/Z$//' <&3 >"$name"/expect.err &&
+	test_cmp "$name"/expect.err "$name"/err
 }
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [PATCH 7/8] test-lib tests: refactor common part of check_sub_test_lib_test*()
  2021-06-14 10:48 [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh Ævar Arnfjörð Bjarmason
                   ` (5 preceding siblings ...)
  2021-06-14 10:48 ` [PATCH 6/8] test-lib tests: avoid subshell for "test_cmp" for readability Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:48 ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:48 ` [PATCH 8/8] test-lib tests: assert 1 exit code, not non-zero Ævar Arnfjörð Bjarmason
                   ` (3 subsequent siblings)
  10 siblings, 0 replies; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:48 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Felipe Contreras, Adam Spiers,
	Thomas Rast, Ilya Bobyr, Ævar Arnfjörð Bjarmason

Refactor the two check_sub_test_lib_test*() functions to avoid
duplicating the same comparison they did of stdout. This duplication
was initially added when check_sub_test_lib_test_err() was added in
0445e6f0a12 (test-lib: '--run' to run only specific tests,
2014-04-30).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/lib-subtest.sh | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/t/lib-subtest.sh b/t/lib-subtest.sh
index 8f4935faf43..02864c9d0e0 100644
--- a/t/lib-subtest.sh
+++ b/t/lib-subtest.sh
@@ -87,18 +87,22 @@ run_sub_test_lib_test_err () {
 	_run_sub_test_lib_test_common '!' "$@"
 }
 
+_check_sub_test_lib_test_common () {
+	name="$1" &&
+	sed -e 's/^> //' -e 's/Z$//' >"$name"/expect.out &&
+	test_cmp "$name"/expect.out "$name"/out
+}
+
 check_sub_test_lib_test () {
 	name="$1" # stdin is the expected output from the test
-	test_must_be_empty "$name"/err &&
-	sed -e 's/^> //' -e 's/Z$//' >"$name"/expect &&
-	test_cmp "$name/"expect "$name"/out
+	_check_sub_test_lib_test_common "$name" &&
+	test_must_be_empty "$name"/err
 }
 
 check_sub_test_lib_test_err () {
 	name="$1" # stdin is the expected output from the test
+	_check_sub_test_lib_test_common "$name" &&
 	# expected error output is in descriptor 3
-	sed -e 's/^> //' -e 's/Z$//' >"$name"/expect.out &&
-	test_cmp "$name"/expect.out "$name"/out &&
 	sed -e 's/^> //' -e 's/Z$//' <&3 >"$name"/expect.err &&
 	test_cmp "$name"/expect.err "$name"/err
 }
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [PATCH 8/8] test-lib tests: assert 1 exit code, not non-zero
  2021-06-14 10:48 [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh Ævar Arnfjörð Bjarmason
                   ` (6 preceding siblings ...)
  2021-06-14 10:48 ` [PATCH 7/8] test-lib tests: refactor common part of check_sub_test_lib_test*() Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:48 ` Ævar Arnfjörð Bjarmason
  2021-06-15  2:24 ` [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh Junio C Hamano
                   ` (2 subsequent siblings)
  10 siblings, 0 replies; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:48 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Felipe Contreras, Adam Spiers,
	Thomas Rast, Ilya Bobyr, Ævar Arnfjörð Bjarmason

Improve the testing for test-lib.sh itself to assert that we have a
exit code of 1, not any non-zero. Improves code added in
0445e6f0a12 (test-lib: '--run' to run only specific tests,
2014-04-30).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/lib-subtest.sh | 21 +++++++++------------
 1 file changed, 9 insertions(+), 12 deletions(-)

diff --git a/t/lib-subtest.sh b/t/lib-subtest.sh
index 02864c9d0e0..bb7153ac9e5 100644
--- a/t/lib-subtest.sh
+++ b/t/lib-subtest.sh
@@ -43,8 +43,8 @@ write_sub_test_lib_test () {
 }
 
 _run_sub_test_lib_test_common () {
-	neg="$1" name="$2" # stdin is the body of the test code
-	shift 2
+	cmp_op="$1" want_code="$2" name="$3" # stdin is the body of the test code
+	shift 3
 	(
 		cd "$name" &&
 
@@ -58,33 +58,30 @@ _run_sub_test_lib_test_common () {
 		TEST_OUTPUT_DIRECTORY=$(pwd) &&
 		export TEST_OUTPUT_DIRECTORY &&
 		sane_unset GIT_TEST_FAIL_PREREQS &&
-		if test -z "$neg"
-		then
-			./"$name.sh" "$@" >out 2>err
-		else
-			! ./"$name.sh" "$@" >out 2>err
-		fi
+		./"$name.sh" "$@" >out 2>err;
+		ret=$? &&
+		test "$ret" "$cmp_op" "$want_code"
 	)
 }
 
 write_and_run_sub_test_lib_test () {
 	name="$1" descr="$2" # stdin is the body of the test code
 	write_sub_test_lib_test "$@" || return 1
-	_run_sub_test_lib_test_common '' "$@"
+	_run_sub_test_lib_test_common -eq 0 "$@"
 }
 
 write_and_run_sub_test_lib_test_err () {
 	name="$1" descr="$2" # stdin is the body of the test code
 	write_sub_test_lib_test "$@" || return 1
-	_run_sub_test_lib_test_common '!' "$@"
+	_run_sub_test_lib_test_common -eq 1 "$@"
 }
 
 run_sub_test_lib_test () {
-	_run_sub_test_lib_test_common '' "$@"
+	_run_sub_test_lib_test_common -eq 0 "$@"
 }
 
 run_sub_test_lib_test_err () {
-	_run_sub_test_lib_test_common '!' "$@"
+	_run_sub_test_lib_test_common -eq 1 "$@"
 }
 
 _check_sub_test_lib_test_common () {
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh
  2021-06-14 10:48 [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh Ævar Arnfjörð Bjarmason
                   ` (7 preceding siblings ...)
  2021-06-14 10:48 ` [PATCH 8/8] test-lib tests: assert 1 exit code, not non-zero Ævar Arnfjörð Bjarmason
@ 2021-06-15  2:24 ` Junio C Hamano
  2021-06-24 10:38   ` Ævar Arnfjörð Bjarmason
  2021-06-15 18:05 ` Felipe Contreras
  2021-07-21 22:57 ` [PATCH v2 " Ævar Arnfjörð Bjarmason
  10 siblings, 1 reply; 24+ messages in thread
From: Junio C Hamano @ 2021-06-15  2:24 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Jeff King, Felipe Contreras, Adam Spiers, Thomas Rast, Ilya Bobyr

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> This refactors the testing of test-lib.sh itself in t0000 into a
> lib-subtest.sh, fixing duplicate setup, bugs and various shell nits
> along the way.
> ...
> I thought it was better to submit this now anyway, while
> this code is fresh in people's minds.

Any time you send it anew is the time it is fresh in people's minds
if they read it, no?  I am not sure I understand the reasoning.

> I'm hoping Felipe will take some variant of my
> http://lore.kernel.org/git/8735tk22if.fsf@evledraar.gmail.com and
> perhaps review this series & base his patch on top of these changes,
> but alternatively they can go in independently with Junio solving the
> conflict.

I'd rather not if it can be avoided.  Please scale the process by
coordinating among contributors yourselves.

^ permalink raw reply	[flat|nested] 24+ messages in thread

* RE: [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh
  2021-06-14 10:48 [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh Ævar Arnfjörð Bjarmason
                   ` (8 preceding siblings ...)
  2021-06-15  2:24 ` [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh Junio C Hamano
@ 2021-06-15 18:05 ` Felipe Contreras
  2021-07-21 22:57 ` [PATCH v2 " Ævar Arnfjörð Bjarmason
  10 siblings, 0 replies; 24+ messages in thread
From: Felipe Contreras @ 2021-06-15 18:05 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason, git
  Cc: Junio C Hamano, Jeff King, Felipe Contreras, Adam Spiers,
	Thomas Rast, Ilya Bobyr, Ævar Arnfjörð Bjarmason

Ævar Arnfjörð Bjarmason wrote:

> There's a conflict here with Felipe's proposed
> http://lore.kernel.org/git/20210609170520.67014-1-felipe.contreras@gmail.com,
> but it's just a matter of moving the fix around and solving a trivial
> conflict. I thought it was better to submit this now anyway, while
> this code is fresh in people's minds.
> 
> I'm hoping Felipe will take some variant of my
> http://lore.kernel.org/git/8735tk22if.fsf@evledraar.gmail.com and
> perhaps review this series & base his patch on top of these changes,
> but alternatively they can go in independently with Junio solving the
> conflict.

I have no problem rebasing my patch on top of your series, but my
current v2 doesn't touch _run_sub_test_lib_test_common at all, and I'm
not sure your proposal of TEST_NO_RESULTS_OUTPUT is the way to go, so
perhaps there's no need to rebase at all.

Unless there's something on this series that would help
TEST_OUTPUT_DIRECTORY, which at a quick glance doesn't seem to be the
case.

Cheers.

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh
  2021-06-15  2:24 ` [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh Junio C Hamano
@ 2021-06-24 10:38   ` Ævar Arnfjörð Bjarmason
  2021-06-30  7:04     ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-24 10:38 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Jeff King, Felipe Contreras, Adam Spiers, Thomas Rast, Ilya Bobyr


On Tue, Jun 15 2021, Junio C Hamano wrote:

> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>
>> This refactors the testing of test-lib.sh itself in t0000 into a
>> lib-subtest.sh, fixing duplicate setup, bugs and various shell nits
>> along the way.
>> ...
>> I thought it was better to submit this now anyway, while
>> this code is fresh in people's minds.
>
> Any time you send it anew is the time it is fresh in people's minds
> if they read it, no?  I am not sure I understand the reasoning.
>
>> I'm hoping Felipe will take some variant of my
>> http://lore.kernel.org/git/8735tk22if.fsf@evledraar.gmail.com and
>> perhaps review this series & base his patch on top of these changes,
>> but alternatively they can go in independently with Junio solving the
>> conflict.
>
> I'd rather not if it can be avoided.  Please scale the process by
> coordinating among contributors yourselves.

Per Felipe's reply and his
https://lore.kernel.org/git/20210615183157.104999-1-felipe.contreras@gmail.com/
I don't think there's any conflict anymore, so this series should apply
on master without any conflicts.

It would be great to have it picked up, it's another set of test
cleanups I need for a subsequent series...

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh
  2021-06-24 10:38   ` Ævar Arnfjörð Bjarmason
@ 2021-06-30  7:04     ` Ævar Arnfjörð Bjarmason
  2021-07-01 14:30       ` Junio C Hamano
  0 siblings, 1 reply; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-30  7:04 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Jeff King, Felipe Contreras, Adam Spiers, Thomas Rast, Ilya Bobyr


On Thu, Jun 24 2021, Ævar Arnfjörð Bjarmason wrote:

> On Tue, Jun 15 2021, Junio C Hamano wrote:
>
>> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>>
>>> This refactors the testing of test-lib.sh itself in t0000 into a
>>> lib-subtest.sh, fixing duplicate setup, bugs and various shell nits
>>> along the way.
>>> ...
>>> I thought it was better to submit this now anyway, while
>>> this code is fresh in people's minds.
>>
>> Any time you send it anew is the time it is fresh in people's minds
>> if they read it, no?  I am not sure I understand the reasoning.
>>
>>> I'm hoping Felipe will take some variant of my
>>> http://lore.kernel.org/git/8735tk22if.fsf@evledraar.gmail.com and
>>> perhaps review this series & base his patch on top of these changes,
>>> but alternatively they can go in independently with Junio solving the
>>> conflict.
>>
>> I'd rather not if it can be avoided.  Please scale the process by
>> coordinating among contributors yourselves.
>
> Per Felipe's reply and his
> https://lore.kernel.org/git/20210615183157.104999-1-felipe.contreras@gmail.com/
> I don't think there's any conflict anymore, so this series should apply
> on master without any conflicts.
>
> It would be great to have it picked up, it's another set of test
> cleanups I need for a subsequent series...

Junio, just a reminder about this series: It has no conflicts with
"seen", and that WIP patch it would have conflicted with it was replaced
by another approach.

I've got some other non-testing changes queued on top of this, it would
be great to have it picked up. Thanks!

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh
  2021-06-30  7:04     ` Ævar Arnfjörð Bjarmason
@ 2021-07-01 14:30       ` Junio C Hamano
  0 siblings, 0 replies; 24+ messages in thread
From: Junio C Hamano @ 2021-07-01 14:30 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Jeff King, Felipe Contreras, Adam Spiers, Thomas Rast, Ilya Bobyr

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> On Thu, Jun 24 2021, Ævar Arnfjörð Bjarmason wrote:
>
>> On Tue, Jun 15 2021, Junio C Hamano wrote:
>>
>>> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>>>
>>>> This refactors the testing of test-lib.sh itself in t0000 into a
>>>> lib-subtest.sh, fixing duplicate setup, bugs and various shell nits
>>>> along the way.
> ...
> Junio, just a reminder about this series: It has no conflicts with
> "seen", and that WIP patch it would have conflicted with it was replaced
> by another approach.

https://lore.kernel.org/git/cover-0.8-00000000000-20210614T104351Z-avarab@gmail.com/
reveals that this hasn't seen much reviews, though.  I'll try to
find time to read it through; no promises if it will be done by the
end of the week, though.

Thanks.


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [PATCH v2 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh
  2021-06-14 10:48 [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh Ævar Arnfjörð Bjarmason
                   ` (9 preceding siblings ...)
  2021-06-15 18:05 ` Felipe Contreras
@ 2021-07-21 22:57 ` Ævar Arnfjörð Bjarmason
  2021-07-21 22:57   ` [PATCH v2 1/8] test-lib tests: move "run_sub_test" to a new lib-subtest.sh Ævar Arnfjörð Bjarmason
                     ` (8 more replies)
  10 siblings, 9 replies; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-21 22:57 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Felipe Contreras, Adam Spiers,
	Thomas Rast, Ilya Bobyr, Patrick Steinhardt,
	Ævar Arnfjörð Bjarmason

This refactors the testing of test-lib.sh itself in t0000 into a
lib-subtest.sh, fixing duplicate setup, bugs and various shell nits
along the way.

The v1[1] of this recently started conflicting with the cooking
jk/t0000-subtests-fix and ps/t0000-output-directory-fix, and got
ejected from "seen". This v2 is rebased on top of those two, so it
merges cleanly with "seen".

1. https://lore.kernel.org/git/cover-0.8-00000000000-20210614T104351Z-avarab@gmail.com/

Ævar Arnfjörð Bjarmason (8):
  test-lib tests: move "run_sub_test" to a new lib-subtest.sh
  test-lib tests: split up "write and run" into two functions
  test-lib tests: stop using a subshell in write_sub_test_lib_test()
  test-lib tests: don't provide a description for the sub-tests
  test-lib tests: get rid of copy/pasted mock test code
  test-lib tests: avoid subshell for "test_cmp" for readability
  test-lib tests: refactor common part of check_sub_test_lib_test*()
  test-lib tests: assert 1 exit code, not non-zero

 t/lib-subtest.sh | 128 ++++++++++++++
 t/t0000-basic.sh | 448 +++++++++++++----------------------------------
 2 files changed, 246 insertions(+), 330 deletions(-)
 create mode 100644 t/lib-subtest.sh

Range-diff against v1:
1:  3c0a78530d2 ! 1:  7a06ea3a7b7 test-lib tests: move "run_sub_test" to a new lib-subtest.sh
    @@ t/lib-subtest.sh (new)
     +_run_sub_test_lib_test_common () {
     +	neg="$1" name="$2" descr="$3" # stdin is the body of the test code
     +	shift 3
    ++
    ++	# intercept pseudo-options at the front of the argument list that we
    ++	# will not pass to child script
    ++	skip=
    ++	while test $# -gt 0
    ++	do
    ++		case "$1" in
    ++		--skip=*)
    ++			skip=${1#--*=}
    ++			shift
    ++			;;
    ++		*)
    ++			break
    ++			;;
    ++		esac
    ++	done
    ++
     +	mkdir "$name" &&
     +	(
     +		# Pretend we're not running under a test harness, whether we
    @@ t/lib-subtest.sh (new)
     +		EOF
     +		cat >>"$name.sh" &&
     +		export TEST_DIRECTORY &&
    -+		TEST_OUTPUT_DIRECTORY=$(pwd) &&
    -+		export TEST_OUTPUT_DIRECTORY &&
    ++		# The child test re-sources GIT-BUILD-OPTIONS and may thus
    ++		# override the test output directory. We thus pass it as an
    ++		# explicit override to the child.
    ++		TEST_OUTPUT_DIRECTORY_OVERRIDE=$(pwd) &&
    ++		export TEST_OUTPUT_DIRECTORY_OVERRIDE &&
    ++		GIT_SKIP_TESTS=$skip &&
    ++		export GIT_SKIP_TESTS &&
     +		sane_unset GIT_TEST_FAIL_PREREQS &&
     +		if test -z "$neg"
     +		then
    @@ t/t0000-basic.sh: test_expect_success 'success is reported like this' '
     -_run_sub_test_lib_test_common () {
     -	neg="$1" name="$2" descr="$3" # stdin is the body of the test code
     -	shift 3
    +-
    +-	# intercept pseudo-options at the front of the argument list that we
    +-	# will not pass to child script
    +-	skip=
    +-	while test $# -gt 0
    +-	do
    +-		case "$1" in
    +-		--skip=*)
    +-			skip=${1#--*=}
    +-			shift
    +-			;;
    +-		*)
    +-			break
    +-			;;
    +-		esac
    +-	done
    +-
     -	mkdir "$name" &&
     -	(
     -		# Pretend we're not running under a test harness, whether we
    @@ t/t0000-basic.sh: test_expect_success 'success is reported like this' '
     -		EOF
     -		cat >>"$name.sh" &&
     -		export TEST_DIRECTORY &&
    --		TEST_OUTPUT_DIRECTORY=$(pwd) &&
    --		export TEST_OUTPUT_DIRECTORY &&
    +-		# The child test re-sources GIT-BUILD-OPTIONS and may thus
    +-		# override the test output directory. We thus pass it as an
    +-		# explicit override to the child.
    +-		TEST_OUTPUT_DIRECTORY_OVERRIDE=$(pwd) &&
    +-		export TEST_OUTPUT_DIRECTORY_OVERRIDE &&
    +-		GIT_SKIP_TESTS=$skip &&
    +-		export GIT_SKIP_TESTS &&
     -		sane_unset GIT_TEST_FAIL_PREREQS &&
     -		if test -z "$neg"
     -		then
2:  9e2715d2e20 ! 2:  28117b84abf test-lib tests: split up "write and run" into two functions
    @@ Commit message
     
      ## t/lib-subtest.sh ##
     @@
    --_run_sub_test_lib_test_common () {
    --	neg="$1" name="$2" descr="$3" # stdin is the body of the test code
    --	shift 3
     +write_sub_test_lib_test () {
     +	name="$1" descr="$2" # stdin is the body of the test code
    - 	mkdir "$name" &&
    - 	(
    --		# Pretend we're not running under a test harness, whether we
    --		# are or not. The test-lib output depends on the setting of
    --		# this variable, so we need a stable setting under which to run
    --		# the sub-test.
    --		sane_unset HARNESS_ACTIVE &&
    - 		cd "$name" &&
    - 		write_script "$name.sh" "$TEST_SHELL_PATH" <<-EOF &&
    - 		test_description='$descr (run in sub test-lib)
    -@@ t/lib-subtest.sh: _run_sub_test_lib_test_common () {
    - 		# Point to the t/test-lib.sh, which isn't in ../ as usual
    - 		. "\$TEST_DIRECTORY"/test-lib.sh
    - 		EOF
    --		cat >>"$name.sh" &&
    ++	mkdir "$name" &&
    ++	(
    ++		cd "$name" &&
    ++		write_script "$name.sh" "$TEST_SHELL_PATH" <<-EOF &&
    ++		test_description='$descr (run in sub test-lib)
    ++
    ++		This is run in a sub test-lib so that we do not get incorrect
    ++		passing metrics
    ++		'
    ++
    ++		# Point to the t/test-lib.sh, which isn't in ../ as usual
    ++		. "\$TEST_DIRECTORY"/test-lib.sh
    ++		EOF
     +		cat >>"$name.sh"
     +	)
     +}
     +
    -+_run_sub_test_lib_test_common () {
    -+	neg="$1" name="$2" descr="$3" # stdin is the body of the test code
    -+	shift 3
    -+	(
    + _run_sub_test_lib_test_common () {
    + 	neg="$1" name="$2" descr="$3" # stdin is the body of the test code
    + 	shift 3
    +@@ t/lib-subtest.sh: _run_sub_test_lib_test_common () {
    + 		esac
    + 	done
    + 
    +-	mkdir "$name" &&
    + 	(
     +		cd "$name" &&
     +
    -+		# Pretend we're not running under a test harness, whether we
    -+		# are or not. The test-lib output depends on the setting of
    -+		# this variable, so we need a stable setting under which to run
    -+		# the sub-test.
    -+		sane_unset HARNESS_ACTIVE &&
    -+
    + 		# Pretend we're not running under a test harness, whether we
    + 		# are or not. The test-lib output depends on the setting of
    + 		# this variable, so we need a stable setting under which to run
    + 		# the sub-test.
    + 		sane_unset HARNESS_ACTIVE &&
    +-		cd "$name" &&
    +-		write_script "$name.sh" "$TEST_SHELL_PATH" <<-EOF &&
    +-		test_description='$descr (run in sub test-lib)
    + 
    +-		This is run in a sub test-lib so that we do not get incorrect
    +-		passing metrics
    +-		'
    +-
    +-		# Point to the t/test-lib.sh, which isn't in ../ as usual
    +-		. "\$TEST_DIRECTORY"/test-lib.sh
    +-		EOF
    +-		cat >>"$name.sh" &&
      		export TEST_DIRECTORY &&
    - 		TEST_OUTPUT_DIRECTORY=$(pwd) &&
    - 		export TEST_OUTPUT_DIRECTORY &&
    + 		# The child test re-sources GIT-BUILD-OPTIONS and may thus
    + 		# override the test output directory. We thus pass it as an
     @@ t/lib-subtest.sh: _run_sub_test_lib_test_common () {
      	)
      }
    @@ t/lib-subtest.sh: _run_sub_test_lib_test_common () {
      run_sub_test_lib_test () {
      	_run_sub_test_lib_test_common '' "$@"
      }
    +@@ t/lib-subtest.sh: check_sub_test_lib_test_err () {
    + 		test_cmp expect.err err
    + 	)
    + }
    ++
     
      ## t/t0000-basic.sh ##
     @@ t/t0000-basic.sh: test_expect_success 'success is reported like this' '
    @@ t/t0000-basic.sh: test_expect_success 'test --verbose' '
      		--verbose-only=2 <<-\EOF &&
      	test_expect_success "passing test" true
     @@ t/t0000-basic.sh: test_expect_success 'test --verbose-only' '
    + 
      test_expect_success 'GIT_SKIP_TESTS' '
      	(
    - 		GIT_SKIP_TESTS="git.2" && export GIT_SKIP_TESTS &&
     -		run_sub_test_lib_test git-skip-tests-basic \
     +		write_and_run_sub_test_lib_test git-skip-tests-basic \
    - 			"GIT_SKIP_TESTS" <<-\EOF &&
    + 			"GIT_SKIP_TESTS" \
    + 			--skip="git.2" <<-\EOF &&
      		for i in 1 2 3
    - 		do
     @@ t/t0000-basic.sh: test_expect_success 'GIT_SKIP_TESTS' '
    + 
      test_expect_success 'GIT_SKIP_TESTS several tests' '
      	(
    - 		GIT_SKIP_TESTS="git.2 git.5" && export GIT_SKIP_TESTS &&
     -		run_sub_test_lib_test git-skip-tests-several \
     +		write_and_run_sub_test_lib_test git-skip-tests-several \
    - 			"GIT_SKIP_TESTS several tests" <<-\EOF &&
    + 			"GIT_SKIP_TESTS several tests" \
    + 			--skip="git.2 git.5" <<-\EOF &&
      		for i in 1 2 3 4 5 6
    - 		do
     @@ t/t0000-basic.sh: test_expect_success 'GIT_SKIP_TESTS several tests' '
    + 
      test_expect_success 'GIT_SKIP_TESTS sh pattern' '
      	(
    - 		GIT_SKIP_TESTS="git.[2-5]" && export GIT_SKIP_TESTS &&
     -		run_sub_test_lib_test git-skip-tests-sh-pattern \
     +		write_and_run_sub_test_lib_test git-skip-tests-sh-pattern \
    - 			"GIT_SKIP_TESTS sh pattern" <<-\EOF &&
    + 			"GIT_SKIP_TESTS sh pattern" \
    + 			--skip="git.[2-5]" <<-\EOF &&
      		for i in 1 2 3 4 5 6
    - 		do
     @@ t/t0000-basic.sh: test_expect_success 'GIT_SKIP_TESTS sh pattern' '
    + 
      test_expect_success 'GIT_SKIP_TESTS entire suite' '
      	(
    - 		GIT_SKIP_TESTS="git" && export GIT_SKIP_TESTS &&
     -		run_sub_test_lib_test git-skip-tests-entire-suite \
     +		write_and_run_sub_test_lib_test git-skip-tests-entire-suite \
    - 			"GIT_SKIP_TESTS entire suite" <<-\EOF &&
    + 			"GIT_SKIP_TESTS entire suite" \
    + 			--skip="git" <<-\EOF &&
      		for i in 1 2 3
    - 		do
     @@ t/t0000-basic.sh: test_expect_success 'GIT_SKIP_TESTS entire suite' '
    + 
      test_expect_success 'GIT_SKIP_TESTS does not skip unmatched suite' '
      	(
    - 		GIT_SKIP_TESTS="notgit" && export GIT_SKIP_TESTS &&
     -		run_sub_test_lib_test git-skip-tests-unmatched-suite \
     +		write_and_run_sub_test_lib_test git-skip-tests-unmatched-suite \
    - 			"GIT_SKIP_TESTS does not skip unmatched suite" <<-\EOF &&
    + 			"GIT_SKIP_TESTS does not skip unmatched suite" \
    + 			--skip="notgit" <<-\EOF &&
      		for i in 1 2 3
    - 		do
     @@ t/t0000-basic.sh: test_expect_success 'GIT_SKIP_TESTS does not skip unmatched suite' '
      '
      
3:  cd6c740c912 = 3:  7d5ed335a36 test-lib tests: stop using a subshell in write_sub_test_lib_test()
4:  0578470d899 ! 4:  2a0dd64da90 test-lib tests: don't provide a description for the sub-tests
    @@ t/lib-subtest.sh: write_sub_test_lib_test () {
     -	shift 3
     +	neg="$1" name="$2" # stdin is the body of the test code
     +	shift 2
    - 	(
    - 		cd "$name" &&
      
    + 	# intercept pseudo-options at the front of the argument list that we
    + 	# will not pass to child script
     
      ## t/t0000-basic.sh ##
     @@ t/t0000-basic.sh: test_expect_success 'success is reported like this' '
    @@ t/t0000-basic.sh: test_expect_success 'test --verbose-only' '
     -test_expect_success 'GIT_SKIP_TESTS' '
     +test_expect_success 'subtest: skip one with GIT_SKIP_TESTS' '
      	(
    - 		GIT_SKIP_TESTS="git.2" && export GIT_SKIP_TESTS &&
      		write_and_run_sub_test_lib_test git-skip-tests-basic \
    --			"GIT_SKIP_TESTS" <<-\EOF &&
    -+			<<-\EOF &&
    +-			"GIT_SKIP_TESTS" \
    + 			--skip="git.2" <<-\EOF &&
      		for i in 1 2 3
      		do
    - 			test_expect_success "passing test #$i" "true"
     @@ t/t0000-basic.sh: test_expect_success 'GIT_SKIP_TESTS' '
      	)
      '
    @@ t/t0000-basic.sh: test_expect_success 'GIT_SKIP_TESTS' '
     -test_expect_success 'GIT_SKIP_TESTS several tests' '
     +test_expect_success 'subtest: skip several with GIT_SKIP_TESTS' '
      	(
    - 		GIT_SKIP_TESTS="git.2 git.5" && export GIT_SKIP_TESTS &&
      		write_and_run_sub_test_lib_test git-skip-tests-several \
    --			"GIT_SKIP_TESTS several tests" <<-\EOF &&
    -+			<<-\EOF &&
    +-			"GIT_SKIP_TESTS several tests" \
    + 			--skip="git.2 git.5" <<-\EOF &&
      		for i in 1 2 3 4 5 6
      		do
    - 			test_expect_success "passing test #$i" "true"
     @@ t/t0000-basic.sh: test_expect_success 'GIT_SKIP_TESTS several tests' '
      	)
      '
    @@ t/t0000-basic.sh: test_expect_success 'GIT_SKIP_TESTS several tests' '
     -test_expect_success 'GIT_SKIP_TESTS sh pattern' '
     +test_expect_success 'subtest: sh pattern skipping with GIT_SKIP_TESTS' '
      	(
    - 		GIT_SKIP_TESTS="git.[2-5]" && export GIT_SKIP_TESTS &&
      		write_and_run_sub_test_lib_test git-skip-tests-sh-pattern \
    --			"GIT_SKIP_TESTS sh pattern" <<-\EOF &&
    -+			<<-\EOF &&
    +-			"GIT_SKIP_TESTS sh pattern" \
    + 			--skip="git.[2-5]" <<-\EOF &&
      		for i in 1 2 3 4 5 6
      		do
    - 			test_expect_success "passing test #$i" "true"
     @@ t/t0000-basic.sh: test_expect_success 'GIT_SKIP_TESTS sh pattern' '
      	)
      '
    @@ t/t0000-basic.sh: test_expect_success 'GIT_SKIP_TESTS sh pattern' '
     -test_expect_success 'GIT_SKIP_TESTS entire suite' '
     +test_expect_success 'subtest: skip entire test suite with GIT_SKIP_TESTS' '
      	(
    - 		GIT_SKIP_TESTS="git" && export GIT_SKIP_TESTS &&
      		write_and_run_sub_test_lib_test git-skip-tests-entire-suite \
    --			"GIT_SKIP_TESTS entire suite" <<-\EOF &&
    -+			<<-\EOF &&
    +-			"GIT_SKIP_TESTS entire suite" \
    + 			--skip="git" <<-\EOF &&
      		for i in 1 2 3
      		do
    - 			test_expect_success "passing test #$i" "true"
     @@ t/t0000-basic.sh: test_expect_success 'GIT_SKIP_TESTS entire suite' '
      	)
      '
    @@ t/t0000-basic.sh: test_expect_success 'GIT_SKIP_TESTS entire suite' '
     -test_expect_success 'GIT_SKIP_TESTS does not skip unmatched suite' '
     +test_expect_success 'subtest: GIT_SKIP_TESTS does not skip unmatched suite' '
      	(
    - 		GIT_SKIP_TESTS="notgit" && export GIT_SKIP_TESTS &&
      		write_and_run_sub_test_lib_test git-skip-tests-unmatched-suite \
    --			"GIT_SKIP_TESTS does not skip unmatched suite" <<-\EOF &&
    -+			<<-\EOF &&
    +-			"GIT_SKIP_TESTS does not skip unmatched suite" \
    + 			--skip="notgit" <<-\EOF &&
      		for i in 1 2 3
      		do
    - 			test_expect_success "passing test #$i" "true"
     @@ t/t0000-basic.sh: test_expect_success 'GIT_SKIP_TESTS does not skip unmatched suite' '
      	)
      '
5:  1909616d9ca ! 5:  f60190dec6a test-lib tests: get rid of copy/pasted mock test code
    @@ t/t0000-basic.sh: test_expect_success 'subtest: --verbose-only option' '
      
      test_expect_success 'subtest: skip one with GIT_SKIP_TESTS' '
      	(
    --		GIT_SKIP_TESTS="git.2" && export GIT_SKIP_TESTS &&
     -		write_and_run_sub_test_lib_test git-skip-tests-basic \
    --			<<-\EOF &&
    +-			--skip="git.2" <<-\EOF &&
     -		for i in 1 2 3
     -		do
     -			test_expect_success "passing test #$i" "true"
    @@ t/t0000-basic.sh: test_expect_success 'subtest: --verbose-only option' '
     -		test_done
     -		EOF
     -		check_sub_test_lib_test git-skip-tests-basic <<-\EOF
    -+		GIT_SKIP_TESTS="full.2" && export GIT_SKIP_TESTS &&
    -+		run_sub_test_lib_test full-pass &&
    ++		run_sub_test_lib_test full-pass \
    ++			--skip="full.2" &&
     +		check_sub_test_lib_test full-pass <<-\EOF
      		> ok 1 - passing test #1
      		> ok 2 # skip passing test #2 (GIT_SKIP_TESTS)
      		> ok 3 - passing test #3
     @@ t/t0000-basic.sh: test_expect_success 'subtest: skip several with GIT_SKIP_TESTS' '
    + 
      test_expect_success 'subtest: sh pattern skipping with GIT_SKIP_TESTS' '
      	(
    - 		GIT_SKIP_TESTS="git.[2-5]" && export GIT_SKIP_TESTS &&
     -		write_and_run_sub_test_lib_test git-skip-tests-sh-pattern \
    --			<<-\EOF &&
    +-			--skip="git.[2-5]" <<-\EOF &&
     -		for i in 1 2 3 4 5 6
     -		do
     -			test_expect_success "passing test #$i" "true"
    @@ t/t0000-basic.sh: test_expect_success 'subtest: skip several with GIT_SKIP_TESTS
     -		test_done
     -		EOF
     -		check_sub_test_lib_test git-skip-tests-sh-pattern <<-\EOF
    -+		run_sub_test_lib_test git-skip-tests-several &&
    ++		run_sub_test_lib_test git-skip-tests-several \
    ++			--skip="git.[2-5]" &&
     +		check_sub_test_lib_test git-skip-tests-several <<-\EOF
      		> ok 1 - passing test #1
      		> ok 2 # skip passing test #2 (GIT_SKIP_TESTS)
      		> ok 3 # skip passing test #3 (GIT_SKIP_TESTS)
     @@ t/t0000-basic.sh: test_expect_success 'subtest: sh pattern skipping with GIT_SKIP_TESTS' '
    + 
      test_expect_success 'subtest: skip entire test suite with GIT_SKIP_TESTS' '
      	(
    - 		GIT_SKIP_TESTS="git" && export GIT_SKIP_TESTS &&
     -		write_and_run_sub_test_lib_test git-skip-tests-entire-suite \
    --			<<-\EOF &&
    +-			--skip="git" <<-\EOF &&
     -		for i in 1 2 3
     -		do
     -			test_expect_success "passing test #$i" "true"
    @@ t/t0000-basic.sh: test_expect_success 'subtest: sh pattern skipping with GIT_SKI
     -		test_done
     -		EOF
     -		check_sub_test_lib_test git-skip-tests-entire-suite <<-\EOF
    -+		run_sub_test_lib_test git-skip-tests-several &&
    ++		GIT_SKIP_TESTS="git" && export GIT_SKIP_TESTS &&
    ++		run_sub_test_lib_test git-skip-tests-several \
    ++			--skip="git" &&
     +		check_sub_test_lib_test git-skip-tests-several <<-\EOF
      		> 1..0 # SKIP skip all tests in git
      		EOF
      	)
     @@ t/t0000-basic.sh: test_expect_success 'subtest: skip entire test suite with GIT_SKIP_TESTS' '
    + 
      test_expect_success 'subtest: GIT_SKIP_TESTS does not skip unmatched suite' '
      	(
    - 		GIT_SKIP_TESTS="notgit" && export GIT_SKIP_TESTS &&
     -		write_and_run_sub_test_lib_test git-skip-tests-unmatched-suite \
    --			<<-\EOF &&
    +-			--skip="notgit" <<-\EOF &&
     -		for i in 1 2 3
     -		do
     -			test_expect_success "passing test #$i" "true"
    @@ t/t0000-basic.sh: test_expect_success 'subtest: skip entire test suite with GIT_
     -		test_done
     -		EOF
     -		check_sub_test_lib_test git-skip-tests-unmatched-suite <<-\EOF
    -+		run_sub_test_lib_test full-pass &&
    ++		GIT_SKIP_TESTS="notgit" && export GIT_SKIP_TESTS &&
    ++		run_sub_test_lib_test full-pass \
    ++			--skip="notfull" &&
     +		check_sub_test_lib_test full-pass <<-\EOF
      		> ok 1 - passing test #1
      		> ok 2 - passing test #2
6:  2f717e437e6 ! 6:  022ddbabf63 test-lib tests: avoid subshell for "test_cmp" for readability
    @@ t/lib-subtest.sh: run_sub_test_lib_test_err () {
     +	sed -e 's/^> //' -e 's/Z$//' <&3 >"$name"/expect.err &&
     +	test_cmp "$name"/expect.err "$name"/err
      }
    + 
7:  1f32eb60411 = 7:  ae0226e164c test-lib tests: refactor common part of check_sub_test_lib_test*()
8:  ddbc8db2802 ! 8:  676547e001e test-lib tests: assert 1 exit code, not non-zero
    @@ t/lib-subtest.sh: write_sub_test_lib_test () {
     -	shift 2
     +	cmp_op="$1" want_code="$2" name="$3" # stdin is the body of the test code
     +	shift 3
    - 	(
    - 		cd "$name" &&
      
    + 	# intercept pseudo-options at the front of the argument list that we
    + 	# will not pass to child script
     @@ t/lib-subtest.sh: _run_sub_test_lib_test_common () {
    - 		TEST_OUTPUT_DIRECTORY=$(pwd) &&
    - 		export TEST_OUTPUT_DIRECTORY &&
    + 		GIT_SKIP_TESTS=$skip &&
    + 		export GIT_SKIP_TESTS &&
      		sane_unset GIT_TEST_FAIL_PREREQS &&
     -		if test -z "$neg"
     -		then
-- 
2.32.0.955.ge7c5360f7e7


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [PATCH v2 1/8] test-lib tests: move "run_sub_test" to a new lib-subtest.sh
  2021-07-21 22:57 ` [PATCH v2 " Ævar Arnfjörð Bjarmason
@ 2021-07-21 22:57   ` Ævar Arnfjörð Bjarmason
  2021-07-21 22:57   ` [PATCH v2 2/8] test-lib tests: split up "write and run" into two functions Ævar Arnfjörð Bjarmason
                     ` (7 subsequent siblings)
  8 siblings, 0 replies; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-21 22:57 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Felipe Contreras, Adam Spiers,
	Thomas Rast, Ilya Bobyr, Patrick Steinhardt,
	Ævar Arnfjörð Bjarmason

Move the "check_sub_test_lib_test()" and its sister functions to a new
lib-subtest.sh.

In the future (not in this series) I'd like to test test-lib's output
in a more targeted and smaller test, and I'll need these functions to
do that.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/lib-subtest.sh | 86 ++++++++++++++++++++++++++++++++++++++++++++++
 t/t0000-basic.sh | 88 +-----------------------------------------------
 2 files changed, 87 insertions(+), 87 deletions(-)
 create mode 100644 t/lib-subtest.sh

diff --git a/t/lib-subtest.sh b/t/lib-subtest.sh
new file mode 100644
index 00000000000..3cfe09911a2
--- /dev/null
+++ b/t/lib-subtest.sh
@@ -0,0 +1,86 @@
+_run_sub_test_lib_test_common () {
+	neg="$1" name="$2" descr="$3" # stdin is the body of the test code
+	shift 3
+
+	# intercept pseudo-options at the front of the argument list that we
+	# will not pass to child script
+	skip=
+	while test $# -gt 0
+	do
+		case "$1" in
+		--skip=*)
+			skip=${1#--*=}
+			shift
+			;;
+		*)
+			break
+			;;
+		esac
+	done
+
+	mkdir "$name" &&
+	(
+		# Pretend we're not running under a test harness, whether we
+		# are or not. The test-lib output depends on the setting of
+		# this variable, so we need a stable setting under which to run
+		# the sub-test.
+		sane_unset HARNESS_ACTIVE &&
+		cd "$name" &&
+		write_script "$name.sh" "$TEST_SHELL_PATH" <<-EOF &&
+		test_description='$descr (run in sub test-lib)
+
+		This is run in a sub test-lib so that we do not get incorrect
+		passing metrics
+		'
+
+		# Point to the t/test-lib.sh, which isn't in ../ as usual
+		. "\$TEST_DIRECTORY"/test-lib.sh
+		EOF
+		cat >>"$name.sh" &&
+		export TEST_DIRECTORY &&
+		# The child test re-sources GIT-BUILD-OPTIONS and may thus
+		# override the test output directory. We thus pass it as an
+		# explicit override to the child.
+		TEST_OUTPUT_DIRECTORY_OVERRIDE=$(pwd) &&
+		export TEST_OUTPUT_DIRECTORY_OVERRIDE &&
+		GIT_SKIP_TESTS=$skip &&
+		export GIT_SKIP_TESTS &&
+		sane_unset GIT_TEST_FAIL_PREREQS &&
+		if test -z "$neg"
+		then
+			./"$name.sh" "$@" >out 2>err
+		else
+			! ./"$name.sh" "$@" >out 2>err
+		fi
+	)
+}
+
+run_sub_test_lib_test () {
+	_run_sub_test_lib_test_common '' "$@"
+}
+
+run_sub_test_lib_test_err () {
+	_run_sub_test_lib_test_common '!' "$@"
+}
+
+check_sub_test_lib_test () {
+	name="$1" # stdin is the expected output from the test
+	(
+		cd "$name" &&
+		test_must_be_empty err &&
+		sed -e 's/^> //' -e 's/Z$//' >expect &&
+		test_cmp expect out
+	)
+}
+
+check_sub_test_lib_test_err () {
+	name="$1" # stdin is the expected output from the test
+	# expected error output is in descriptor 3
+	(
+		cd "$name" &&
+		sed -e 's/^> //' -e 's/Z$//' >expect.out &&
+		test_cmp expect.out out &&
+		sed -e 's/^> //' -e 's/Z$//' <&3 >expect.err &&
+		test_cmp expect.err err
+	)
+}
diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index cb87768513c..a3865dd77ba 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -19,6 +19,7 @@ modification *should* take notice and update the test vectors here.
 '
 
 . ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-subtest.sh
 
 try_local_xy () {
 	local x="local" y="alsolocal" &&
@@ -66,93 +67,6 @@ test_expect_success 'success is reported like this' '
 	:
 '
 
-_run_sub_test_lib_test_common () {
-	neg="$1" name="$2" descr="$3" # stdin is the body of the test code
-	shift 3
-
-	# intercept pseudo-options at the front of the argument list that we
-	# will not pass to child script
-	skip=
-	while test $# -gt 0
-	do
-		case "$1" in
-		--skip=*)
-			skip=${1#--*=}
-			shift
-			;;
-		*)
-			break
-			;;
-		esac
-	done
-
-	mkdir "$name" &&
-	(
-		# Pretend we're not running under a test harness, whether we
-		# are or not. The test-lib output depends on the setting of
-		# this variable, so we need a stable setting under which to run
-		# the sub-test.
-		sane_unset HARNESS_ACTIVE &&
-		cd "$name" &&
-		write_script "$name.sh" "$TEST_SHELL_PATH" <<-EOF &&
-		test_description='$descr (run in sub test-lib)
-
-		This is run in a sub test-lib so that we do not get incorrect
-		passing metrics
-		'
-
-		# Point to the t/test-lib.sh, which isn't in ../ as usual
-		. "\$TEST_DIRECTORY"/test-lib.sh
-		EOF
-		cat >>"$name.sh" &&
-		export TEST_DIRECTORY &&
-		# The child test re-sources GIT-BUILD-OPTIONS and may thus
-		# override the test output directory. We thus pass it as an
-		# explicit override to the child.
-		TEST_OUTPUT_DIRECTORY_OVERRIDE=$(pwd) &&
-		export TEST_OUTPUT_DIRECTORY_OVERRIDE &&
-		GIT_SKIP_TESTS=$skip &&
-		export GIT_SKIP_TESTS &&
-		sane_unset GIT_TEST_FAIL_PREREQS &&
-		if test -z "$neg"
-		then
-			./"$name.sh" "$@" >out 2>err
-		else
-			! ./"$name.sh" "$@" >out 2>err
-		fi
-	)
-}
-
-run_sub_test_lib_test () {
-	_run_sub_test_lib_test_common '' "$@"
-}
-
-run_sub_test_lib_test_err () {
-	_run_sub_test_lib_test_common '!' "$@"
-}
-
-check_sub_test_lib_test () {
-	name="$1" # stdin is the expected output from the test
-	(
-		cd "$name" &&
-		test_must_be_empty err &&
-		sed -e 's/^> //' -e 's/Z$//' >expect &&
-		test_cmp expect out
-	)
-}
-
-check_sub_test_lib_test_err () {
-	name="$1" # stdin is the expected output from the test
-	# expected error output is in descriptor 3
-	(
-		cd "$name" &&
-		sed -e 's/^> //' -e 's/Z$//' >expect.out &&
-		test_cmp expect.out out &&
-		sed -e 's/^> //' -e 's/Z$//' <&3 >expect.err &&
-		test_cmp expect.err err
-	)
-}
-
 test_expect_success 'pretend we have a fully passing test suite' '
 	run_sub_test_lib_test full-pass "3 passing tests" <<-\EOF &&
 	for i in 1 2 3
-- 
2.32.0.955.ge7c5360f7e7


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [PATCH v2 2/8] test-lib tests: split up "write and run" into two functions
  2021-07-21 22:57 ` [PATCH v2 " Ævar Arnfjörð Bjarmason
  2021-07-21 22:57   ` [PATCH v2 1/8] test-lib tests: move "run_sub_test" to a new lib-subtest.sh Ævar Arnfjörð Bjarmason
@ 2021-07-21 22:57   ` Ævar Arnfjörð Bjarmason
  2021-07-21 22:57   ` [PATCH v2 3/8] test-lib tests: stop using a subshell in write_sub_test_lib_test() Ævar Arnfjörð Bjarmason
                     ` (6 subsequent siblings)
  8 siblings, 0 replies; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-21 22:57 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Felipe Contreras, Adam Spiers,
	Thomas Rast, Ilya Bobyr, Patrick Steinhardt,
	Ævar Arnfjörð Bjarmason

Refactor the function to write and run tests of the test-lib.sh output
into two functions.

When this was added back in 565b6fa87bb (tests: refactor mechanics of
testing in a sub test-lib, 2012-12-16) there was no reason to do this,
but since we started supporting test arguments in
517cd55fd51 (test-lib: self-test that --verbose works, 2013-06-23)
we've started to write out duplicate tests simply to test different
arguments, now we'll be able to re-use them.

This change doesn't consolidate any of those tests yet, it just makes
it possible to do so. All the changes in t0000-basic.sh are a simple
search-replacement.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/lib-subtest.sh | 46 ++++++++++++++++++++++--------
 t/t0000-basic.sh | 74 ++++++++++++++++++++++++------------------------
 2 files changed, 71 insertions(+), 49 deletions(-)

diff --git a/t/lib-subtest.sh b/t/lib-subtest.sh
index 3cfe09911a2..8f3f6dddcf4 100644
--- a/t/lib-subtest.sh
+++ b/t/lib-subtest.sh
@@ -1,3 +1,22 @@
+write_sub_test_lib_test () {
+	name="$1" descr="$2" # stdin is the body of the test code
+	mkdir "$name" &&
+	(
+		cd "$name" &&
+		write_script "$name.sh" "$TEST_SHELL_PATH" <<-EOF &&
+		test_description='$descr (run in sub test-lib)
+
+		This is run in a sub test-lib so that we do not get incorrect
+		passing metrics
+		'
+
+		# Point to the t/test-lib.sh, which isn't in ../ as usual
+		. "\$TEST_DIRECTORY"/test-lib.sh
+		EOF
+		cat >>"$name.sh"
+	)
+}
+
 _run_sub_test_lib_test_common () {
 	neg="$1" name="$2" descr="$3" # stdin is the body of the test code
 	shift 3
@@ -18,25 +37,15 @@ _run_sub_test_lib_test_common () {
 		esac
 	done
 
-	mkdir "$name" &&
 	(
+		cd "$name" &&
+
 		# Pretend we're not running under a test harness, whether we
 		# are or not. The test-lib output depends on the setting of
 		# this variable, so we need a stable setting under which to run
 		# the sub-test.
 		sane_unset HARNESS_ACTIVE &&
-		cd "$name" &&
-		write_script "$name.sh" "$TEST_SHELL_PATH" <<-EOF &&
-		test_description='$descr (run in sub test-lib)
 
-		This is run in a sub test-lib so that we do not get incorrect
-		passing metrics
-		'
-
-		# Point to the t/test-lib.sh, which isn't in ../ as usual
-		. "\$TEST_DIRECTORY"/test-lib.sh
-		EOF
-		cat >>"$name.sh" &&
 		export TEST_DIRECTORY &&
 		# The child test re-sources GIT-BUILD-OPTIONS and may thus
 		# override the test output directory. We thus pass it as an
@@ -55,6 +64,18 @@ _run_sub_test_lib_test_common () {
 	)
 }
 
+write_and_run_sub_test_lib_test () {
+	name="$1" descr="$2" # stdin is the body of the test code
+	write_sub_test_lib_test "$@" || return 1
+	_run_sub_test_lib_test_common '' "$@"
+}
+
+write_and_run_sub_test_lib_test_err () {
+	name="$1" descr="$2" # stdin is the body of the test code
+	write_sub_test_lib_test "$@" || return 1
+	_run_sub_test_lib_test_common '!' "$@"
+}
+
 run_sub_test_lib_test () {
 	_run_sub_test_lib_test_common '' "$@"
 }
@@ -84,3 +105,4 @@ check_sub_test_lib_test_err () {
 		test_cmp expect.err err
 	)
 }
+
diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index a3865dd77ba..a7c5aaacab6 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -68,7 +68,7 @@ test_expect_success 'success is reported like this' '
 '
 
 test_expect_success 'pretend we have a fully passing test suite' '
-	run_sub_test_lib_test full-pass "3 passing tests" <<-\EOF &&
+	write_and_run_sub_test_lib_test full-pass "3 passing tests" <<-\EOF &&
 	for i in 1 2 3
 	do
 		test_expect_success "passing test #$i" "true"
@@ -85,7 +85,7 @@ test_expect_success 'pretend we have a fully passing test suite' '
 '
 
 test_expect_success 'pretend we have a partially passing test suite' '
-	run_sub_test_lib_test_err \
+	write_and_run_sub_test_lib_test_err \
 		partial-pass "2/3 tests passing" <<-\EOF &&
 	test_expect_success "passing test #1" "true"
 	test_expect_success "failing test #2" "false"
@@ -103,7 +103,7 @@ test_expect_success 'pretend we have a partially passing test suite' '
 '
 
 test_expect_success 'pretend we have a known breakage' '
-	run_sub_test_lib_test failing-todo "A failing TODO test" <<-\EOF &&
+	write_and_run_sub_test_lib_test failing-todo "A failing TODO test" <<-\EOF &&
 	test_expect_success "passing test" "true"
 	test_expect_failure "pretend we have a known breakage" "false"
 	test_done
@@ -118,7 +118,7 @@ test_expect_success 'pretend we have a known breakage' '
 '
 
 test_expect_success 'pretend we have fixed a known breakage' '
-	run_sub_test_lib_test passing-todo "A passing TODO test" <<-\EOF &&
+	write_and_run_sub_test_lib_test passing-todo "A passing TODO test" <<-\EOF &&
 	test_expect_failure "pretend we have fixed a known breakage" "true"
 	test_done
 	EOF
@@ -130,7 +130,7 @@ test_expect_success 'pretend we have fixed a known breakage' '
 '
 
 test_expect_success 'pretend we have fixed one of two known breakages (run in sub test-lib)' '
-	run_sub_test_lib_test partially-passing-todos \
+	write_and_run_sub_test_lib_test partially-passing-todos \
 		"2 TODO tests, one passing" <<-\EOF &&
 	test_expect_failure "pretend we have a known breakage" "false"
 	test_expect_success "pretend we have a passing test" "true"
@@ -149,7 +149,7 @@ test_expect_success 'pretend we have fixed one of two known breakages (run in su
 '
 
 test_expect_success 'pretend we have a pass, fail, and known breakage' '
-	run_sub_test_lib_test_err \
+	write_and_run_sub_test_lib_test_err \
 		mixed-results1 "mixed results #1" <<-\EOF &&
 	test_expect_success "passing test" "true"
 	test_expect_success "failing test" "false"
@@ -168,7 +168,7 @@ test_expect_success 'pretend we have a pass, fail, and known breakage' '
 '
 
 test_expect_success 'pretend we have a mix of all possible results' '
-	run_sub_test_lib_test_err \
+	write_and_run_sub_test_lib_test_err \
 		mixed-results2 "mixed results #2" <<-\EOF &&
 	test_expect_success "passing test" "true"
 	test_expect_success "passing test" "true"
@@ -204,7 +204,7 @@ test_expect_success 'pretend we have a mix of all possible results' '
 '
 
 test_expect_success 'test --verbose' '
-	run_sub_test_lib_test_err \
+	write_and_run_sub_test_lib_test_err \
 		t1234-verbose "test verbose" --verbose <<-\EOF &&
 	test_expect_success "passing test" true
 	test_expect_success "test with output" "echo foo"
@@ -231,7 +231,7 @@ test_expect_success 'test --verbose' '
 '
 
 test_expect_success 'test --verbose-only' '
-	run_sub_test_lib_test_err \
+	write_and_run_sub_test_lib_test_err \
 		t2345-verbose-only-2 "test verbose-only=2" \
 		--verbose-only=2 <<-\EOF &&
 	test_expect_success "passing test" true
@@ -255,7 +255,7 @@ test_expect_success 'test --verbose-only' '
 
 test_expect_success 'GIT_SKIP_TESTS' '
 	(
-		run_sub_test_lib_test git-skip-tests-basic \
+		write_and_run_sub_test_lib_test git-skip-tests-basic \
 			"GIT_SKIP_TESTS" \
 			--skip="git.2" <<-\EOF &&
 		for i in 1 2 3
@@ -276,7 +276,7 @@ test_expect_success 'GIT_SKIP_TESTS' '
 
 test_expect_success 'GIT_SKIP_TESTS several tests' '
 	(
-		run_sub_test_lib_test git-skip-tests-several \
+		write_and_run_sub_test_lib_test git-skip-tests-several \
 			"GIT_SKIP_TESTS several tests" \
 			--skip="git.2 git.5" <<-\EOF &&
 		for i in 1 2 3 4 5 6
@@ -300,7 +300,7 @@ test_expect_success 'GIT_SKIP_TESTS several tests' '
 
 test_expect_success 'GIT_SKIP_TESTS sh pattern' '
 	(
-		run_sub_test_lib_test git-skip-tests-sh-pattern \
+		write_and_run_sub_test_lib_test git-skip-tests-sh-pattern \
 			"GIT_SKIP_TESTS sh pattern" \
 			--skip="git.[2-5]" <<-\EOF &&
 		for i in 1 2 3 4 5 6
@@ -324,7 +324,7 @@ test_expect_success 'GIT_SKIP_TESTS sh pattern' '
 
 test_expect_success 'GIT_SKIP_TESTS entire suite' '
 	(
-		run_sub_test_lib_test git-skip-tests-entire-suite \
+		write_and_run_sub_test_lib_test git-skip-tests-entire-suite \
 			"GIT_SKIP_TESTS entire suite" \
 			--skip="git" <<-\EOF &&
 		for i in 1 2 3
@@ -341,7 +341,7 @@ test_expect_success 'GIT_SKIP_TESTS entire suite' '
 
 test_expect_success 'GIT_SKIP_TESTS does not skip unmatched suite' '
 	(
-		run_sub_test_lib_test git-skip-tests-unmatched-suite \
+		write_and_run_sub_test_lib_test git-skip-tests-unmatched-suite \
 			"GIT_SKIP_TESTS does not skip unmatched suite" \
 			--skip="notgit" <<-\EOF &&
 		for i in 1 2 3
@@ -361,7 +361,7 @@ test_expect_success 'GIT_SKIP_TESTS does not skip unmatched suite' '
 '
 
 test_expect_success '--run basic' '
-	run_sub_test_lib_test run-basic \
+	write_and_run_sub_test_lib_test run-basic \
 		"--run basic" --run="1,3,5" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -382,7 +382,7 @@ test_expect_success '--run basic' '
 '
 
 test_expect_success '--run with a range' '
-	run_sub_test_lib_test run-range \
+	write_and_run_sub_test_lib_test run-range \
 		"--run with a range" --run="1-3" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -403,7 +403,7 @@ test_expect_success '--run with a range' '
 '
 
 test_expect_success '--run with two ranges' '
-	run_sub_test_lib_test run-two-ranges \
+	write_and_run_sub_test_lib_test run-two-ranges \
 		"--run with two ranges" --run="1-2,5-6" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -424,7 +424,7 @@ test_expect_success '--run with two ranges' '
 '
 
 test_expect_success '--run with a left open range' '
-	run_sub_test_lib_test run-left-open-range \
+	write_and_run_sub_test_lib_test run-left-open-range \
 		"--run with a left open range" --run="-3" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -445,7 +445,7 @@ test_expect_success '--run with a left open range' '
 '
 
 test_expect_success '--run with a right open range' '
-	run_sub_test_lib_test run-right-open-range \
+	write_and_run_sub_test_lib_test run-right-open-range \
 		"--run with a right open range" --run="4-" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -466,7 +466,7 @@ test_expect_success '--run with a right open range' '
 '
 
 test_expect_success '--run with basic negation' '
-	run_sub_test_lib_test run-basic-neg \
+	write_and_run_sub_test_lib_test run-basic-neg \
 		"--run with basic negation" --run="!3" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -487,7 +487,7 @@ test_expect_success '--run with basic negation' '
 '
 
 test_expect_success '--run with two negations' '
-	run_sub_test_lib_test run-two-neg \
+	write_and_run_sub_test_lib_test run-two-neg \
 		"--run with two negations" --run="!3,!6" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -508,7 +508,7 @@ test_expect_success '--run with two negations' '
 '
 
 test_expect_success '--run a range and negation' '
-	run_sub_test_lib_test run-range-and-neg \
+	write_and_run_sub_test_lib_test run-range-and-neg \
 		"--run a range and negation" --run="-4,!2" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -529,7 +529,7 @@ test_expect_success '--run a range and negation' '
 '
 
 test_expect_success '--run range negation' '
-	run_sub_test_lib_test run-range-neg \
+	write_and_run_sub_test_lib_test run-range-neg \
 		"--run range negation" --run="!1-3" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -550,7 +550,7 @@ test_expect_success '--run range negation' '
 '
 
 test_expect_success '--run include, exclude and include' '
-	run_sub_test_lib_test run-inc-neg-inc \
+	write_and_run_sub_test_lib_test run-inc-neg-inc \
 		"--run include, exclude and include" \
 		--run="1-5,!1-3,2" <<-\EOF &&
 	for i in 1 2 3 4 5 6
@@ -572,7 +572,7 @@ test_expect_success '--run include, exclude and include' '
 '
 
 test_expect_success '--run include, exclude and include, comma separated' '
-	run_sub_test_lib_test run-inc-neg-inc-comma \
+	write_and_run_sub_test_lib_test run-inc-neg-inc-comma \
 		"--run include, exclude and include, comma separated" \
 		--run=1-5,!1-3,2 <<-\EOF &&
 	for i in 1 2 3 4 5 6
@@ -594,7 +594,7 @@ test_expect_success '--run include, exclude and include, comma separated' '
 '
 
 test_expect_success '--run exclude and include' '
-	run_sub_test_lib_test run-neg-inc \
+	write_and_run_sub_test_lib_test run-neg-inc \
 		"--run exclude and include" \
 		--run="!3-,5" <<-\EOF &&
 	for i in 1 2 3 4 5 6
@@ -616,7 +616,7 @@ test_expect_success '--run exclude and include' '
 '
 
 test_expect_success '--run empty selectors' '
-	run_sub_test_lib_test run-empty-sel \
+	write_and_run_sub_test_lib_test run-empty-sel \
 		"--run empty selectors" \
 		--run="1,,3,,,5" <<-\EOF &&
 	for i in 1 2 3 4 5 6
@@ -638,7 +638,7 @@ test_expect_success '--run empty selectors' '
 '
 
 test_expect_success '--run substring selector' '
-	run_sub_test_lib_test run-substring-selector \
+	write_and_run_sub_test_lib_test run-substring-selector \
 		"--run empty selectors" \
 		--run="relevant" <<-\EOF &&
 	test_expect_success "relevant test" "true"
@@ -662,7 +662,7 @@ test_expect_success '--run substring selector' '
 '
 
 test_expect_success '--run keyword selection' '
-	run_sub_test_lib_test_err run-inv-range-start \
+	write_and_run_sub_test_lib_test_err run-inv-range-start \
 		"--run invalid range start" \
 		--run="a-5" <<-\EOF &&
 	test_expect_success "passing test #1" "true"
@@ -677,7 +677,7 @@ test_expect_success '--run keyword selection' '
 '
 
 test_expect_success '--run invalid range end' '
-	run_sub_test_lib_test_err run-inv-range-end \
+	write_and_run_sub_test_lib_test_err run-inv-range-end \
 		"--run invalid range end" \
 		--run="1-z" <<-\EOF &&
 	test_expect_success "passing test #1" "true"
@@ -692,7 +692,7 @@ test_expect_success '--run invalid range end' '
 '
 
 test_expect_success 'tests respect prerequisites' '
-	run_sub_test_lib_test prereqs "tests respect prereqs" <<-\EOF &&
+	write_and_run_sub_test_lib_test prereqs "tests respect prereqs" <<-\EOF &&
 
 	test_set_prereq HAVEIT
 	test_expect_success HAVEIT "prereq is satisfied" "true"
@@ -722,7 +722,7 @@ test_expect_success 'tests respect prerequisites' '
 '
 
 test_expect_success 'tests respect lazy prerequisites' '
-	run_sub_test_lib_test lazy-prereqs "respect lazy prereqs" <<-\EOF &&
+	write_and_run_sub_test_lib_test lazy-prereqs "respect lazy prereqs" <<-\EOF &&
 
 	test_lazy_prereq LAZY_TRUE true
 	test_expect_success LAZY_TRUE "lazy prereq is satisifed" "true"
@@ -746,7 +746,7 @@ test_expect_success 'tests respect lazy prerequisites' '
 '
 
 test_expect_success 'nested lazy prerequisites' '
-	run_sub_test_lib_test nested-lazy "nested lazy prereqs" <<-\EOF &&
+	write_and_run_sub_test_lib_test nested-lazy "nested lazy prereqs" <<-\EOF &&
 
 	test_lazy_prereq NESTED_INNER "
 		>inner &&
@@ -772,7 +772,7 @@ test_expect_success 'nested lazy prerequisites' '
 '
 
 test_expect_success 'lazy prereqs do not turn off tracing' '
-	run_sub_test_lib_test lazy-prereq-and-tracing \
+	write_and_run_sub_test_lib_test lazy-prereq-and-tracing \
 		"lazy prereqs and -x" -v -x <<-\EOF &&
 	test_lazy_prereq LAZY true
 
@@ -785,7 +785,7 @@ test_expect_success 'lazy prereqs do not turn off tracing' '
 '
 
 test_expect_success 'tests clean up after themselves' '
-	run_sub_test_lib_test cleanup "test with cleanup" <<-\EOF &&
+	write_and_run_sub_test_lib_test cleanup "test with cleanup" <<-\EOF &&
 	clean=no
 	test_expect_success "do cleanup" "
 		test_when_finished clean=yes
@@ -805,7 +805,7 @@ test_expect_success 'tests clean up after themselves' '
 '
 
 test_expect_success 'tests clean up even on failures' '
-	run_sub_test_lib_test_err \
+	write_and_run_sub_test_lib_test_err \
 		failing-cleanup "Failing tests with cleanup commands" <<-\EOF &&
 	test_expect_success "tests clean up even after a failure" "
 		touch clean-after-failure &&
@@ -834,7 +834,7 @@ test_expect_success 'tests clean up even on failures' '
 '
 
 test_expect_success 'test_atexit is run' '
-	run_sub_test_lib_test_err \
+	write_and_run_sub_test_lib_test_err \
 		atexit-cleanup "Run atexit commands" -i <<-\EOF &&
 	test_expect_success "tests clean up even after a failure" "
 		> ../../clean-atexit &&
-- 
2.32.0.955.ge7c5360f7e7


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [PATCH v2 3/8] test-lib tests: stop using a subshell in write_sub_test_lib_test()
  2021-07-21 22:57 ` [PATCH v2 " Ævar Arnfjörð Bjarmason
  2021-07-21 22:57   ` [PATCH v2 1/8] test-lib tests: move "run_sub_test" to a new lib-subtest.sh Ævar Arnfjörð Bjarmason
  2021-07-21 22:57   ` [PATCH v2 2/8] test-lib tests: split up "write and run" into two functions Ævar Arnfjörð Bjarmason
@ 2021-07-21 22:57   ` Ævar Arnfjörð Bjarmason
  2021-07-21 22:57   ` [PATCH v2 4/8] test-lib tests: don't provide a description for the sub-tests Ævar Arnfjörð Bjarmason
                     ` (5 subsequent siblings)
  8 siblings, 0 replies; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-21 22:57 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Felipe Contreras, Adam Spiers,
	Thomas Rast, Ilya Bobyr, Patrick Steinhardt,
	Ævar Arnfjörð Bjarmason

Now that this function doesn't handle running the test anymore we can
do away with the sub-shell, which was used to scope an "unset" and
"export" shell variables.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/lib-subtest.sh | 21 +++++++++------------
 1 file changed, 9 insertions(+), 12 deletions(-)

diff --git a/t/lib-subtest.sh b/t/lib-subtest.sh
index 8f3f6dddcf4..727b0850e9b 100644
--- a/t/lib-subtest.sh
+++ b/t/lib-subtest.sh
@@ -1,20 +1,17 @@
 write_sub_test_lib_test () {
 	name="$1" descr="$2" # stdin is the body of the test code
 	mkdir "$name" &&
-	(
-		cd "$name" &&
-		write_script "$name.sh" "$TEST_SHELL_PATH" <<-EOF &&
-		test_description='$descr (run in sub test-lib)
+	write_script "$name/$name.sh" "$TEST_SHELL_PATH" <<-EOF &&
+	test_description='$descr (run in sub test-lib)
 
-		This is run in a sub test-lib so that we do not get incorrect
-		passing metrics
-		'
+	This is run in a sub test-lib so that we do not get incorrect
+	passing metrics
+	'
 
-		# Point to the t/test-lib.sh, which isn't in ../ as usual
-		. "\$TEST_DIRECTORY"/test-lib.sh
-		EOF
-		cat >>"$name.sh"
-	)
+	# Point to the t/test-lib.sh, which isn't in ../ as usual
+	. "\$TEST_DIRECTORY"/test-lib.sh
+	EOF
+	cat >>"$name/$name.sh"
 }
 
 _run_sub_test_lib_test_common () {
-- 
2.32.0.955.ge7c5360f7e7


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [PATCH v2 4/8] test-lib tests: don't provide a description for the sub-tests
  2021-07-21 22:57 ` [PATCH v2 " Ævar Arnfjörð Bjarmason
                     ` (2 preceding siblings ...)
  2021-07-21 22:57   ` [PATCH v2 3/8] test-lib tests: stop using a subshell in write_sub_test_lib_test() Ævar Arnfjörð Bjarmason
@ 2021-07-21 22:57   ` Ævar Arnfjörð Bjarmason
  2021-07-21 22:57   ` [PATCH v2 5/8] test-lib tests: get rid of copy/pasted mock test code Ævar Arnfjörð Bjarmason
                     ` (4 subsequent siblings)
  8 siblings, 0 replies; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-21 22:57 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Felipe Contreras, Adam Spiers,
	Thomas Rast, Ilya Bobyr, Patrick Steinhardt,
	Ævar Arnfjörð Bjarmason

Change the $test_description provided for the generated subtests to be
constant, since the only purpose of having it is that test-lib.sh will
barf if it isn't supplied.

The other purpose of having it was to effectively split up the test
description between the argument to test_expect_success and the
argument to "write_and_run_sub_test_lib_test". Let's only use one of
the two.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/lib-subtest.sh |  12 ++--
 t/t0000-basic.sh | 142 +++++++++++++++++++++--------------------------
 2 files changed, 66 insertions(+), 88 deletions(-)

diff --git a/t/lib-subtest.sh b/t/lib-subtest.sh
index 727b0850e9b..7899518abb0 100644
--- a/t/lib-subtest.sh
+++ b/t/lib-subtest.sh
@@ -1,12 +1,8 @@
 write_sub_test_lib_test () {
-	name="$1" descr="$2" # stdin is the body of the test code
+	name="$1" # stdin is the body of the test code
 	mkdir "$name" &&
 	write_script "$name/$name.sh" "$TEST_SHELL_PATH" <<-EOF &&
-	test_description='$descr (run in sub test-lib)
-
-	This is run in a sub test-lib so that we do not get incorrect
-	passing metrics
-	'
+	test_description='A test of test-lib.sh itself'
 
 	# Point to the t/test-lib.sh, which isn't in ../ as usual
 	. "\$TEST_DIRECTORY"/test-lib.sh
@@ -15,8 +11,8 @@ write_sub_test_lib_test () {
 }
 
 _run_sub_test_lib_test_common () {
-	neg="$1" name="$2" descr="$3" # stdin is the body of the test code
-	shift 3
+	neg="$1" name="$2" # stdin is the body of the test code
+	shift 2
 
 	# intercept pseudo-options at the front of the argument list that we
 	# will not pass to child script
diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index a7c5aaacab6..6fdd5f43cae 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -67,8 +67,8 @@ test_expect_success 'success is reported like this' '
 	:
 '
 
-test_expect_success 'pretend we have a fully passing test suite' '
-	write_and_run_sub_test_lib_test full-pass "3 passing tests" <<-\EOF &&
+test_expect_success 'subtest: 3 passing tests' '
+	write_and_run_sub_test_lib_test full-pass <<-\EOF &&
 	for i in 1 2 3
 	do
 		test_expect_success "passing test #$i" "true"
@@ -84,9 +84,8 @@ test_expect_success 'pretend we have a fully passing test suite' '
 	EOF
 '
 
-test_expect_success 'pretend we have a partially passing test suite' '
-	write_and_run_sub_test_lib_test_err \
-		partial-pass "2/3 tests passing" <<-\EOF &&
+test_expect_success 'subtest: 2/3 tests passing' '
+	write_and_run_sub_test_lib_test_err partial-pass <<-\EOF &&
 	test_expect_success "passing test #1" "true"
 	test_expect_success "failing test #2" "false"
 	test_expect_success "passing test #3" "true"
@@ -102,8 +101,8 @@ test_expect_success 'pretend we have a partially passing test suite' '
 	EOF
 '
 
-test_expect_success 'pretend we have a known breakage' '
-	write_and_run_sub_test_lib_test failing-todo "A failing TODO test" <<-\EOF &&
+test_expect_success 'subtest: a failing TODO test' '
+	write_and_run_sub_test_lib_test failing-todo <<-\EOF &&
 	test_expect_success "passing test" "true"
 	test_expect_failure "pretend we have a known breakage" "false"
 	test_done
@@ -117,8 +116,8 @@ test_expect_success 'pretend we have a known breakage' '
 	EOF
 '
 
-test_expect_success 'pretend we have fixed a known breakage' '
-	write_and_run_sub_test_lib_test passing-todo "A passing TODO test" <<-\EOF &&
+test_expect_success 'subtest: a passing TODO test' '
+	write_and_run_sub_test_lib_test passing-todo <<-\EOF &&
 	test_expect_failure "pretend we have fixed a known breakage" "true"
 	test_done
 	EOF
@@ -129,9 +128,8 @@ test_expect_success 'pretend we have fixed a known breakage' '
 	EOF
 '
 
-test_expect_success 'pretend we have fixed one of two known breakages (run in sub test-lib)' '
-	write_and_run_sub_test_lib_test partially-passing-todos \
-		"2 TODO tests, one passing" <<-\EOF &&
+test_expect_success 'subtest: 2 TODO tests, one passin' '
+	write_and_run_sub_test_lib_test partially-passing-todos <<-\EOF &&
 	test_expect_failure "pretend we have a known breakage" "false"
 	test_expect_success "pretend we have a passing test" "true"
 	test_expect_failure "pretend we have fixed another known breakage" "true"
@@ -148,9 +146,8 @@ test_expect_success 'pretend we have fixed one of two known breakages (run in su
 	EOF
 '
 
-test_expect_success 'pretend we have a pass, fail, and known breakage' '
-	write_and_run_sub_test_lib_test_err \
-		mixed-results1 "mixed results #1" <<-\EOF &&
+test_expect_success 'subtest: mixed results: pass, failure and a TODO test' '
+	write_and_run_sub_test_lib_test_err mixed-results1 <<-\EOF &&
 	test_expect_success "passing test" "true"
 	test_expect_success "failing test" "false"
 	test_expect_failure "pretend we have a known breakage" "false"
@@ -167,9 +164,8 @@ test_expect_success 'pretend we have a pass, fail, and known breakage' '
 	EOF
 '
 
-test_expect_success 'pretend we have a mix of all possible results' '
-	write_and_run_sub_test_lib_test_err \
-		mixed-results2 "mixed results #2" <<-\EOF &&
+test_expect_success 'subtest: mixed results: a mixture of all possible results' '
+	write_and_run_sub_test_lib_test_err mixed-results2 <<-\EOF &&
 	test_expect_success "passing test" "true"
 	test_expect_success "passing test" "true"
 	test_expect_success "passing test" "true"
@@ -203,9 +199,8 @@ test_expect_success 'pretend we have a mix of all possible results' '
 	EOF
 '
 
-test_expect_success 'test --verbose' '
-	write_and_run_sub_test_lib_test_err \
-		t1234-verbose "test verbose" --verbose <<-\EOF &&
+test_expect_success 'subtest: --verbose option' '
+	write_and_run_sub_test_lib_test_err t1234-verbose --verbose <<-\EOF &&
 	test_expect_success "passing test" true
 	test_expect_success "test with output" "echo foo"
 	test_expect_success "failing test" false
@@ -230,9 +225,9 @@ test_expect_success 'test --verbose' '
 	EOF
 '
 
-test_expect_success 'test --verbose-only' '
+test_expect_success 'subtest: --verbose-only option' '
 	write_and_run_sub_test_lib_test_err \
-		t2345-verbose-only-2 "test verbose-only=2" \
+		t2345-verbose-only-2 \
 		--verbose-only=2 <<-\EOF &&
 	test_expect_success "passing test" true
 	test_expect_success "test with output" "echo foo"
@@ -253,10 +248,9 @@ test_expect_success 'test --verbose-only' '
 	EOF
 '
 
-test_expect_success 'GIT_SKIP_TESTS' '
+test_expect_success 'subtest: skip one with GIT_SKIP_TESTS' '
 	(
 		write_and_run_sub_test_lib_test git-skip-tests-basic \
-			"GIT_SKIP_TESTS" \
 			--skip="git.2" <<-\EOF &&
 		for i in 1 2 3
 		do
@@ -274,10 +268,9 @@ test_expect_success 'GIT_SKIP_TESTS' '
 	)
 '
 
-test_expect_success 'GIT_SKIP_TESTS several tests' '
+test_expect_success 'subtest: skip several with GIT_SKIP_TESTS' '
 	(
 		write_and_run_sub_test_lib_test git-skip-tests-several \
-			"GIT_SKIP_TESTS several tests" \
 			--skip="git.2 git.5" <<-\EOF &&
 		for i in 1 2 3 4 5 6
 		do
@@ -298,10 +291,9 @@ test_expect_success 'GIT_SKIP_TESTS several tests' '
 	)
 '
 
-test_expect_success 'GIT_SKIP_TESTS sh pattern' '
+test_expect_success 'subtest: sh pattern skipping with GIT_SKIP_TESTS' '
 	(
 		write_and_run_sub_test_lib_test git-skip-tests-sh-pattern \
-			"GIT_SKIP_TESTS sh pattern" \
 			--skip="git.[2-5]" <<-\EOF &&
 		for i in 1 2 3 4 5 6
 		do
@@ -322,10 +314,9 @@ test_expect_success 'GIT_SKIP_TESTS sh pattern' '
 	)
 '
 
-test_expect_success 'GIT_SKIP_TESTS entire suite' '
+test_expect_success 'subtest: skip entire test suite with GIT_SKIP_TESTS' '
 	(
 		write_and_run_sub_test_lib_test git-skip-tests-entire-suite \
-			"GIT_SKIP_TESTS entire suite" \
 			--skip="git" <<-\EOF &&
 		for i in 1 2 3
 		do
@@ -339,10 +330,9 @@ test_expect_success 'GIT_SKIP_TESTS entire suite' '
 	)
 '
 
-test_expect_success 'GIT_SKIP_TESTS does not skip unmatched suite' '
+test_expect_success 'subtest: GIT_SKIP_TESTS does not skip unmatched suite' '
 	(
 		write_and_run_sub_test_lib_test git-skip-tests-unmatched-suite \
-			"GIT_SKIP_TESTS does not skip unmatched suite" \
 			--skip="notgit" <<-\EOF &&
 		for i in 1 2 3
 		do
@@ -360,9 +350,8 @@ test_expect_success 'GIT_SKIP_TESTS does not skip unmatched suite' '
 	)
 '
 
-test_expect_success '--run basic' '
-	write_and_run_sub_test_lib_test run-basic \
-		"--run basic" --run="1,3,5" <<-\EOF &&
+test_expect_success 'subtest: --run basic' '
+	write_and_run_sub_test_lib_test run-basic --run="1,3,5" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
 		test_expect_success "passing test #$i" "true"
@@ -381,9 +370,9 @@ test_expect_success '--run basic' '
 	EOF
 '
 
-test_expect_success '--run with a range' '
+test_expect_success 'subtest: --run with a range' '
 	write_and_run_sub_test_lib_test run-range \
-		"--run with a range" --run="1-3" <<-\EOF &&
+		--run="1-3" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
 		test_expect_success "passing test #$i" "true"
@@ -402,9 +391,9 @@ test_expect_success '--run with a range' '
 	EOF
 '
 
-test_expect_success '--run with two ranges' '
+test_expect_success 'subtest: --run with two ranges' '
 	write_and_run_sub_test_lib_test run-two-ranges \
-		"--run with two ranges" --run="1-2,5-6" <<-\EOF &&
+		--run="1-2,5-6" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
 		test_expect_success "passing test #$i" "true"
@@ -423,9 +412,9 @@ test_expect_success '--run with two ranges' '
 	EOF
 '
 
-test_expect_success '--run with a left open range' '
+test_expect_success 'subtest: --run with a left open range' '
 	write_and_run_sub_test_lib_test run-left-open-range \
-		"--run with a left open range" --run="-3" <<-\EOF &&
+		--run="-3" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
 		test_expect_success "passing test #$i" "true"
@@ -444,9 +433,9 @@ test_expect_success '--run with a left open range' '
 	EOF
 '
 
-test_expect_success '--run with a right open range' '
+test_expect_success 'subtest: --run with a right open range' '
 	write_and_run_sub_test_lib_test run-right-open-range \
-		"--run with a right open range" --run="4-" <<-\EOF &&
+		--run="4-" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
 		test_expect_success "passing test #$i" "true"
@@ -465,9 +454,9 @@ test_expect_success '--run with a right open range' '
 	EOF
 '
 
-test_expect_success '--run with basic negation' '
+test_expect_success 'subtest: --run with basic negation' '
 	write_and_run_sub_test_lib_test run-basic-neg \
-		"--run with basic negation" --run="!3" <<-\EOF &&
+		--run="!3" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
 		test_expect_success "passing test #$i" "true"
@@ -486,9 +475,9 @@ test_expect_success '--run with basic negation' '
 	EOF
 '
 
-test_expect_success '--run with two negations' '
+test_expect_success 'subtest: --run with two negations' '
 	write_and_run_sub_test_lib_test run-two-neg \
-		"--run with two negations" --run="!3,!6" <<-\EOF &&
+		--run="!3,!6" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
 		test_expect_success "passing test #$i" "true"
@@ -507,9 +496,9 @@ test_expect_success '--run with two negations' '
 	EOF
 '
 
-test_expect_success '--run a range and negation' '
+test_expect_success 'subtest: --run a range and negation' '
 	write_and_run_sub_test_lib_test run-range-and-neg \
-		"--run a range and negation" --run="-4,!2" <<-\EOF &&
+		--run="-4,!2" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
 		test_expect_success "passing test #$i" "true"
@@ -528,9 +517,9 @@ test_expect_success '--run a range and negation' '
 	EOF
 '
 
-test_expect_success '--run range negation' '
+test_expect_success 'subtest: --run range negation' '
 	write_and_run_sub_test_lib_test run-range-neg \
-		"--run range negation" --run="!1-3" <<-\EOF &&
+		--run="!1-3" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
 		test_expect_success "passing test #$i" "true"
@@ -549,9 +538,8 @@ test_expect_success '--run range negation' '
 	EOF
 '
 
-test_expect_success '--run include, exclude and include' '
+test_expect_success 'subtest: --run include, exclude and include' '
 	write_and_run_sub_test_lib_test run-inc-neg-inc \
-		"--run include, exclude and include" \
 		--run="1-5,!1-3,2" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -571,9 +559,8 @@ test_expect_success '--run include, exclude and include' '
 	EOF
 '
 
-test_expect_success '--run include, exclude and include, comma separated' '
+test_expect_success 'subtest: --run include, exclude and include, comma separated' '
 	write_and_run_sub_test_lib_test run-inc-neg-inc-comma \
-		"--run include, exclude and include, comma separated" \
 		--run=1-5,!1-3,2 <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -593,9 +580,8 @@ test_expect_success '--run include, exclude and include, comma separated' '
 	EOF
 '
 
-test_expect_success '--run exclude and include' '
+test_expect_success 'subtest: --run exclude and include' '
 	write_and_run_sub_test_lib_test run-neg-inc \
-		"--run exclude and include" \
 		--run="!3-,5" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -615,9 +601,8 @@ test_expect_success '--run exclude and include' '
 	EOF
 '
 
-test_expect_success '--run empty selectors' '
+test_expect_success 'subtest: --run empty selectors' '
 	write_and_run_sub_test_lib_test run-empty-sel \
-		"--run empty selectors" \
 		--run="1,,3,,,5" <<-\EOF &&
 	for i in 1 2 3 4 5 6
 	do
@@ -637,9 +622,8 @@ test_expect_success '--run empty selectors' '
 	EOF
 '
 
-test_expect_success '--run substring selector' '
+test_expect_success 'subtest: --run substring selector' '
 	write_and_run_sub_test_lib_test run-substring-selector \
-		"--run empty selectors" \
 		--run="relevant" <<-\EOF &&
 	test_expect_success "relevant test" "true"
 	for i in 1 2 3 4 5 6
@@ -661,9 +645,8 @@ test_expect_success '--run substring selector' '
 	EOF
 '
 
-test_expect_success '--run keyword selection' '
+test_expect_success 'subtest: --run keyword selection' '
 	write_and_run_sub_test_lib_test_err run-inv-range-start \
-		"--run invalid range start" \
 		--run="a-5" <<-\EOF &&
 	test_expect_success "passing test #1" "true"
 	test_done
@@ -676,9 +659,8 @@ test_expect_success '--run keyword selection' '
 	EOF_ERR
 '
 
-test_expect_success '--run invalid range end' '
+test_expect_success 'subtest: --run invalid range end' '
 	write_and_run_sub_test_lib_test_err run-inv-range-end \
-		"--run invalid range end" \
 		--run="1-z" <<-\EOF &&
 	test_expect_success "passing test #1" "true"
 	test_done
@@ -691,8 +673,8 @@ test_expect_success '--run invalid range end' '
 	EOF_ERR
 '
 
-test_expect_success 'tests respect prerequisites' '
-	write_and_run_sub_test_lib_test prereqs "tests respect prereqs" <<-\EOF &&
+test_expect_success 'subtest: tests respect prerequisites' '
+	write_and_run_sub_test_lib_test prereqs <<-\EOF &&
 
 	test_set_prereq HAVEIT
 	test_expect_success HAVEIT "prereq is satisfied" "true"
@@ -721,8 +703,8 @@ test_expect_success 'tests respect prerequisites' '
 	EOF
 '
 
-test_expect_success 'tests respect lazy prerequisites' '
-	write_and_run_sub_test_lib_test lazy-prereqs "respect lazy prereqs" <<-\EOF &&
+test_expect_success 'subtest: tests respect lazy prerequisites' '
+	write_and_run_sub_test_lib_test lazy-prereqs <<-\EOF &&
 
 	test_lazy_prereq LAZY_TRUE true
 	test_expect_success LAZY_TRUE "lazy prereq is satisifed" "true"
@@ -745,8 +727,8 @@ test_expect_success 'tests respect lazy prerequisites' '
 	EOF
 '
 
-test_expect_success 'nested lazy prerequisites' '
-	write_and_run_sub_test_lib_test nested-lazy "nested lazy prereqs" <<-\EOF &&
+test_expect_success 'subtest: nested lazy prerequisites' '
+	write_and_run_sub_test_lib_test nested-lazy <<-\EOF &&
 
 	test_lazy_prereq NESTED_INNER "
 		>inner &&
@@ -771,9 +753,9 @@ test_expect_success 'nested lazy prerequisites' '
 	EOF
 '
 
-test_expect_success 'lazy prereqs do not turn off tracing' '
+test_expect_success 'subtest: lazy prereqs do not turn off tracing' '
 	write_and_run_sub_test_lib_test lazy-prereq-and-tracing \
-		"lazy prereqs and -x" -v -x <<-\EOF &&
+		-v -x <<-\EOF &&
 	test_lazy_prereq LAZY true
 
 	test_expect_success lazy "test_have_prereq LAZY && echo trace"
@@ -784,8 +766,8 @@ test_expect_success 'lazy prereqs do not turn off tracing' '
 	grep "echo trace" lazy-prereq-and-tracing/err
 '
 
-test_expect_success 'tests clean up after themselves' '
-	write_and_run_sub_test_lib_test cleanup "test with cleanup" <<-\EOF &&
+test_expect_success 'subtest: tests clean up after themselves' '
+	write_and_run_sub_test_lib_test cleanup <<-\EOF &&
 	clean=no
 	test_expect_success "do cleanup" "
 		test_when_finished clean=yes
@@ -804,9 +786,9 @@ test_expect_success 'tests clean up after themselves' '
 	EOF
 '
 
-test_expect_success 'tests clean up even on failures' '
+test_expect_success 'subtest: tests clean up even on failures' '
 	write_and_run_sub_test_lib_test_err \
-		failing-cleanup "Failing tests with cleanup commands" <<-\EOF &&
+		failing-cleanup <<-\EOF &&
 	test_expect_success "tests clean up even after a failure" "
 		touch clean-after-failure &&
 		test_when_finished rm clean-after-failure &&
@@ -833,9 +815,9 @@ test_expect_success 'tests clean up even on failures' '
 	EOF
 '
 
-test_expect_success 'test_atexit is run' '
+test_expect_success 'subtest: test_atexit is run' '
 	write_and_run_sub_test_lib_test_err \
-		atexit-cleanup "Run atexit commands" -i <<-\EOF &&
+		atexit-cleanup -i <<-\EOF &&
 	test_expect_success "tests clean up even after a failure" "
 		> ../../clean-atexit &&
 		test_atexit rm ../../clean-atexit &&
-- 
2.32.0.955.ge7c5360f7e7


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [PATCH v2 5/8] test-lib tests: get rid of copy/pasted mock test code
  2021-07-21 22:57 ` [PATCH v2 " Ævar Arnfjörð Bjarmason
                     ` (3 preceding siblings ...)
  2021-07-21 22:57   ` [PATCH v2 4/8] test-lib tests: don't provide a description for the sub-tests Ævar Arnfjörð Bjarmason
@ 2021-07-21 22:57   ` Ævar Arnfjörð Bjarmason
  2021-07-21 22:57   ` [PATCH v2 6/8] test-lib tests: avoid subshell for "test_cmp" for readability Ævar Arnfjörð Bjarmason
                     ` (3 subsequent siblings)
  8 siblings, 0 replies; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-21 22:57 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Felipe Contreras, Adam Spiers,
	Thomas Rast, Ilya Bobyr, Patrick Steinhardt,
	Ævar Arnfjörð Bjarmason

Now that we've split up the write_sub_test_lib_test*() and
run_sub_test_lib_test*() functions let's fix those tests in
t0000-basic.sh that were verbosely copy/pasting earlier tests.

I'm (ab)using writing a tag object under a ref-name that's
content-addressable from the content of the test script. If we can
update-ref that it's unique, if not we've got a duplicate. The tag
object stores the name of the earlier test for reporting the error.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/lib-subtest.sh |  34 ++++++-
 t/t0000-basic.sh | 228 +++++++++++++----------------------------------
 2 files changed, 93 insertions(+), 169 deletions(-)

diff --git a/t/lib-subtest.sh b/t/lib-subtest.sh
index 7899518abb0..d98390606b8 100644
--- a/t/lib-subtest.sh
+++ b/t/lib-subtest.sh
@@ -1,3 +1,34 @@
+_assert_unique_sub_test () {
+	name=$1 &&
+
+	# Alert about the copy/paste programming
+	hash=$(git hash-object -w "$name") &&
+	cat >tag.sig <<-EOF &&
+	object $hash
+	type blob
+	tag $hash
+	tagger . <> 0 +0000
+
+	duplicate script detected!
+
+	This test script was already written as:
+
+	$name
+
+	You can just re-use its test code with your own
+	run_sub_test_lib_test*()
+	EOF
+
+	tag=$(git mktag <tag.sig) &&
+	if ! git update-ref refs/tags/blob-$hash $tag $(test_oid zero) 2>/dev/null
+	then
+		msg=$(git for-each-ref refs/tags/blob-$hash \
+			--format='%(contents)' refs/tags/blob-$hash)
+		error "on write of $name: $msg"
+		return 1
+	fi
+}
+
 write_sub_test_lib_test () {
 	name="$1" # stdin is the body of the test code
 	mkdir "$name" &&
@@ -7,7 +38,8 @@ write_sub_test_lib_test () {
 	# Point to the t/test-lib.sh, which isn't in ../ as usual
 	. "\$TEST_DIRECTORY"/test-lib.sh
 	EOF
-	cat >>"$name/$name.sh"
+	cat >>"$name/$name.sh" &&
+	_assert_unique_sub_test "$name/$name.sh"
 }
 
 _run_sub_test_lib_test_common () {
diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index 6fdd5f43cae..a0b99d83349 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -226,18 +226,13 @@ test_expect_success 'subtest: --verbose option' '
 '
 
 test_expect_success 'subtest: --verbose-only option' '
-	write_and_run_sub_test_lib_test_err \
-		t2345-verbose-only-2 \
-		--verbose-only=2 <<-\EOF &&
-	test_expect_success "passing test" true
-	test_expect_success "test with output" "echo foo"
-	test_expect_success "failing test" false
-	test_done
-	EOF
-	check_sub_test_lib_test t2345-verbose-only-2 <<-\EOF
+	run_sub_test_lib_test_err \
+		t1234-verbose \
+		--verbose-only=2 &&
+	check_sub_test_lib_test t1234-verbose <<-\EOF
 	> ok 1 - passing test
 	> Z
-	> expecting success of 2345.2 '\''test with output'\'': echo foo
+	> expecting success of 1234.2 '\''test with output'\'': echo foo
 	> foo
 	> ok 2 - test with output
 	> Z
@@ -250,15 +245,9 @@ test_expect_success 'subtest: --verbose-only option' '
 
 test_expect_success 'subtest: skip one with GIT_SKIP_TESTS' '
 	(
-		write_and_run_sub_test_lib_test git-skip-tests-basic \
-			--skip="git.2" <<-\EOF &&
-		for i in 1 2 3
-		do
-			test_expect_success "passing test #$i" "true"
-		done
-		test_done
-		EOF
-		check_sub_test_lib_test git-skip-tests-basic <<-\EOF
+		run_sub_test_lib_test full-pass \
+			--skip="full.2" &&
+		check_sub_test_lib_test full-pass <<-\EOF
 		> ok 1 - passing test #1
 		> ok 2 # skip passing test #2 (GIT_SKIP_TESTS)
 		> ok 3 - passing test #3
@@ -293,15 +282,9 @@ test_expect_success 'subtest: skip several with GIT_SKIP_TESTS' '
 
 test_expect_success 'subtest: sh pattern skipping with GIT_SKIP_TESTS' '
 	(
-		write_and_run_sub_test_lib_test git-skip-tests-sh-pattern \
-			--skip="git.[2-5]" <<-\EOF &&
-		for i in 1 2 3 4 5 6
-		do
-			test_expect_success "passing test #$i" "true"
-		done
-		test_done
-		EOF
-		check_sub_test_lib_test git-skip-tests-sh-pattern <<-\EOF
+		run_sub_test_lib_test git-skip-tests-several \
+			--skip="git.[2-5]" &&
+		check_sub_test_lib_test git-skip-tests-several <<-\EOF
 		> ok 1 - passing test #1
 		> ok 2 # skip passing test #2 (GIT_SKIP_TESTS)
 		> ok 3 # skip passing test #3 (GIT_SKIP_TESTS)
@@ -316,15 +299,10 @@ test_expect_success 'subtest: sh pattern skipping with GIT_SKIP_TESTS' '
 
 test_expect_success 'subtest: skip entire test suite with GIT_SKIP_TESTS' '
 	(
-		write_and_run_sub_test_lib_test git-skip-tests-entire-suite \
-			--skip="git" <<-\EOF &&
-		for i in 1 2 3
-		do
-			test_expect_success "passing test #$i" "true"
-		done
-		test_done
-		EOF
-		check_sub_test_lib_test git-skip-tests-entire-suite <<-\EOF
+		GIT_SKIP_TESTS="git" && export GIT_SKIP_TESTS &&
+		run_sub_test_lib_test git-skip-tests-several \
+			--skip="git" &&
+		check_sub_test_lib_test git-skip-tests-several <<-\EOF
 		> 1..0 # SKIP skip all tests in git
 		EOF
 	)
@@ -332,15 +310,10 @@ test_expect_success 'subtest: skip entire test suite with GIT_SKIP_TESTS' '
 
 test_expect_success 'subtest: GIT_SKIP_TESTS does not skip unmatched suite' '
 	(
-		write_and_run_sub_test_lib_test git-skip-tests-unmatched-suite \
-			--skip="notgit" <<-\EOF &&
-		for i in 1 2 3
-		do
-			test_expect_success "passing test #$i" "true"
-		done
-		test_done
-		EOF
-		check_sub_test_lib_test git-skip-tests-unmatched-suite <<-\EOF
+		GIT_SKIP_TESTS="notgit" && export GIT_SKIP_TESTS &&
+		run_sub_test_lib_test full-pass \
+			--skip="notfull" &&
+		check_sub_test_lib_test full-pass <<-\EOF
 		> ok 1 - passing test #1
 		> ok 2 - passing test #2
 		> ok 3 - passing test #3
@@ -351,14 +324,8 @@ test_expect_success 'subtest: GIT_SKIP_TESTS does not skip unmatched suite' '
 '
 
 test_expect_success 'subtest: --run basic' '
-	write_and_run_sub_test_lib_test run-basic --run="1,3,5" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-basic <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several --run="1,3,5" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 - passing test #1
 	> ok 2 # skip passing test #2 (--run)
 	> ok 3 - passing test #3
@@ -371,15 +338,9 @@ test_expect_success 'subtest: --run basic' '
 '
 
 test_expect_success 'subtest: --run with a range' '
-	write_and_run_sub_test_lib_test run-range \
-		--run="1-3" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-range <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="1-3" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 - passing test #1
 	> ok 2 - passing test #2
 	> ok 3 - passing test #3
@@ -392,15 +353,9 @@ test_expect_success 'subtest: --run with a range' '
 '
 
 test_expect_success 'subtest: --run with two ranges' '
-	write_and_run_sub_test_lib_test run-two-ranges \
-		--run="1-2,5-6" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-two-ranges <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="1-2,5-6" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 - passing test #1
 	> ok 2 - passing test #2
 	> ok 3 # skip passing test #3 (--run)
@@ -413,15 +368,9 @@ test_expect_success 'subtest: --run with two ranges' '
 '
 
 test_expect_success 'subtest: --run with a left open range' '
-	write_and_run_sub_test_lib_test run-left-open-range \
-		--run="-3" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-left-open-range <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="-3" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 - passing test #1
 	> ok 2 - passing test #2
 	> ok 3 - passing test #3
@@ -434,15 +383,9 @@ test_expect_success 'subtest: --run with a left open range' '
 '
 
 test_expect_success 'subtest: --run with a right open range' '
-	write_and_run_sub_test_lib_test run-right-open-range \
-		--run="4-" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-right-open-range <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="4-" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 # skip passing test #1 (--run)
 	> ok 2 # skip passing test #2 (--run)
 	> ok 3 # skip passing test #3 (--run)
@@ -455,15 +398,9 @@ test_expect_success 'subtest: --run with a right open range' '
 '
 
 test_expect_success 'subtest: --run with basic negation' '
-	write_and_run_sub_test_lib_test run-basic-neg \
-		--run="!3" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-basic-neg <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="!3" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 - passing test #1
 	> ok 2 - passing test #2
 	> ok 3 # skip passing test #3 (--run)
@@ -476,15 +413,9 @@ test_expect_success 'subtest: --run with basic negation' '
 '
 
 test_expect_success 'subtest: --run with two negations' '
-	write_and_run_sub_test_lib_test run-two-neg \
-		--run="!3,!6" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-two-neg <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="!3,!6" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 - passing test #1
 	> ok 2 - passing test #2
 	> ok 3 # skip passing test #3 (--run)
@@ -497,15 +428,9 @@ test_expect_success 'subtest: --run with two negations' '
 '
 
 test_expect_success 'subtest: --run a range and negation' '
-	write_and_run_sub_test_lib_test run-range-and-neg \
-		--run="-4,!2" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-range-and-neg <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="-4,!2" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 - passing test #1
 	> ok 2 # skip passing test #2 (--run)
 	> ok 3 - passing test #3
@@ -518,15 +443,9 @@ test_expect_success 'subtest: --run a range and negation' '
 '
 
 test_expect_success 'subtest: --run range negation' '
-	write_and_run_sub_test_lib_test run-range-neg \
-		--run="!1-3" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-range-neg <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="!1-3" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 # skip passing test #1 (--run)
 	> ok 2 # skip passing test #2 (--run)
 	> ok 3 # skip passing test #3 (--run)
@@ -539,15 +458,9 @@ test_expect_success 'subtest: --run range negation' '
 '
 
 test_expect_success 'subtest: --run include, exclude and include' '
-	write_and_run_sub_test_lib_test run-inc-neg-inc \
-		--run="1-5,!1-3,2" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-inc-neg-inc <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="1-5,!1-3,2" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 # skip passing test #1 (--run)
 	> ok 2 - passing test #2
 	> ok 3 # skip passing test #3 (--run)
@@ -560,15 +473,9 @@ test_expect_success 'subtest: --run include, exclude and include' '
 '
 
 test_expect_success 'subtest: --run include, exclude and include, comma separated' '
-	write_and_run_sub_test_lib_test run-inc-neg-inc-comma \
-		--run=1-5,!1-3,2 <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-inc-neg-inc-comma <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run=1-5,!1-3,2 &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 # skip passing test #1 (--run)
 	> ok 2 - passing test #2
 	> ok 3 # skip passing test #3 (--run)
@@ -581,15 +488,9 @@ test_expect_success 'subtest: --run include, exclude and include, comma separate
 '
 
 test_expect_success 'subtest: --run exclude and include' '
-	write_and_run_sub_test_lib_test run-neg-inc \
-		--run="!3-,5" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-neg-inc <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="!3-,5" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 - passing test #1
 	> ok 2 - passing test #2
 	> ok 3 # skip passing test #3 (--run)
@@ -602,15 +503,9 @@ test_expect_success 'subtest: --run exclude and include' '
 '
 
 test_expect_success 'subtest: --run empty selectors' '
-	write_and_run_sub_test_lib_test run-empty-sel \
-		--run="1,,3,,,5" <<-\EOF &&
-	for i in 1 2 3 4 5 6
-	do
-		test_expect_success "passing test #$i" "true"
-	done
-	test_done
-	EOF
-	check_sub_test_lib_test run-empty-sel <<-\EOF
+	run_sub_test_lib_test git-skip-tests-several \
+		--run="1,,3,,,5" &&
+	check_sub_test_lib_test git-skip-tests-several <<-\EOF
 	> ok 1 - passing test #1
 	> ok 2 # skip passing test #2 (--run)
 	> ok 3 - passing test #3
@@ -660,12 +555,9 @@ test_expect_success 'subtest: --run keyword selection' '
 '
 
 test_expect_success 'subtest: --run invalid range end' '
-	write_and_run_sub_test_lib_test_err run-inv-range-end \
-		--run="1-z" <<-\EOF &&
-	test_expect_success "passing test #1" "true"
-	test_done
-	EOF
-	check_sub_test_lib_test_err run-inv-range-end \
+	run_sub_test_lib_test_err run-inv-range-start \
+		--run="1-z" &&
+	check_sub_test_lib_test_err run-inv-range-start \
 		<<-\EOF_OUT 3<<-EOF_ERR
 	> FATAL: Unexpected exit with code 1
 	EOF_OUT
-- 
2.32.0.955.ge7c5360f7e7


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [PATCH v2 6/8] test-lib tests: avoid subshell for "test_cmp" for readability
  2021-07-21 22:57 ` [PATCH v2 " Ævar Arnfjörð Bjarmason
                     ` (4 preceding siblings ...)
  2021-07-21 22:57   ` [PATCH v2 5/8] test-lib tests: get rid of copy/pasted mock test code Ævar Arnfjörð Bjarmason
@ 2021-07-21 22:57   ` Ævar Arnfjörð Bjarmason
  2021-07-21 22:57   ` [PATCH v2 7/8] test-lib tests: refactor common part of check_sub_test_lib_test*() Ævar Arnfjörð Bjarmason
                     ` (2 subsequent siblings)
  8 siblings, 0 replies; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-21 22:57 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Felipe Contreras, Adam Spiers,
	Thomas Rast, Ilya Bobyr, Patrick Steinhardt,
	Ævar Arnfjörð Bjarmason

The use of a sub-shell for running the test_cmp of stdout/stderr for
the test author was introduced in this form in 565b6fa87bb (tests:
refactor mechanics of testing in a sub test-lib, 2012-12-16), but from
looking at the history that seemed to have diligently copied my
original ad-hoc implementation in 7b905119703 (t/t0000-basic.sh: Run
the passing TODO test inside its own test-lib, 2010-08-19).

There's no reason to use a subshell here, we try to avoid it in
general. It also improves readability, if the test fails we print out
the relative path in the trash directory that needs to be looked
at.

Before that was mostly obscured, since the "write_sub_test_lib_test"
will pick the directory for you from the test name.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/lib-subtest.sh | 20 +++++++-------------
 1 file changed, 7 insertions(+), 13 deletions(-)

diff --git a/t/lib-subtest.sh b/t/lib-subtest.sh
index d98390606b8..96a4e6e259a 100644
--- a/t/lib-subtest.sh
+++ b/t/lib-subtest.sh
@@ -111,23 +111,17 @@ run_sub_test_lib_test_err () {
 
 check_sub_test_lib_test () {
 	name="$1" # stdin is the expected output from the test
-	(
-		cd "$name" &&
-		test_must_be_empty err &&
-		sed -e 's/^> //' -e 's/Z$//' >expect &&
-		test_cmp expect out
-	)
+	test_must_be_empty "$name"/err &&
+	sed -e 's/^> //' -e 's/Z$//' >"$name"/expect &&
+	test_cmp "$name/"expect "$name"/out
 }
 
 check_sub_test_lib_test_err () {
 	name="$1" # stdin is the expected output from the test
 	# expected error output is in descriptor 3
-	(
-		cd "$name" &&
-		sed -e 's/^> //' -e 's/Z$//' >expect.out &&
-		test_cmp expect.out out &&
-		sed -e 's/^> //' -e 's/Z$//' <&3 >expect.err &&
-		test_cmp expect.err err
-	)
+	sed -e 's/^> //' -e 's/Z$//' >"$name"/expect.out &&
+	test_cmp "$name"/expect.out "$name"/out &&
+	sed -e 's/^> //' -e 's/Z$//' <&3 >"$name"/expect.err &&
+	test_cmp "$name"/expect.err "$name"/err
 }
 
-- 
2.32.0.955.ge7c5360f7e7


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [PATCH v2 7/8] test-lib tests: refactor common part of check_sub_test_lib_test*()
  2021-07-21 22:57 ` [PATCH v2 " Ævar Arnfjörð Bjarmason
                     ` (5 preceding siblings ...)
  2021-07-21 22:57   ` [PATCH v2 6/8] test-lib tests: avoid subshell for "test_cmp" for readability Ævar Arnfjörð Bjarmason
@ 2021-07-21 22:57   ` Ævar Arnfjörð Bjarmason
  2021-07-21 22:57   ` [PATCH v2 8/8] test-lib tests: assert 1 exit code, not non-zero Ævar Arnfjörð Bjarmason
  2021-07-21 23:23   ` [PATCH v2 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh Junio C Hamano
  8 siblings, 0 replies; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-21 22:57 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Felipe Contreras, Adam Spiers,
	Thomas Rast, Ilya Bobyr, Patrick Steinhardt,
	Ævar Arnfjörð Bjarmason

Refactor the two check_sub_test_lib_test*() functions to avoid
duplicating the same comparison they did of stdout. This duplication
was initially added when check_sub_test_lib_test_err() was added in
0445e6f0a12 (test-lib: '--run' to run only specific tests,
2014-04-30).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/lib-subtest.sh | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/t/lib-subtest.sh b/t/lib-subtest.sh
index 96a4e6e259a..2fde75e8863 100644
--- a/t/lib-subtest.sh
+++ b/t/lib-subtest.sh
@@ -109,18 +109,22 @@ run_sub_test_lib_test_err () {
 	_run_sub_test_lib_test_common '!' "$@"
 }
 
+_check_sub_test_lib_test_common () {
+	name="$1" &&
+	sed -e 's/^> //' -e 's/Z$//' >"$name"/expect.out &&
+	test_cmp "$name"/expect.out "$name"/out
+}
+
 check_sub_test_lib_test () {
 	name="$1" # stdin is the expected output from the test
-	test_must_be_empty "$name"/err &&
-	sed -e 's/^> //' -e 's/Z$//' >"$name"/expect &&
-	test_cmp "$name/"expect "$name"/out
+	_check_sub_test_lib_test_common "$name" &&
+	test_must_be_empty "$name"/err
 }
 
 check_sub_test_lib_test_err () {
 	name="$1" # stdin is the expected output from the test
+	_check_sub_test_lib_test_common "$name" &&
 	# expected error output is in descriptor 3
-	sed -e 's/^> //' -e 's/Z$//' >"$name"/expect.out &&
-	test_cmp "$name"/expect.out "$name"/out &&
 	sed -e 's/^> //' -e 's/Z$//' <&3 >"$name"/expect.err &&
 	test_cmp "$name"/expect.err "$name"/err
 }
-- 
2.32.0.955.ge7c5360f7e7


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [PATCH v2 8/8] test-lib tests: assert 1 exit code, not non-zero
  2021-07-21 22:57 ` [PATCH v2 " Ævar Arnfjörð Bjarmason
                     ` (6 preceding siblings ...)
  2021-07-21 22:57   ` [PATCH v2 7/8] test-lib tests: refactor common part of check_sub_test_lib_test*() Ævar Arnfjörð Bjarmason
@ 2021-07-21 22:57   ` Ævar Arnfjörð Bjarmason
  2021-07-21 23:23   ` [PATCH v2 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh Junio C Hamano
  8 siblings, 0 replies; 24+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-21 22:57 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Felipe Contreras, Adam Spiers,
	Thomas Rast, Ilya Bobyr, Patrick Steinhardt,
	Ævar Arnfjörð Bjarmason

Improve the testing for test-lib.sh itself to assert that we have a
exit code of 1, not any non-zero. Improves code added in
0445e6f0a12 (test-lib: '--run' to run only specific tests,
2014-04-30).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/lib-subtest.sh | 21 +++++++++------------
 1 file changed, 9 insertions(+), 12 deletions(-)

diff --git a/t/lib-subtest.sh b/t/lib-subtest.sh
index 2fde75e8863..f7ad7bc7d7d 100644
--- a/t/lib-subtest.sh
+++ b/t/lib-subtest.sh
@@ -43,8 +43,8 @@ write_sub_test_lib_test () {
 }
 
 _run_sub_test_lib_test_common () {
-	neg="$1" name="$2" # stdin is the body of the test code
-	shift 2
+	cmp_op="$1" want_code="$2" name="$3" # stdin is the body of the test code
+	shift 3
 
 	# intercept pseudo-options at the front of the argument list that we
 	# will not pass to child script
@@ -80,33 +80,30 @@ _run_sub_test_lib_test_common () {
 		GIT_SKIP_TESTS=$skip &&
 		export GIT_SKIP_TESTS &&
 		sane_unset GIT_TEST_FAIL_PREREQS &&
-		if test -z "$neg"
-		then
-			./"$name.sh" "$@" >out 2>err
-		else
-			! ./"$name.sh" "$@" >out 2>err
-		fi
+		./"$name.sh" "$@" >out 2>err;
+		ret=$? &&
+		test "$ret" "$cmp_op" "$want_code"
 	)
 }
 
 write_and_run_sub_test_lib_test () {
 	name="$1" descr="$2" # stdin is the body of the test code
 	write_sub_test_lib_test "$@" || return 1
-	_run_sub_test_lib_test_common '' "$@"
+	_run_sub_test_lib_test_common -eq 0 "$@"
 }
 
 write_and_run_sub_test_lib_test_err () {
 	name="$1" descr="$2" # stdin is the body of the test code
 	write_sub_test_lib_test "$@" || return 1
-	_run_sub_test_lib_test_common '!' "$@"
+	_run_sub_test_lib_test_common -eq 1 "$@"
 }
 
 run_sub_test_lib_test () {
-	_run_sub_test_lib_test_common '' "$@"
+	_run_sub_test_lib_test_common -eq 0 "$@"
 }
 
 run_sub_test_lib_test_err () {
-	_run_sub_test_lib_test_common '!' "$@"
+	_run_sub_test_lib_test_common -eq 1 "$@"
 }
 
 _check_sub_test_lib_test_common () {
-- 
2.32.0.955.ge7c5360f7e7


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH v2 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh
  2021-07-21 22:57 ` [PATCH v2 " Ævar Arnfjörð Bjarmason
                     ` (7 preceding siblings ...)
  2021-07-21 22:57   ` [PATCH v2 8/8] test-lib tests: assert 1 exit code, not non-zero Ævar Arnfjörð Bjarmason
@ 2021-07-21 23:23   ` Junio C Hamano
  8 siblings, 0 replies; 24+ messages in thread
From: Junio C Hamano @ 2021-07-21 23:23 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Jeff King, Felipe Contreras, Adam Spiers, Thomas Rast,
	Ilya Bobyr, Patrick Steinhardt

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> The v1[1] of this recently started conflicting with the cooking
> jk/t0000-subtests-fix and ps/t0000-output-directory-fix, and got
> ejected from "seen". This v2 is rebased on top of those two, so it
> merges cleanly with "seen".

Thanks for noticing and proactively updating before I got around to
asking you ;-)

^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2021-07-21 23:25 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-14 10:48 [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh Ævar Arnfjörð Bjarmason
2021-06-14 10:48 ` [PATCH 1/8] test-lib tests: move "run_sub_test" to a new lib-subtest.sh Ævar Arnfjörð Bjarmason
2021-06-14 10:48 ` [PATCH 2/8] test-lib tests: split up "write and run" into two functions Ævar Arnfjörð Bjarmason
2021-06-14 10:48 ` [PATCH 3/8] test-lib tests: stop using a subshell in write_sub_test_lib_test() Ævar Arnfjörð Bjarmason
2021-06-14 10:48 ` [PATCH 4/8] test-lib tests: don't provide a description for the sub-tests Ævar Arnfjörð Bjarmason
2021-06-14 10:48 ` [PATCH 5/8] test-lib tests: get rid of copy/pasted mock test code Ævar Arnfjörð Bjarmason
2021-06-14 10:48 ` [PATCH 6/8] test-lib tests: avoid subshell for "test_cmp" for readability Ævar Arnfjörð Bjarmason
2021-06-14 10:48 ` [PATCH 7/8] test-lib tests: refactor common part of check_sub_test_lib_test*() Ævar Arnfjörð Bjarmason
2021-06-14 10:48 ` [PATCH 8/8] test-lib tests: assert 1 exit code, not non-zero Ævar Arnfjörð Bjarmason
2021-06-15  2:24 ` [PATCH 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh Junio C Hamano
2021-06-24 10:38   ` Ævar Arnfjörð Bjarmason
2021-06-30  7:04     ` Ævar Arnfjörð Bjarmason
2021-07-01 14:30       ` Junio C Hamano
2021-06-15 18:05 ` Felipe Contreras
2021-07-21 22:57 ` [PATCH v2 " Ævar Arnfjörð Bjarmason
2021-07-21 22:57   ` [PATCH v2 1/8] test-lib tests: move "run_sub_test" to a new lib-subtest.sh Ævar Arnfjörð Bjarmason
2021-07-21 22:57   ` [PATCH v2 2/8] test-lib tests: split up "write and run" into two functions Ævar Arnfjörð Bjarmason
2021-07-21 22:57   ` [PATCH v2 3/8] test-lib tests: stop using a subshell in write_sub_test_lib_test() Ævar Arnfjörð Bjarmason
2021-07-21 22:57   ` [PATCH v2 4/8] test-lib tests: don't provide a description for the sub-tests Ævar Arnfjörð Bjarmason
2021-07-21 22:57   ` [PATCH v2 5/8] test-lib tests: get rid of copy/pasted mock test code Ævar Arnfjörð Bjarmason
2021-07-21 22:57   ` [PATCH v2 6/8] test-lib tests: avoid subshell for "test_cmp" for readability Ævar Arnfjörð Bjarmason
2021-07-21 22:57   ` [PATCH v2 7/8] test-lib tests: refactor common part of check_sub_test_lib_test*() Ævar Arnfjörð Bjarmason
2021-07-21 22:57   ` [PATCH v2 8/8] test-lib tests: assert 1 exit code, not non-zero Ævar Arnfjörð Bjarmason
2021-07-21 23:23   ` [PATCH v2 0/8] test-lib tests: split off subtest code in t0000 into lib-subtest.sh Junio C Hamano

git@vger.kernel.org list mirror (unofficial, one of many)

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 git git/ https://public-inbox.org/git \
		git@vger.kernel.org
	public-inbox-index git

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://7fh6tueqddpjyxjmgtdiueylzoqt6pt7hec3pukyptlmohoowvhde4yd.onion/inbox.comp.version-control.git
	nntp://ie5yzdi7fg72h7s4sdcztq5evakq23rdt33mfyfcddc5u3ndnw24ogqd.onion/inbox.comp.version-control.git
	nntp://4uok3hntl7oi7b4uf4rtfwefqeexfzil2w6kgk2jn5z2f764irre7byd.onion/inbox.comp.version-control.git
	nntp://news.gmane.io/gmane.comp.version-control.git
 note: .onion URLs require Tor: https://www.torproject.org/

code repositories for project(s) associated with this inbox:

	https://80x24.org/mirrors/git.git

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git