From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>, "Jeff King" <peff@peff.net>,
"Derrick Stolee" <derrickstolee@github.com>,
"Taylor Blau" <me@ttaylorr.com>,
"SZEDER Gábor" <szeder.dev@gmail.com>,
"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: [PATCH v2 0/2] test-lib.sh: have all tests pass under "-x", remove BASH_XTRACEFD
Date: Wed, 1 Dec 2021 21:11:40 +0100 [thread overview]
Message-ID: <cover-v2-0.2-00000000000-20211201T200801Z-avarab@gmail.com> (raw)
In-Reply-To: <patch-1.1-9f735bd0d49-20211129T200950Z-avarab@gmail.com>
I think this v2 should nicely address the outstanding comments on
v1. I.e. v1 made the trade-off of not caring about -x in t1510 for
some of its assertions.
But as noted in the new 1/2 here we can get a much better and easily
debuggable test regardless of which shell we use if we don't use the
pattern that's being worked around by test_untraceable.
Which, is the updated 2/2 notes is mostly what SZEDER was doing in
2018 when "test_untraceable" was introduced, it's just that t1510
wasn't migrated over at the time, as some other tests in the his
series discussed in 2/2 were.
Ævar Arnfjörð Bjarmason (2):
t1510: remove need for "test_untraceable", retain coverage
test-lib.sh: remove the now-unused "test_untraceable" facility
t/README | 3 --
t/t1510-repo-setup.sh | 83 +++++++++++++++++++++----------------------
t/test-lib.sh | 66 ++++------------------------------
3 files changed, 47 insertions(+), 105 deletions(-)
Range-diff against v1:
-: ----------- > 1: 91402624777 t1510: remove need for "test_untraceable", retain coverage
1: 9f735bd0d49 ! 2: 867d18d14bd test-lib.sh: have all tests pass under "-x", remove BASH_XTRACEFD
@@ Metadata
Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
## Commit message ##
- test-lib.sh: have all tests pass under "-x", remove BASH_XTRACEFD
+ test-lib.sh: remove the now-unused "test_untraceable" facility
- Change the "t1510-repo-setup.sh" test to use a new
- "test_must_be_empty_trace" wrapper, instead of turning on
- "test_untraceable=UnfortunatelyYes".
+ In the preceding commit the use of "test_untraceable=UnfortunatelyYes"
+ was removed from "t1510-repo-setup.sh" in favor of more narrow
+ redirections of the output of specific commands (and not entire
+ sub-shells or functions).
- The only reason the test was incompatible with "-x" was because of
- these "test_must_be_empty" checks, which we can instead instead skip
- if we're running under "set -x".
+ This is in line with the fixes in the series that introduced the
+ "test_untraceable" facility. See 571e472dc43 (Merge branch
+ 'sg/test-x', 2018-03-14) for the series as a whole, and
+ e.g. 91538d0cde9 (t5570-git-daemon: don't check the stderr of a
+ subshell, 2018-02-24) for a commit that's in line with the changes in
+ the preceding commit.
- Skipping the tests is preferable to not having the "-x" output at all,
- as it's much easier to debug the test. The result loss of test
- coverage is minimal. If someone is adjusting a "test_must_be_empty"
- test a failure might go away under "-x", but the new "say" we emit
- here should highlight that appropriately.
+ We've thus solved the TODO item noted when "test_untraceable" was
+ added to "t1510-repo-setup.sh" in 58275069288 (t1510-repo-setup: mark
+ as untraceable with '-x', 2018-02-24).
- Since the only user of "test_untraceable" is gone, we can remove that,
- not only isn't it used now, but I think the rationale for using it in
- the future no longer applies.
+ So let's remove the feature entirely. Not only is it currently unused,
+ but it actively encourages an anti-pattern in our tests. We should be
+ testing the output of specific commands, not entire subshells or
+ functions.
- We'll be better off by using a simple wrapper like the new
- "test_must_be_empty_trace". See 58275069288 (t1510-repo-setup: mark as
- untraceable with '-x', 2018-02-24) and 5fc98e79fc0 (t: add means to
- disable '-x' tracing for individual test scripts, 2018-02-24) for the
- addition of "test_untraceable".
-
- Once that's been removed we can dig deeper and see that we only have
- "BASH_XTRACEFD" due to an earlier attempt to work around the same
- issue. See d88785e424a (test-lib: set BASH_XTRACEFD automatically,
- 2016-05-11) and the 90c8a1db9d6 (test-lib: silence "-x" cleanup under
- bash, 2017-12-08) follow-up.
-
- I.e. we had redirection in "test_eval_" to get more relevant trace
- output under bash, which in turn was only needed because
- BASH_XTRACEFD=1 was set, which in turn was trying to work around test
- failures under "set -x".
-
- It's better if our test suite works the same way on all shells, rather
- than getting a passing run under "bash", only to have it fail with
- "-x" under say "dash". As the deleted code shows this is much simpler
- to implement across our supported POSIX shells.
+ That the "-x" output had to be disabled as a result is only one
+ symptom, but even under bash those tests will be harder to debug as
+ the subsequent check of the redirected file will be far removed from
+ the command that emitted the output.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
@@ t/README: appropriately before running "make". Short options can be bundled, i.e
-d::
--debug::
- ## t/t1510-repo-setup.sh ##
-@@ t/t1510-repo-setup.sh: A few rules for repo setup:
- prefix is NULL.
- "
-
--# This test heavily relies on the standard error of nested function calls.
--test_untraceable=UnfortunatelyYes
-+test_must_be_empty_trace () {
-+ if want_trace
-+ then
-+ say "$TEST_NAME does not check test_must_be_empty on \"$@\" under -x"
-+ return 0
-+ fi
-+ test_must_be_empty "$@"
-+}
-
- TEST_PASSES_SANITIZE_LEAK=true
- . ./test-lib.sh
-@@ t/t1510-repo-setup.sh: test_expect_success '#0: nonbare repo, no explicit configuration' '
- try_repo 0 unset unset unset "" unset \
- .git "$here/0" "$here/0" "(null)" \
- .git "$here/0" "$here/0" sub/ 2>message &&
-- test_must_be_empty message
-+ test_must_be_empty_trace message
- '
-
- test_expect_success '#1: GIT_WORK_TREE without explicit GIT_DIR is accepted' '
- try_repo 1 "$here" unset unset "" unset \
- "$here/1/.git" "$here" "$here" 1/ \
- "$here/1/.git" "$here" "$here" 1/sub/ 2>message &&
-- test_must_be_empty message
-+ test_must_be_empty_trace message
- '
-
- test_expect_success '#2: worktree defaults to cwd with explicit GIT_DIR' '
-@@ t/t1510-repo-setup.sh: test_expect_success '#4: core.worktree without GIT_DIR set is accepted' '
- try_case 4 unset unset \
- .git "$here/4/sub" "$here/4" "(null)" \
- "$here/4/.git" "$here/4/sub" "$here/4/sub" "(null)" 2>message &&
-- test_must_be_empty message
-+ test_must_be_empty_trace message
- '
-
- test_expect_success '#5: core.worktree + GIT_WORK_TREE is accepted' '
-@@ t/t1510-repo-setup.sh: test_expect_success '#5: core.worktree + GIT_WORK_TREE is accepted' '
- try_repo 5a .. unset "$here/5a" "" unset \
- "$here/5a/.git" "$here" "$here" 5a/ \
- "$here/5a/.git" "$here/5a" "$here/5a" sub/ &&
-- test_must_be_empty message
-+ test_must_be_empty_trace message
- '
-
- test_expect_success '#6: setting GIT_DIR brings core.worktree to life' '
-@@ t/t1510-repo-setup.sh: test_expect_success '#9: GIT_WORK_TREE accepted with gitfile' '
- try_repo 9 wt unset unset gitfile unset \
- "$here/9.git" "$here/9/wt" "$here/9" "(null)" \
- "$here/9.git" "$here/9/sub/wt" "$here/9/sub" "(null)" 2>message &&
-- test_must_be_empty message
-+ test_must_be_empty_trace message
- '
-
- test_expect_success '#10: GIT_DIR can point to gitfile' '
-@@ t/t1510-repo-setup.sh: test_expect_success '#12: core.worktree with gitfile is accepted' '
- try_repo 12 unset unset "$here/12" gitfile unset \
- "$here/12.git" "$here/12" "$here/12" "(null)" \
- "$here/12.git" "$here/12" "$here/12" sub/ 2>message &&
-- test_must_be_empty message
-+ test_must_be_empty_trace message
- '
-
- test_expect_success '#13: core.worktree+GIT_WORK_TREE accepted (with gitfile)' '
-@@ t/t1510-repo-setup.sh: test_expect_success '#13: core.worktree+GIT_WORK_TREE accepted (with gitfile)' '
- try_repo 13 non-existent-too unset non-existent gitfile unset \
- "$here/13.git" "$here/13/non-existent-too" "$here/13" "(null)" \
- "$here/13.git" "$here/13/sub/non-existent-too" "$here/13/sub" "(null)" 2>message &&
-- test_must_be_empty message
-+ test_must_be_empty_trace message
- '
-
- # case #14.
-@@ t/t1510-repo-setup.sh: test_expect_success '#17: GIT_WORK_TREE without explicit GIT_DIR is accepted (ba
- try_repo 17c "$here/17c" unset unset "" true \
- .git "$here/17c" "$here/17c" "(null)" \
- "$here/17c/.git" "$here/17c" "$here/17c" sub/ 2>message &&
-- test_must_be_empty message
-+ test_must_be_empty_trace message
- '
-
- test_expect_success '#18: bare .git named by GIT_DIR has no worktree' '
-@@ t/t1510-repo-setup.sh: test_expect_success '#20a: core.worktree without GIT_DIR accepted (inside .git)'
- "$here/20a/.git" "$here/20a" "$here/20a" .git/wt/ &&
- try_case 20a/.git/wt/sub unset unset \
- "$here/20a/.git" "$here/20a" "$here/20a" .git/wt/sub/ &&
-- test_must_be_empty message
-+ test_must_be_empty_trace message
- '
-
- test_expect_success '#20b/c: core.worktree and core.bare conflict' '
-@@ t/t1510-repo-setup.sh: test_expect_success '#21: setup, core.worktree warns before overriding core.bare
- export GIT_WORK_TREE &&
- git status >/dev/null
- ) 2>message &&
-- test_must_be_empty message
-+ test_must_be_empty_trace message
-
- '
- run_wt_tests 21
-@@ t/t1510-repo-setup.sh: test_expect_success '#25: GIT_WORK_TREE accepted if GIT_DIR unset (bare gitfile
- try_repo 25 "$here/25" unset unset gitfile true \
- "$here/25.git" "$here/25" "$here/25" "(null)" \
- "$here/25.git" "$here/25" "$here/25" "sub/" 2>message &&
-- test_must_be_empty message
-+ test_must_be_empty_trace message
- '
-
- test_expect_success '#26: bare repo has no worktree (GIT_DIR -> gitfile case)' '
-@@ t/t1510-repo-setup.sh: test_expect_success '#29: setup' '
- export GIT_WORK_TREE &&
- git status
- ) 2>message &&
-- test_must_be_empty message
-+ test_must_be_empty_trace message
- '
- run_wt_tests 29 gitfile
-
-
## t/test-lib.sh ##
@@ t/test-lib.sh: then
exit
--
2.34.1.876.gdb91009a90c
next prev parent reply other threads:[~2021-12-01 20:12 UTC|newest]
Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-11-29 13:47 [PATCH 0/2] Set GIT_TRACE2_EVENT_NESTING in test-lib.sh Derrick Stolee via GitGitGadget
2021-11-29 13:47 ` [PATCH 1/2] test-lib.sh: set GIT_TRACE2_EVENT_NESTING Derrick Stolee via GitGitGadget
2021-11-29 14:12 ` Ævar Arnfjörð Bjarmason
2021-11-29 18:28 ` Junio C Hamano
2021-11-29 18:32 ` Junio C Hamano
2021-11-29 13:47 ` [PATCH 2/2] t/t*: remove custom GIT_TRACE2_EVENT_NESTING Derrick Stolee via GitGitGadget
2021-11-29 18:40 ` [PATCH 0/2] Set GIT_TRACE2_EVENT_NESTING in test-lib.sh Jeff King
2021-11-29 18:44 ` Jeff King
2021-11-30 0:04 ` Taylor Blau
2021-11-30 15:34 ` Derrick Stolee
2021-11-30 22:43 ` Jeff Hostetler
2021-12-01 19:46 ` Jeff King
2021-11-29 20:13 ` [PATCH] test-lib.sh: have all tests pass under "-x", remove BASH_XTRACEFD Ævar Arnfjörð Bjarmason
2021-11-29 23:23 ` Eric Sunshine
2021-11-30 21:08 ` Jeff King
2021-11-30 21:50 ` Ævar Arnfjörð Bjarmason
2021-11-30 22:44 ` SZEDER Gábor
2021-12-01 14:06 ` Ævar Arnfjörð Bjarmason
2021-12-01 19:38 ` Jeff King
2021-12-01 18:38 ` Junio C Hamano
2021-12-01 20:11 ` Ævar Arnfjörð Bjarmason [this message]
2021-12-01 20:11 ` [PATCH v2 1/2] t1510: remove need for "test_untraceable", retain coverage Ævar Arnfjörð Bjarmason
2021-12-02 19:16 ` SZEDER Gábor
2021-12-02 19:28 ` SZEDER Gábor
2021-12-10 9:47 ` Jeff King
2021-12-10 10:08 ` Ævar Arnfjörð Bjarmason
2022-02-06 21:40 ` SZEDER Gábor
2021-12-01 20:11 ` [PATCH v2 2/2] test-lib.sh: remove the now-unused "test_untraceable" facility Ævar Arnfjörð Bjarmason
2021-12-10 10:07 ` [PATCH v3 0/2] test-lib.sh: have all tests pass under "-x", remove BASH_XTRACEFD Ævar Arnfjörð Bjarmason
2021-12-10 10:07 ` [PATCH v3 1/2] t1510: remove need for "test_untraceable", retain coverage Ævar Arnfjörð Bjarmason
2021-12-10 10:07 ` [PATCH v3 2/2] test-lib.sh: remove the now-unused "test_untraceable" facility Ævar Arnfjörð Bjarmason
2021-12-12 16:32 ` SZEDER Gábor
2021-12-12 17:06 ` Ævar Arnfjörð Bjarmason
2021-12-12 20:14 ` SZEDER Gábor
2021-12-13 18:51 ` Junio C Hamano
2021-12-14 16:43 ` Jeff King
2021-12-15 17:05 ` Junio C Hamano
2021-12-15 17:17 ` Jeff King
2021-12-15 17:32 ` Junio C Hamano
2021-12-16 13:04 ` Ævar Arnfjörð Bjarmason
2021-12-13 1:38 ` [PATCH v4 0/3] test-lib.sh: have all tests pass under "-x", remove BASH_XTRACEFD Ævar Arnfjörð Bjarmason
2021-12-13 1:38 ` [PATCH v4 1/3] t1510: remove need for "test_untraceable", retain coverage Ævar Arnfjörð Bjarmason
2021-12-13 1:38 ` [PATCH v4 2/3] test-lib.sh: remove the now-unused "test_untraceable" facility Ævar Arnfjörð Bjarmason
2021-12-13 1:38 ` [PATCH v4 3/3] test-lib.sh: remove "BASH_XTRACEFD" Ævar Arnfjörð Bjarmason
2022-02-21 23:11 ` SZEDER Gábor
2022-02-22 15:14 ` Ævar Arnfjörð Bjarmason
2021-12-13 5:43 ` [PATCH v4 0/3] test-lib.sh: have all tests pass under "-x", remove BASH_XTRACEFD SZEDER Gábor
2022-02-21 19:52 ` Ævar Arnfjörð Bjarmason
2022-02-21 21:03 ` SZEDER Gábor
2022-02-21 22:41 ` Ævar Arnfjörð Bjarmason
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover-v2-0.2-00000000000-20211201T200801Z-avarab@gmail.com \
--to=avarab@gmail.com \
--cc=derrickstolee@github.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=me@ttaylorr.com \
--cc=peff@peff.net \
--cc=szeder.dev@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).