From: Josh Steadmon <steadmon@google.com>
To: git@vger.kernel.org
Cc: johannes.schindelin@gmx.de, peff@peff.net, phillip.wood@dunelm.org.uk
Subject: [RFC PATCH 1/4] t0080: turn t-basic unit test into a helper
Date: Tue, 16 Jan 2024 14:22:58 -0800 [thread overview]
Message-ID: <a9f67ed703c8314f0f0507ffb83b503717b846b3.1705443632.git.steadmon@google.com> (raw)
In-Reply-To: <cover.1705443632.git.steadmon@google.com>
While t/unit-tests/t-basic.c uses the unit-test framework added in
e137fe3b29 (unit tests: add TAP unit test framework, 2023-11-09), it is
not a true unit test in that it intentionally fails in order to exercise
various codepaths in the unit-test framework. Thus, we intentionally
exclude it when running unit tests through the various t/Makefile
targets. Instead, it is executed by t0080-unit-test-output.sh, which
verifies its output follows the TAP format expected for the various
pass, skip, or fail cases.
As such, it makes more sense for t-basic to be a helper item for
t0080-unit-test-output.sh, so let's move it to t/t0080/t-basic.c and
adjust Makefiles and .gitignores as necessary.
This has the additional benefit that test harnesses seeking to run all
unit tests can find them with a simple glob of "t/unit-tests/bin/t-*",
with no exceptions needed. This will be important in a later patch where
we add support for running the unit tests via a test-tool subcommand.
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
Makefile | 17 +++++++++++++----
t/Makefile | 2 +-
t/t0080-unit-test-output.sh | 24 ++++++++++++------------
t/t0080/.gitignore | 1 +
t/{unit-tests => t0080}/t-basic.c | 2 +-
5 files changed, 28 insertions(+), 18 deletions(-)
create mode 100644 t/t0080/.gitignore
rename t/{unit-tests => t0080}/t-basic.c (98%)
diff --git a/Makefile b/Makefile
index 88ba7a3c51..ab32ec1101 100644
--- a/Makefile
+++ b/Makefile
@@ -683,6 +683,7 @@ TEST_OBJS =
TEST_PROGRAMS_NEED_X =
THIRD_PARTY_SOURCES =
UNIT_TEST_PROGRAMS =
+UNIT_TEST_HELPERS =
UNIT_TEST_DIR = t/unit-tests
UNIT_TEST_BIN = $(UNIT_TEST_DIR)/bin
@@ -1339,10 +1340,12 @@ THIRD_PARTY_SOURCES += compat/regex/%
THIRD_PARTY_SOURCES += sha1collisiondetection/%
THIRD_PARTY_SOURCES += sha1dc/%
-UNIT_TEST_PROGRAMS += t-basic
UNIT_TEST_PROGRAMS += t-strbuf
UNIT_TEST_PROGS = $(patsubst %,$(UNIT_TEST_BIN)/%$X,$(UNIT_TEST_PROGRAMS))
+UNIT_TEST_HELPERS += t/t0080/t-basic
+UNIT_TEST_HELPER_PROGS = $(patsubst %,%$X,$(UNIT_TEST_HELPERS))
UNIT_TEST_OBJS = $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(UNIT_TEST_PROGRAMS))
+UNIT_TEST_OBJS += $(patsubst %,%.o,$(UNIT_TEST_HELPERS))
UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/test-lib.o
# xdiff and reftable libs may in turn depend on what is in libgit.a
@@ -3189,7 +3192,9 @@ endif
test_bindir_programs := $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))
-all:: $(TEST_PROGRAMS) $(test_bindir_programs) $(UNIT_TEST_PROGS)
+all:: $(TEST_PROGRAMS) $(test_bindir_programs)
+
+all:: $(UNIT_TEST_PROGS) $(UNIT_TEST_HELPER_PROGS)
bin-wrappers/%: wrap-for-bin.sh
$(call mkdir_p_parent_template)
@@ -3620,7 +3625,7 @@ endif
artifacts-tar:: $(ALL_COMMANDS_TO_INSTALL) $(SCRIPT_LIB) $(OTHER_PROGRAMS) \
GIT-BUILD-OPTIONS $(TEST_PROGRAMS) $(test_bindir_programs) \
- $(UNIT_TEST_PROGS) $(MOFILES)
+ $(UNIT_TEST_PROGS) $(UNIT_TEST_HELPER_PROGS) $(MOFILES)
$(QUIET_SUBDIR0)templates $(QUIET_SUBDIR1) \
SHELL_PATH='$(SHELL_PATH_SQ)' PERL_PATH='$(PERL_PATH_SQ)'
test -n "$(ARTIFACTS_DIRECTORY)"
@@ -3682,7 +3687,7 @@ clean: profile-clean coverage-clean cocciclean
$(RM) headless-git.o
$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB)
$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS)
- $(RM) $(TEST_PROGRAMS) $(UNIT_TEST_PROGS)
+ $(RM) $(TEST_PROGRAMS) $(UNIT_TEST_PROGS) $(UNIT_TEST_HELPER_PROGS)
$(RM) $(FUZZ_PROGRAMS)
$(RM) $(SP_OBJ)
$(RM) $(HCC)
@@ -3869,6 +3874,10 @@ $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o $(UNIT_TEST_DIR)/
$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \
$(filter %.o,$^) $(filter %.a,$^) $(LIBS)
+$(UNIT_TEST_HELPER_PROGS): %$X: %.o $(UNIT_TEST_DIR)/test-lib.o $(GITLIBS) GIT-LDFLAGS
+ $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \
+ $(filter %.o,$^) $(filter %.a,$^) $(LIBS)
+
.PHONY: build-unit-tests unit-tests
build-unit-tests: $(UNIT_TEST_PROGS)
unit-tests: $(UNIT_TEST_PROGS)
diff --git a/t/Makefile b/t/Makefile
index b7a6fefe28..0bee7bc6ea 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -42,7 +42,7 @@ TPERF = $(sort $(wildcard perf/p[0-9][0-9][0-9][0-9]-*.sh))
TINTEROP = $(sort $(wildcard interop/i[0-9][0-9][0-9][0-9]-*.sh))
CHAINLINTTESTS = $(sort $(patsubst chainlint/%.test,%,$(wildcard chainlint/*.test)))
CHAINLINT = '$(PERL_PATH_SQ)' chainlint.pl
-UNIT_TESTS = $(sort $(filter-out %.pdb unit-tests/bin/t-basic%,$(wildcard unit-tests/bin/t-*)))
+UNIT_TESTS = $(sort $(filter-out %.pdb,$(wildcard unit-tests/bin/t-*)))
# `test-chainlint` (which is a dependency of `test-lint`, `test` and `prove`)
# checks all tests in all scripts via a single invocation, so tell individual
diff --git a/t/t0080-unit-test-output.sh b/t/t0080-unit-test-output.sh
index 961b54b06c..7431023d97 100755
--- a/t/t0080-unit-test-output.sh
+++ b/t/t0080-unit-test-output.sh
@@ -8,50 +8,50 @@ test_expect_success 'TAP output from unit tests' '
cat >expect <<-EOF &&
ok 1 - passing test
ok 2 - passing test and assertion return 1
- # check "1 == 2" failed at t/unit-tests/t-basic.c:76
+ # check "1 == 2" failed at t/t0080/t-basic.c:76
# left: 1
# right: 2
not ok 3 - failing test
ok 4 - failing test and assertion return 0
not ok 5 - passing TEST_TODO() # TODO
ok 6 - passing TEST_TODO() returns 1
- # todo check ${SQ}check(x)${SQ} succeeded at t/unit-tests/t-basic.c:25
+ # todo check ${SQ}check(x)${SQ} succeeded at t/t0080/t-basic.c:25
not ok 7 - failing TEST_TODO()
ok 8 - failing TEST_TODO() returns 0
- # check "0" failed at t/unit-tests/t-basic.c:30
+ # check "0" failed at t/t0080/t-basic.c:30
# skipping test - missing prerequisite
- # skipping check ${SQ}1${SQ} at t/unit-tests/t-basic.c:32
+ # skipping check ${SQ}1${SQ} at t/t0080/t-basic.c:32
ok 9 - test_skip() # SKIP
ok 10 - skipped test returns 1
# skipping test - missing prerequisite
ok 11 - test_skip() inside TEST_TODO() # SKIP
ok 12 - test_skip() inside TEST_TODO() returns 1
- # check "0" failed at t/unit-tests/t-basic.c:48
+ # check "0" failed at t/t0080/t-basic.c:48
not ok 13 - TEST_TODO() after failing check
ok 14 - TEST_TODO() after failing check returns 0
- # check "0" failed at t/unit-tests/t-basic.c:56
+ # check "0" failed at t/t0080/t-basic.c:56
not ok 15 - failing check after TEST_TODO()
ok 16 - failing check after TEST_TODO() returns 0
- # check "!strcmp("\thello\\\\", "there\"\n")" failed at t/unit-tests/t-basic.c:61
+ # check "!strcmp("\thello\\\\", "there\"\n")" failed at t/t0080/t-basic.c:61
# left: "\011hello\\\\"
# right: "there\"\012"
- # check "!strcmp("NULL", NULL)" failed at t/unit-tests/t-basic.c:62
+ # check "!strcmp("NULL", NULL)" failed at t/t0080/t-basic.c:62
# left: "NULL"
# right: NULL
- # check "${SQ}a${SQ} == ${SQ}\n${SQ}" failed at t/unit-tests/t-basic.c:63
+ # check "${SQ}a${SQ} == ${SQ}\n${SQ}" failed at t/t0080/t-basic.c:63
# left: ${SQ}a${SQ}
# right: ${SQ}\012${SQ}
- # check "${SQ}\\\\${SQ} == ${SQ}\\${SQ}${SQ}" failed at t/unit-tests/t-basic.c:64
+ # check "${SQ}\\\\${SQ} == ${SQ}\\${SQ}${SQ}" failed at t/t0080/t-basic.c:64
# left: ${SQ}\\\\${SQ}
# right: ${SQ}\\${SQ}${SQ}
not ok 17 - messages from failing string and char comparison
- # BUG: test has no checks at t/unit-tests/t-basic.c:91
+ # BUG: test has no checks at t/t0080/t-basic.c:91
not ok 18 - test with no checks
ok 19 - test with no checks returns 0
1..19
EOF
- ! "$GIT_BUILD_DIR"/t/unit-tests/bin/t-basic >actual &&
+ ! "$GIT_BUILD_DIR"/t/t0080/t-basic >actual &&
test_cmp expect actual
'
diff --git a/t/t0080/.gitignore b/t/t0080/.gitignore
new file mode 100644
index 0000000000..1903542827
--- /dev/null
+++ b/t/t0080/.gitignore
@@ -0,0 +1 @@
+/t-basic
diff --git a/t/unit-tests/t-basic.c b/t/t0080/t-basic.c
similarity index 98%
rename from t/unit-tests/t-basic.c
rename to t/t0080/t-basic.c
index fda1ae59a6..83727221b1 100644
--- a/t/unit-tests/t-basic.c
+++ b/t/t0080/t-basic.c
@@ -1,4 +1,4 @@
-#include "test-lib.h"
+#include "../unit-tests/test-lib.h"
/*
* The purpose of this "unit test" is to verify a few invariants of the unit
--
2.43.0.381.gb435a96ce8-goog
next prev parent reply other threads:[~2024-01-16 22:42 UTC|newest]
Thread overview: 99+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-01-16 22:22 [RFC PATCH 0/4] test-tool: add unit test suite runner Josh Steadmon
2024-01-16 22:22 ` Josh Steadmon [this message]
2024-01-16 22:54 ` [RFC PATCH 1/4] t0080: turn t-basic unit test into a helper Junio C Hamano
2024-01-23 0:43 ` Jeff King
2024-02-01 19:40 ` Josh Steadmon
2024-01-16 22:22 ` [RFC PATCH 2/4] test-tool run-command testsuite: support unit tests Josh Steadmon
2024-01-16 23:18 ` Junio C Hamano
2024-01-16 23:40 ` Junio C Hamano
2024-01-23 0:59 ` Jeff King
2024-02-01 22:06 ` Josh Steadmon
2024-02-01 22:26 ` Junio C Hamano
2024-02-01 23:08 ` Josh Steadmon
2024-02-01 23:14 ` Josh Steadmon
2024-01-16 22:23 ` [RFC PATCH 3/4] unit tests: add rule for running with test-tool Josh Steadmon
2024-01-16 22:23 ` [RFC PATCH 4/4] t/Makefile: run unit tests alongside shell tests Josh Steadmon
2024-01-16 23:24 ` [RFC PATCH 0/4] test-tool: add unit test suite runner Junio C Hamano
2024-02-03 0:50 ` [RFC PATCH v2 0/6] " Josh Steadmon
2024-02-03 0:50 ` [RFC PATCH v2 1/6] t0080: turn t-basic unit test into a helper Josh Steadmon
2024-02-07 20:55 ` Junio C Hamano
2024-02-12 20:42 ` Josh Steadmon
2024-02-07 22:58 ` Jeff King
2024-02-08 0:09 ` Junio C Hamano
2024-02-12 20:53 ` Josh Steadmon
2024-02-12 21:27 ` Junio C Hamano
2024-02-13 7:41 ` Jeff King
2024-02-13 14:36 ` rsbecker
2024-02-22 23:57 ` Josh Steadmon
2024-02-23 0:06 ` rsbecker
2024-02-13 17:28 ` Junio C Hamano
2024-02-22 23:55 ` Josh Steadmon
2024-02-03 0:50 ` [RFC PATCH v2 2/6] test-tool run-command testsuite: get shell from env Josh Steadmon
2024-02-07 20:55 ` Junio C Hamano
2024-02-12 21:35 ` Josh Steadmon
2024-02-03 0:50 ` [RFC PATCH v2 3/6] test-tool run-command testsuite: remove hardcoded filter Josh Steadmon
2024-02-07 20:55 ` Junio C Hamano
2024-02-12 22:48 ` Josh Steadmon
2024-02-12 22:51 ` Junio C Hamano
2024-02-03 0:50 ` [RFC PATCH v2 4/6] test-tool run-command testsuite: support unit tests Josh Steadmon
2024-02-05 16:16 ` phillip.wood123
2024-02-12 21:15 ` Josh Steadmon
2024-02-07 20:48 ` Junio C Hamano
2024-02-23 22:45 ` Josh Steadmon
2024-02-03 0:50 ` [RFC PATCH v2 5/6] unit tests: add rule for running with test-tool Josh Steadmon
2024-02-07 20:50 ` Junio C Hamano
2024-02-03 0:50 ` [RFC PATCH v2 6/6] t/Makefile: run unit tests alongside shell tests Josh Steadmon
2024-02-07 20:55 ` Junio C Hamano
2024-02-07 22:43 ` Jeff King
2024-02-07 23:26 ` Junio C Hamano
2024-02-03 18:52 ` [RFC PATCH v2 0/6] test-tool: add unit test suite runner Junio C Hamano
2024-02-12 22:50 ` Josh Steadmon
2024-02-07 21:14 ` Junio C Hamano
2024-02-23 23:33 ` [PATCH v3 0/7] " Josh Steadmon
2024-02-23 23:33 ` [PATCH v3 1/7] t0080: turn t-basic unit test into a helper Josh Steadmon
2024-02-23 23:33 ` [PATCH v3 2/7] test-tool run-command testsuite: get shell from env Josh Steadmon
2024-02-23 23:33 ` [PATCH v3 3/7] test-tool run-command testsuite: remove hardcoded filter Josh Steadmon
2024-02-23 23:33 ` [PATCH v3 4/7] test-tool run-command testsuite: support unit tests Josh Steadmon
2024-02-23 23:33 ` [PATCH v3 5/7] unit tests: add rule for running with test-tool Josh Steadmon
2024-02-23 23:33 ` [PATCH v3 6/7] t/Makefile: run unit tests alongside shell tests Josh Steadmon
2024-03-27 8:58 ` Jeff King
2024-04-11 18:44 ` Josh Steadmon
2024-04-12 4:29 ` Jeff King
2024-04-24 18:57 ` Josh Steadmon
2024-02-23 23:33 ` [PATCH v3 7/7] ci: use test-tool as unit test runner on Windows Josh Steadmon
2024-03-26 21:33 ` [PATCH v3 0/7] test-tool: add unit test suite runner Junio C Hamano
2024-03-27 9:00 ` Jeff King
2024-04-24 19:14 ` [PATCH v4 " Josh Steadmon
2024-04-24 19:14 ` [PATCH v4 1/7] t0080: turn t-basic unit test into a helper Josh Steadmon
2024-04-24 19:14 ` [PATCH v4 2/7] test-tool run-command testsuite: get shell from env Josh Steadmon
2024-04-24 19:14 ` [PATCH v4 3/7] test-tool run-command testsuite: remove hardcoded filter Josh Steadmon
2024-04-24 19:14 ` [PATCH v4 4/7] test-tool run-command testsuite: support unit tests Josh Steadmon
2024-04-24 19:14 ` [PATCH v4 5/7] unit tests: add rule for running with test-tool Josh Steadmon
2024-04-24 19:14 ` [PATCH v4 6/7] t/Makefile: run unit tests alongside shell tests Josh Steadmon
2024-04-24 21:25 ` Junio C Hamano
2024-04-30 19:49 ` Josh Steadmon
2024-05-03 18:02 ` Jeff King
2024-05-03 19:17 ` Junio C Hamano
2024-05-06 19:58 ` Josh Steadmon
2024-04-24 19:14 ` [PATCH v4 7/7] ci: use test-tool as unit test runner on Windows Josh Steadmon
2024-04-30 19:55 ` [PATCH v5 0/7] test-tool: add unit test suite runner Josh Steadmon
2024-04-30 19:55 ` [PATCH v5 1/7] t0080: turn t-basic unit test into a helper Josh Steadmon
2024-04-30 19:55 ` [PATCH v5 2/7] test-tool run-command testsuite: get shell from env Josh Steadmon
2024-04-30 19:55 ` [PATCH v5 3/7] test-tool run-command testsuite: remove hardcoded filter Josh Steadmon
2024-04-30 19:55 ` [PATCH v5 4/7] test-tool run-command testsuite: support unit tests Josh Steadmon
2024-04-30 19:55 ` [PATCH v5 5/7] unit tests: add rule for running with test-tool Josh Steadmon
2024-04-30 19:55 ` [PATCH v5 6/7] t/Makefile: run unit tests alongside shell tests Josh Steadmon
2024-04-30 21:05 ` Junio C Hamano
2024-05-06 19:58 ` Josh Steadmon
2024-04-30 19:55 ` [PATCH v5 7/7] ci: use test-tool as unit test runner on Windows Josh Steadmon
2024-04-30 21:15 ` [PATCH v5 0/7] test-tool: add unit test suite runner Junio C Hamano
2024-05-06 20:02 ` Josh Steadmon
2024-05-06 19:57 ` [PATCH v6 " Josh Steadmon
2024-05-06 19:57 ` [PATCH v6 1/7] t0080: turn t-basic unit test into a helper Josh Steadmon
2024-05-06 19:57 ` [PATCH v6 2/7] test-tool run-command testsuite: get shell from env Josh Steadmon
2024-05-06 19:57 ` [PATCH v6 3/7] test-tool run-command testsuite: remove hardcoded filter Josh Steadmon
2024-05-06 19:57 ` [PATCH v6 4/7] test-tool run-command testsuite: support unit tests Josh Steadmon
2024-05-06 19:57 ` [PATCH v6 5/7] unit tests: add rule for running with test-tool Josh Steadmon
2024-05-06 19:57 ` [PATCH v6 6/7] t/Makefile: run unit tests alongside shell tests Josh Steadmon
2024-05-06 19:57 ` [PATCH v6 7/7] ci: use test-tool as unit test runner on Windows Josh Steadmon
2024-05-06 21:11 ` [PATCH v6 0/7] test-tool: add unit test suite runner Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a9f67ed703c8314f0f0507ffb83b503717b846b3.1705443632.git.steadmon@google.com \
--to=steadmon@google.com \
--cc=git@vger.kernel.org \
--cc=johannes.schindelin@gmx.de \
--cc=peff@peff.net \
--cc=phillip.wood@dunelm.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).