git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Josh Steadmon <steadmon@google.com>
To: git@vger.kernel.org
Cc: johannes.schindelin@gmx.de, peff@peff.net,
	phillip.wood@dunelm.org.uk,  gitster@pobox.com
Subject: [RFC PATCH v2 1/6] t0080: turn t-basic unit test into a helper
Date: Fri,  2 Feb 2024 16:50:26 -0800	[thread overview]
Message-ID: <da756b4bfb9d1ce0d1213d585e72acfbf667e2a2.1706921262.git.steadmon@google.com> (raw)
In-Reply-To: <cover.1706921262.git.steadmon@google.com>

While t/unit-tests/t-basic.c uses the unit-test framework added in
e137fe3b29 (unit tests: add TAP unit test framework, 2023-11-09), it is
not a true unit test in that it intentionally fails in order to exercise
various codepaths in the unit-test framework. Thus, we intentionally
exclude it when running unit tests through the various t/Makefile
targets. Instead, it is executed by t0080-unit-test-output.sh, which
verifies its output follows the TAP format expected for the various
pass, skip, or fail cases.

As such, it makes more sense for t-basic to be a helper item for
t0080-unit-test-output.sh, so let's move it to
t/helper/test-example-tap.c and adjust Makefiles as necessary.

This has the additional benefit that test harnesses seeking to run all
unit tests can find them with a simple glob of "t/unit-tests/bin/t-*",
with no exceptions needed. This will be important in a later patch where
we add support for running the unit tests via a test-tool subcommand.

Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 Makefile                                      |  4 ++--
 t/Makefile                                    |  3 +--
 .../t-basic.c => helper/test-example-tap.c}   |  5 ++--
 t/helper/test-tool.c                          |  1 +
 t/helper/test-tool.h                          |  1 +
 t/t0080-unit-test-output.sh                   | 24 +++++++++----------
 6 files changed, 20 insertions(+), 18 deletions(-)
 rename t/{unit-tests/t-basic.c => helper/test-example-tap.c} (95%)

diff --git a/Makefile b/Makefile
index 23723367b8..ba55d817ee 100644
--- a/Makefile
+++ b/Makefile
@@ -802,6 +802,7 @@ TEST_BUILTINS_OBJS += test-dump-split-index.o
 TEST_BUILTINS_OBJS += test-dump-untracked-cache.o
 TEST_BUILTINS_OBJS += test-env-helper.o
 TEST_BUILTINS_OBJS += test-example-decorate.o
+TEST_BUILTINS_OBJS += test-example-tap.o
 TEST_BUILTINS_OBJS += test-find-pack.o
 TEST_BUILTINS_OBJS += test-fsmonitor-client.o
 TEST_BUILTINS_OBJS += test-genrandom.o
@@ -1338,7 +1339,6 @@ THIRD_PARTY_SOURCES += compat/regex/%
 THIRD_PARTY_SOURCES += sha1collisiondetection/%
 THIRD_PARTY_SOURCES += sha1dc/%
 
-UNIT_TEST_PROGRAMS += t-basic
 UNIT_TEST_PROGRAMS += t-mem-pool
 UNIT_TEST_PROGRAMS += t-strbuf
 UNIT_TEST_PROGRAMS += t-ctype
@@ -3218,7 +3218,7 @@ perf: all
 
 .PRECIOUS: $(TEST_OBJS)
 
-t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS))
+t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS)) $(UNIT_TEST_DIR)/test-lib.o
 
 t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS) $(REFTABLE_TEST_LIB)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(filter %.a,$^) $(LIBS)
diff --git a/t/Makefile b/t/Makefile
index 281f4c3534..1283c90c10 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -44,8 +44,7 @@ TINTEROP = $(sort $(wildcard interop/i[0-9][0-9][0-9][0-9]-*.sh))
 CHAINLINTTESTS = $(sort $(patsubst chainlint/%.test,%,$(wildcard chainlint/*.test)))
 CHAINLINT = '$(PERL_PATH_SQ)' chainlint.pl
 UNIT_TEST_SOURCES = $(wildcard unit-tests/t-*.c)
-UNIT_TEST_PROGRAMS = $(patsubst unit-tests/%.c,unit-tests/bin/%$(X),$(UNIT_TEST_SOURCES))
-UNIT_TESTS = $(sort $(filter-out unit-tests/bin/t-basic%,$(UNIT_TEST_PROGRAMS)))
+UNIT_TESTS = $(patsubst unit-tests/%.c,unit-tests/bin/%$(X),$(UNIT_TEST_SOURCES))
 
 # `test-chainlint` (which is a dependency of `test-lint`, `test` and `prove`)
 # checks all tests in all scripts via a single invocation, so tell individual
diff --git a/t/unit-tests/t-basic.c b/t/helper/test-example-tap.c
similarity index 95%
rename from t/unit-tests/t-basic.c
rename to t/helper/test-example-tap.c
index fda1ae59a6..21e4848e78 100644
--- a/t/unit-tests/t-basic.c
+++ b/t/helper/test-example-tap.c
@@ -1,4 +1,5 @@
-#include "test-lib.h"
+#include "t/unit-tests/test-lib.h"
+#include "test-tool.h"
 
 /*
  * The purpose of this "unit test" is to verify a few invariants of the unit
@@ -69,7 +70,7 @@ static void t_empty(void)
 	; /* empty */
 }
 
-int cmd_main(int argc, const char **argv)
+int cmd__example_tap(int argc, const char **argv)
 {
 	test_res = TEST(check_res = check_int(1, ==, 1), "passing test");
 	TEST(t_res(1), "passing test and assertion return 1");
diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c
index 33b9501c21..bb5c04c9c0 100644
--- a/t/helper/test-tool.c
+++ b/t/helper/test-tool.c
@@ -29,6 +29,7 @@ static struct test_cmd cmds[] = {
 	{ "dump-untracked-cache", cmd__dump_untracked_cache },
 	{ "env-helper", cmd__env_helper },
 	{ "example-decorate", cmd__example_decorate },
+	{ "example-tap", cmd__example_tap },
 	{ "find-pack", cmd__find_pack },
 	{ "fsmonitor-client", cmd__fsmonitor_client },
 	{ "genrandom", cmd__genrandom },
diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h
index b72f07ded9..38001bd1c6 100644
--- a/t/helper/test-tool.h
+++ b/t/helper/test-tool.h
@@ -23,6 +23,7 @@ int cmd__dump_untracked_cache(int argc, const char **argv);
 int cmd__dump_reftable(int argc, const char **argv);
 int cmd__env_helper(int argc, const char **argv);
 int cmd__example_decorate(int argc, const char **argv);
+int cmd__example_tap(int argc, const char **argv);
 int cmd__find_pack(int argc, const char **argv);
 int cmd__fsmonitor_client(int argc, const char **argv);
 int cmd__genrandom(int argc, const char **argv);
diff --git a/t/t0080-unit-test-output.sh b/t/t0080-unit-test-output.sh
index 961b54b06c..83b1e3b7f5 100755
--- a/t/t0080-unit-test-output.sh
+++ b/t/t0080-unit-test-output.sh
@@ -8,50 +8,50 @@ test_expect_success 'TAP output from unit tests' '
 	cat >expect <<-EOF &&
 	ok 1 - passing test
 	ok 2 - passing test and assertion return 1
-	# check "1 == 2" failed at t/unit-tests/t-basic.c:76
+	# check "1 == 2" failed at t/helper/test-example-tap.c:77
 	#    left: 1
 	#   right: 2
 	not ok 3 - failing test
 	ok 4 - failing test and assertion return 0
 	not ok 5 - passing TEST_TODO() # TODO
 	ok 6 - passing TEST_TODO() returns 1
-	# todo check ${SQ}check(x)${SQ} succeeded at t/unit-tests/t-basic.c:25
+	# todo check ${SQ}check(x)${SQ} succeeded at t/helper/test-example-tap.c:26
 	not ok 7 - failing TEST_TODO()
 	ok 8 - failing TEST_TODO() returns 0
-	# check "0" failed at t/unit-tests/t-basic.c:30
+	# check "0" failed at t/helper/test-example-tap.c:31
 	# skipping test - missing prerequisite
-	# skipping check ${SQ}1${SQ} at t/unit-tests/t-basic.c:32
+	# skipping check ${SQ}1${SQ} at t/helper/test-example-tap.c:33
 	ok 9 - test_skip() # SKIP
 	ok 10 - skipped test returns 1
 	# skipping test - missing prerequisite
 	ok 11 - test_skip() inside TEST_TODO() # SKIP
 	ok 12 - test_skip() inside TEST_TODO() returns 1
-	# check "0" failed at t/unit-tests/t-basic.c:48
+	# check "0" failed at t/helper/test-example-tap.c:49
 	not ok 13 - TEST_TODO() after failing check
 	ok 14 - TEST_TODO() after failing check returns 0
-	# check "0" failed at t/unit-tests/t-basic.c:56
+	# check "0" failed at t/helper/test-example-tap.c:57
 	not ok 15 - failing check after TEST_TODO()
 	ok 16 - failing check after TEST_TODO() returns 0
-	# check "!strcmp("\thello\\\\", "there\"\n")" failed at t/unit-tests/t-basic.c:61
+	# check "!strcmp("\thello\\\\", "there\"\n")" failed at t/helper/test-example-tap.c:62
 	#    left: "\011hello\\\\"
 	#   right: "there\"\012"
-	# check "!strcmp("NULL", NULL)" failed at t/unit-tests/t-basic.c:62
+	# check "!strcmp("NULL", NULL)" failed at t/helper/test-example-tap.c:63
 	#    left: "NULL"
 	#   right: NULL
-	# check "${SQ}a${SQ} == ${SQ}\n${SQ}" failed at t/unit-tests/t-basic.c:63
+	# check "${SQ}a${SQ} == ${SQ}\n${SQ}" failed at t/helper/test-example-tap.c:64
 	#    left: ${SQ}a${SQ}
 	#   right: ${SQ}\012${SQ}
-	# check "${SQ}\\\\${SQ} == ${SQ}\\${SQ}${SQ}" failed at t/unit-tests/t-basic.c:64
+	# check "${SQ}\\\\${SQ} == ${SQ}\\${SQ}${SQ}" failed at t/helper/test-example-tap.c:65
 	#    left: ${SQ}\\\\${SQ}
 	#   right: ${SQ}\\${SQ}${SQ}
 	not ok 17 - messages from failing string and char comparison
-	# BUG: test has no checks at t/unit-tests/t-basic.c:91
+	# BUG: test has no checks at t/helper/test-example-tap.c:92
 	not ok 18 - test with no checks
 	ok 19 - test with no checks returns 0
 	1..19
 	EOF
 
-	! "$GIT_BUILD_DIR"/t/unit-tests/bin/t-basic >actual &&
+	! test-tool example-tap >actual &&
 	test_cmp expect actual
 '
 
-- 
2.43.0.594.gd9cf4e227d-goog



  reply	other threads:[~2024-02-03  0:50 UTC|newest]

Thread overview: 87+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-01-16 22:22 [RFC PATCH 0/4] test-tool: add unit test suite runner Josh Steadmon
2024-01-16 22:22 ` [RFC PATCH 1/4] t0080: turn t-basic unit test into a helper Josh Steadmon
2024-01-16 22:54   ` Junio C Hamano
2024-01-23  0:43     ` Jeff King
2024-02-01 19:40       ` Josh Steadmon
2024-01-16 22:22 ` [RFC PATCH 2/4] test-tool run-command testsuite: support unit tests Josh Steadmon
2024-01-16 23:18   ` Junio C Hamano
2024-01-16 23:40     ` Junio C Hamano
2024-01-23  0:59       ` Jeff King
2024-02-01 22:06     ` Josh Steadmon
2024-02-01 22:26       ` Junio C Hamano
2024-02-01 23:08     ` Josh Steadmon
2024-02-01 23:14       ` Josh Steadmon
2024-01-16 22:23 ` [RFC PATCH 3/4] unit tests: add rule for running with test-tool Josh Steadmon
2024-01-16 22:23 ` [RFC PATCH 4/4] t/Makefile: run unit tests alongside shell tests Josh Steadmon
2024-01-16 23:24 ` [RFC PATCH 0/4] test-tool: add unit test suite runner Junio C Hamano
2024-02-03  0:50 ` [RFC PATCH v2 0/6] " Josh Steadmon
2024-02-03  0:50   ` Josh Steadmon [this message]
2024-02-07 20:55     ` [RFC PATCH v2 1/6] t0080: turn t-basic unit test into a helper Junio C Hamano
2024-02-12 20:42       ` Josh Steadmon
2024-02-07 22:58     ` Jeff King
2024-02-08  0:09       ` Junio C Hamano
2024-02-12 20:53       ` Josh Steadmon
2024-02-12 21:27         ` Junio C Hamano
2024-02-13  7:41           ` Jeff King
2024-02-13 14:36             ` rsbecker
2024-02-22 23:57               ` Josh Steadmon
2024-02-23  0:06                 ` rsbecker
2024-02-13 17:28             ` Junio C Hamano
2024-02-22 23:55             ` Josh Steadmon
2024-02-03  0:50   ` [RFC PATCH v2 2/6] test-tool run-command testsuite: get shell from env Josh Steadmon
2024-02-07 20:55     ` Junio C Hamano
2024-02-12 21:35       ` Josh Steadmon
2024-02-03  0:50   ` [RFC PATCH v2 3/6] test-tool run-command testsuite: remove hardcoded filter Josh Steadmon
2024-02-07 20:55     ` Junio C Hamano
2024-02-12 22:48       ` Josh Steadmon
2024-02-12 22:51         ` Junio C Hamano
2024-02-03  0:50   ` [RFC PATCH v2 4/6] test-tool run-command testsuite: support unit tests Josh Steadmon
2024-02-05 16:16     ` phillip.wood123
2024-02-12 21:15       ` Josh Steadmon
2024-02-07 20:48     ` Junio C Hamano
2024-02-23 22:45       ` Josh Steadmon
2024-02-03  0:50   ` [RFC PATCH v2 5/6] unit tests: add rule for running with test-tool Josh Steadmon
2024-02-07 20:50     ` Junio C Hamano
2024-02-03  0:50   ` [RFC PATCH v2 6/6] t/Makefile: run unit tests alongside shell tests Josh Steadmon
2024-02-07 20:55     ` Junio C Hamano
2024-02-07 22:43       ` Jeff King
2024-02-07 23:26         ` Junio C Hamano
2024-02-03 18:52   ` [RFC PATCH v2 0/6] test-tool: add unit test suite runner Junio C Hamano
2024-02-12 22:50     ` Josh Steadmon
2024-02-07 21:14   ` Junio C Hamano
2024-02-23 23:33 ` [PATCH v3 0/7] " Josh Steadmon
2024-02-23 23:33   ` [PATCH v3 1/7] t0080: turn t-basic unit test into a helper Josh Steadmon
2024-02-23 23:33   ` [PATCH v3 2/7] test-tool run-command testsuite: get shell from env Josh Steadmon
2024-02-23 23:33   ` [PATCH v3 3/7] test-tool run-command testsuite: remove hardcoded filter Josh Steadmon
2024-02-23 23:33   ` [PATCH v3 4/7] test-tool run-command testsuite: support unit tests Josh Steadmon
2024-02-23 23:33   ` [PATCH v3 5/7] unit tests: add rule for running with test-tool Josh Steadmon
2024-02-23 23:33   ` [PATCH v3 6/7] t/Makefile: run unit tests alongside shell tests Josh Steadmon
2024-03-27  8:58     ` Jeff King
2024-04-11 18:44       ` Josh Steadmon
2024-04-12  4:29         ` Jeff King
2024-04-24 18:57           ` Josh Steadmon
2024-02-23 23:33   ` [PATCH v3 7/7] ci: use test-tool as unit test runner on Windows Josh Steadmon
2024-03-26 21:33   ` [PATCH v3 0/7] test-tool: add unit test suite runner Junio C Hamano
2024-03-27  9:00     ` Jeff King
2024-04-24 19:14 ` [PATCH v4 " Josh Steadmon
2024-04-24 19:14   ` [PATCH v4 1/7] t0080: turn t-basic unit test into a helper Josh Steadmon
2024-04-24 19:14   ` [PATCH v4 2/7] test-tool run-command testsuite: get shell from env Josh Steadmon
2024-04-24 19:14   ` [PATCH v4 3/7] test-tool run-command testsuite: remove hardcoded filter Josh Steadmon
2024-04-24 19:14   ` [PATCH v4 4/7] test-tool run-command testsuite: support unit tests Josh Steadmon
2024-04-24 19:14   ` [PATCH v4 5/7] unit tests: add rule for running with test-tool Josh Steadmon
2024-04-24 19:14   ` [PATCH v4 6/7] t/Makefile: run unit tests alongside shell tests Josh Steadmon
2024-04-24 21:25     ` Junio C Hamano
2024-04-30 19:49       ` Josh Steadmon
2024-05-03 18:02       ` Jeff King
2024-05-03 19:17         ` Junio C Hamano
2024-04-24 19:14   ` [PATCH v4 7/7] ci: use test-tool as unit test runner on Windows Josh Steadmon
2024-04-30 19:55 ` [PATCH v5 0/7] test-tool: add unit test suite runner Josh Steadmon
2024-04-30 19:55   ` [PATCH v5 1/7] t0080: turn t-basic unit test into a helper Josh Steadmon
2024-04-30 19:55   ` [PATCH v5 2/7] test-tool run-command testsuite: get shell from env Josh Steadmon
2024-04-30 19:55   ` [PATCH v5 3/7] test-tool run-command testsuite: remove hardcoded filter Josh Steadmon
2024-04-30 19:55   ` [PATCH v5 4/7] test-tool run-command testsuite: support unit tests Josh Steadmon
2024-04-30 19:55   ` [PATCH v5 5/7] unit tests: add rule for running with test-tool Josh Steadmon
2024-04-30 19:55   ` [PATCH v5 6/7] t/Makefile: run unit tests alongside shell tests Josh Steadmon
2024-04-30 21:05     ` Junio C Hamano
2024-04-30 19:55   ` [PATCH v5 7/7] ci: use test-tool as unit test runner on Windows Josh Steadmon
2024-04-30 21:15   ` [PATCH v5 0/7] test-tool: add unit test suite runner Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=da756b4bfb9d1ce0d1213d585e72acfbf667e2a2.1706921262.git.steadmon@google.com \
    --to=steadmon@google.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=johannes.schindelin@gmx.de \
    --cc=peff@peff.net \
    --cc=phillip.wood@dunelm.org.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).