[PATCH v3 3/8] tests: add a test mode for SANITIZE=leak, run it in CI

["Ævar Arnfjörð Bjarmason" <avarab@gmail.com>]

While git can be compiled with SANITIZE=leak there has been no
corresponding GIT_TEST_* mode for it, i.e. memory leaks have been
fixed as one-offs without structured regression testing.

This change add such a mode, and a new linux-SANITIZE=leak CI
target. The test mode and CI target only runs a whitelist of
known-good tests using a mechanism discussed below, to ensure that we
won't add regressions to code that's had its memory leaks fixed.

The CI target uses a new GIT_TEST_PASSING_SANITIZE_LEAK=true test
mode. When running in that mode all tests except those that have opted
themselves in to running by setting and exporting
TEST_PASSES_SANITIZE_LEAK=true before sourcing test-lib.sh.

I'm adding a "test-pragma-SANITIZE=leak-ok.sh" wrapper for setting and
exporting that variable, as the assignment/export boilerplate would
otherwise get quite verbose and repetitive in subsequent commits.

The tests using the "test-pragma-SANITIZE=leak-ok.sh" pragma can in
turn make use of the "SANITIZE_LEAK" prerequisite added in a preceding
commit, should they wish to selectively skip tests even under
"GIT_TEST_PASSING_SANITIZE_LEAK=true".

Now tests that don't set the "test-pragma-SANITIZE=leak-ok.sh" pragma
will be skipped under GIT_TEST_PASSING_SANITIZE_LEAK=true:

    $ GIT_TEST_PASSING_SANITIZE_LEAK=true ./t0001-init.sh
    1..0 # SKIP skip all tests in t0001 under SANITIZE=leak, TEST_PASSES_SANITIZE_LEAK not set

In subsequents commit we'll conservatively add more
TEST_PASSES_SANITIZE_LEAK=true annotations. The idea is that as memory
leaks are fixed we can add more known-good tests to this CI target, to
ensure that we won't have regressions.

As of writing this we've got major regressions between master..seen,
i.e. the t000*.sh tests and more fixed since 31f9acf9ce2 (Merge branch
'ah/plugleaks', 2021-08-04) have regressed recently.

See the discussion at <87czsv2idy.fsf@evledraar.gmail.com> about the
lack of this sort of test mode, and 0e5bba53af (add UNLEAK annotation
for reducing leak false positives, 2017-09-08) for the initial
addition of SANITIZE=leak.

See also 09595ab381 (Merge branch 'jk/leak-checkers', 2017-09-19),
7782066f67 (Merge branch 'jk/apache-lsan', 2019-05-19) and the recent
936e58851a (Merge branch 'ah/plugleaks', 2021-05-07) for some of the
past history of "one-off" SANITIZE=leak (and more) fixes.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml        |  2 ++
 ci/install-dependencies.sh        |  4 ++--
 ci/lib.sh                         | 11 +++++++++--
 ci/run-build-and-tests.sh         |  4 ++--
 t/README                          |  7 +++++++
 t/t0000-basic.sh                  |  1 +
 t/test-lib.sh                     | 21 +++++++++++++++++++++
 t/test-pragma-SANITIZE=leak-ok.sh |  8 ++++++++
 8 files changed, 52 insertions(+), 6 deletions(-)
 create mode 100644 t/test-pragma-SANITIZE=leak-ok.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 47876a4f02e..d11b971f970 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -232,6 +232,8 @@ jobs:
           - jobname: linux-gcc-default
             cc: gcc
             pool: ubuntu-latest
+          - jobname: linux-SANITIZE=leak
+            pool: ubuntu-latest
     env:
       CC: ${{matrix.vector.cc}}
       jobname: ${{matrix.vector.jobname}}
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 5772081b6e5..30276ae1e00 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -12,13 +12,13 @@ UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
  libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl"
 
 case "$jobname" in
-linux-clang|linux-gcc)
+linux-clang|linux-gcc|linux-SANITIZE=leak)
 	sudo apt-add-repository -y "ppa:ubuntu-toolchain-r/test"
 	sudo apt-get -q update
 	sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \
 		$UBUNTU_COMMON_PKGS
 	case "$jobname" in
-	linux-gcc)
+	linux-gcc|linux-SANITIZE=leak)
 		sudo apt-get -q -y install gcc-8
 		;;
 	esac
diff --git a/ci/lib.sh b/ci/lib.sh
index 33b9777ab7e..d86b83ed203 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -183,9 +183,9 @@ export GIT_TEST_CLONE_2GB=true
 export SKIP_DASHED_BUILT_INS=YesPlease
 
 case "$jobname" in
-linux-clang|linux-gcc)
+linux-clang|linux-gcc|linux-SANITIZE=leak)
 	case "$jobname" in
-	linux-gcc)
+	linux-gcc|linux-SANITIZE=leak)
 		export CC=gcc-8
 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python3"
 		;;
@@ -237,4 +237,11 @@ linux-musl)
 	;;
 esac
 
+case "$jobname" in
+linux-SANITIZE=leak)
+	export SANITIZE=leak
+	export GIT_TEST_PASSING_SANITIZE_LEAK=true
+	;;
+esac
+
 MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 3ce81ffee94..f0b9775b6c7 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -12,7 +12,7 @@ esac
 
 make
 case "$jobname" in
-linux-gcc)
+linux-gcc|linux-SANITIZE=leak)
 	export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
 	make test
 	export GIT_TEST_SPLIT_INDEX=yes
@@ -29,7 +29,7 @@ linux-gcc)
 	export GIT_TEST_CHECKOUT_WORKERS=2
 	make test
 	;;
-linux-clang)
+linux-clang|linux-SANITIZE=leak)
 	export GIT_TEST_DEFAULT_HASH=sha1
 	make test
 	export GIT_TEST_DEFAULT_HASH=sha256
diff --git a/t/README b/t/README
index 9e701223020..f5dfac568d1 100644
--- a/t/README
+++ b/t/README
@@ -448,6 +448,13 @@ GIT_TEST_CHECKOUT_WORKERS=<n> overrides the 'checkout.workers' setting
 to <n> and 'checkout.thresholdForParallelism' to 0, forcing the
 execution of the parallel-checkout code.
 
+GIT_TEST_PASSING_SANITIZE_LEAK=<boolean> when compiled with
+SANITIZE=leak will run only those tests that have whitelisted
+themselves as passing with no memory leaks. Do this by sourcing
+"test-pragma-SANITIZE=leak-ok.sh" before sourcing "test-lib.sh" itself
+at the top of the test script. This test mode is used by the
+"linux-SANITIZE=leak" CI target.
+
 Naming Tests
 ------------
 
diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index cb87768513c..14836c97cc6 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -18,6 +18,7 @@ swapping compression and hashing order, the person who is making the
 modification *should* take notice and update the test vectors here.
 '
 
+. ./test-pragma-SANITIZE=leak-ok.sh
 . ./test-lib.sh
 
 try_local_xy () {
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 4ab18914a3d..332dd59257d 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1379,6 +1379,27 @@ then
 	test_done
 fi
 
+# Aggressively skip non-whitelisted tests when compiled with
+# SANITIZE=leak
+if test -n "$SANITIZE_LEAK"
+then
+	if test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false
+	then
+		# We need to see it in "git env--helper" (via
+		# test_bool_env)
+		export TEST_PASSES_SANITIZE_LEAK
+
+		if ! test_bool_env TEST_PASSES_SANITIZE_LEAK false
+		then
+			skip_all="skipping $this_test under GIT_TEST_PASSING_SANITIZE_LEAK=true"
+			test_done
+		fi
+	fi
+elif test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false
+then
+	error "GIT_TEST_PASSING_SANITIZE_LEAK=true has no effect except when compiled with SANITIZE=leak"
+fi
+
 # Last-minute variable setup
 HOME="$TRASH_DIRECTORY"
 GNUPGHOME="$HOME/gnupg-home-not-used"
diff --git a/t/test-pragma-SANITIZE=leak-ok.sh b/t/test-pragma-SANITIZE=leak-ok.sh
new file mode 100644
index 00000000000..5f03397075d
--- /dev/null
+++ b/t/test-pragma-SANITIZE=leak-ok.sh
@@ -0,0 +1,8 @@
+#!/bin/sh
+
+## This "pragma" (as in "perldoc perlpragma") declares that the test
+## will pass under GIT_TEST_PASSING_SANITIZE_LEAK=true. Source this
+## before sourcing test-lib.sh
+
+TEST_PASSES_SANITIZE_LEAK=true
+export TEST_PASSES_SANITIZE_LEAK
-- 
2.33.0.805.g739b16c2189