[PATCH 00/25] CI: run "make [test]" directly, use $GITHUB_ENV

[PATCH 00/25] CI: run "make [test]" directly, use $GITHUB_ENV

[Ævar Arnfjörð Bjarmason]

== Applying this

Merging this has a trivial conflct with "seen"'s
js/use-builtin-add-i. The
s/GIT_TEST_ADD_I_USE_BUILTIN=1/GIT_TEST_ADD_I_USE_BUILTIN=0/g" there
needs to be correspondingly changed (using "setenv", no "export") in
ci/lib.sh

== Summary

This series makes the CI less of a special-case by getting rid of
wrapper scripts like "ci/run-build-and-tests.sh" in favor of simply
running "make" or "make test" directly.

It also has changes that make us do less special-ness in
CI. E.g. there's now a "make ci-static-analysis" target that can be
run locally to do what "ci/run-static-analysis.sh" did before. The CI
just invokes that new "make" target, and the
"ci/run-static-analysis.sh" script has been "git rm'd".

The 12/25 here has a more detailed summary[1], but basically the
"ci/lib.sh" is now a script run directly from an early CI step within
a given job. That script runs sets the environment variables needed in
$GITHUB_ENV (what GitHub CI uses to carry forward the environment
between steps).

== Details

This series conflicts with and is a proposed alternative to Johannes's
proposed changes to teach "ci/lib.sh" GitHub specific syntax to
"group" certain parts of the output[2]. That series proposes to keep
the "ci/run-build-and-tests.sh" and similar scripts, but to teach them
to emit markers indicating when the "build" part commences, then
"test" etc.

I think the approach offered here is better as elaborated on in
12/25[1], especially because as Johannes notes[3] we only have one
level of "grouping" within a "step" available to us. In his version
using it for "build" v.s. "test" precludes being able to group parts
of that "build" or "test" output in the future.

The following compares CI output between the three with a
change-on-top which caused a compilation error ("this" is this series,
"js" is Johannes's) when clicking on the "linux-gcc" failure:

  master: https://github.com/avar/git/runs/5274251909?check_suite_focus=true
  this: https://github.com/avar/git/runs/5274464670?check_suite_focus=true
  js: https://github.com/avar/git/runs/5272239403?check_suite_focus=true

  Here we went from 93 lines of output on "master" to 47 in "this"
  (107 in "js"). Note also how in "this" we can:

  - Expand the "Run make" to get the "MAKEFLAGS" and other variables
    used in the step. That's now a 3-line summary instead of the
    first 50 lines being the old "ci/lib.sh" "set -x" output.

  - Because "make" failed, we have an elided "make test" step that's
    not being run below. We can thus see what steps our failure caused
    us to skip.

  - Unlike the "js" version we'll show the compilation error by
    default (the "js" version is grouped and needs to be expanded),
    but our output is now brief enough that that's no longer
    surrounded by other irrelevant output.

  - Unlike the "js" version there is no "CI setup" group within each
    step. That work is in the earlier "ci/lib.sh --build" step
    instead, which sets the config for all subsequent steps.

For other output changes look at the difference between "master" and
"this" at:

  master: https://github.com/git/git/actions/runs/1866786595
  this: https://github.com/avar/git/actions/runs/1876270588

Not explicitly covered in the summary above is that various other
parts of ci/* were doing the same sort of within-step setup, but now
do so via cross-step passing of variables via $GITHUB_ENV. E.g. on
"master" the test slices for the Windows tests have a lot of verbosity
before they get to running the test itself:

    https://github.com/git/git/runs/5254267914?check_suite_focus=true#step:5:56

In the "js" version the test output is hidden ("grouped"), but we've
got the same amount of verbosity by default:

    https://github.com/gitgitgadget/git/runs/4937491771?check_suite_focus=true#step:5:67

Whereas in "this" version that verbosity is in the preceding "select
tests" step, with the "test" step showing only the relevant end-state
of the "$T" variable we'll use in the Makefile (hidden by default,
expanded in this link):

    https://github.com/avar/git/runs/5274558869?check_suite_focus=true#step:7:13

This series does not attempt to replace the use of the "group" output
used for the "ok" or "not ok" portion of tests in Johannes's
series. When a test fails the output in this series (sans config
discovery not being repeated, and now summarized in the $GITHUB_ENV
drop-down) is substantially the same as on "master".

My summary at [4] goes into other overlap & non-overlap between the
two. I think using the "group" output for those purposes might be
useful, although I left some feedback on [5] with problems in the
current "js" implementation.

I do think any such changes should follow behind this series, as doing
that sort of grouping once we can effectively free up an extra "group"
level (by peeling those off into "steps", as is done here) would be
much more useful.

1. https://lore.kernel.org/git/patch-12.25-dfd823f2e7d-20220221T122605Z-avarab@gmail.com
2. https://lore.kernel.org/git/pull.1117.git.1643050574.gitgitgadget@gmail.com/
3. https://lore.kernel.org/git/9333ba781b8240f704e739b00d274f8c3d887e39.1643050574.git.gitgitgadget@gmail.com/
4. https://lore.kernel.org/git/220127.86ilu5cdnf.gmgdl@evledraar.gmail.com/
5. https://lore.kernel.org/git/220126.86sftbfjl4.gmgdl@evledraar.gmail.com/

Ævar Arnfjörð Bjarmason (25):
  CI: run "set -ex" early in ci/lib.sh
  CI: make "$jobname" explicit, remove fallback
  CI: remove more dead Travis CI support
  CI: remove dead "tree skipping" code
  CI: remove unused Azure ci/* code
  CI: don't have "git grep" invoke a pager in tree content check
  CI: have "static-analysis" run a "make ci-static-analysis" target
  CI: have "static-analysis" run "check-builtins", not "documentation"
  CI: move p4 and git-lfs variables to ci/install-dependencies.sh
  CI: consistently use "export" in ci/lib.sh
  CI: export variables via a wrapper
  CI: remove "run-build-and-tests.sh", run "make [test]" directly
  CI: check ignored unignored build artifacts in "win[+VS] build" too
  CI: invoke "make artifacts-tar" directly in windows-build
  CI: split up and reduce "ci/test-documentation.sh"
  CI: combine ci/install{,-docker}-dependencies.sh
  CI: move "env" definitions into ci/lib.sh
  ci/run-test-slice.sh: replace shelling out with "echo"
  CI: pre-select test slice in Windows & VS tests
  CI: only invoke ci/lib.sh as "steps" in main.yml
  CI: narrow down variable definitions in --build and --test
  CI: add more variables to MAKEFLAGS, except under vs-build
  CI: stop over-setting the $CC variable
  CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case
  CI: don't use "set -x" in "ci/lib.sh" output

 .github/workflows/main.yml            | 100 +++++---
 Makefile                              |  32 ++-
 ci/check-directional-formatting.bash  |   2 +-
 ci/check-unignored-build-artifacts.sh |  20 ++
 ci/install-dependencies.sh            |  53 ++++-
 ci/install-docker-dependencies.sh     |  22 --
 ci/lib-ci-type.sh                     |  10 +
 ci/lib-tput.sh                        |   2 +
 ci/lib.sh                             | 315 ++++++++++++--------------
 ci/make-test-artifacts.sh             |  12 -
 ci/mount-fileshare.sh                 |  25 --
 ci/print-test-failures.sh             |  16 +-
 ci/run-build-and-tests.sh             |  54 -----
 ci/run-docker-build.sh                |  66 ------
 ci/run-docker.sh                      |  47 ----
 ci/run-static-analysis.sh             |  32 ---
 ci/run-test-slice.sh                  |  17 --
 ci/select-test-slice.sh               |  10 +
 ci/test-documentation.sh              |  37 +--
 ci/util/extract-trash-dirs.sh         |  50 ----
 20 files changed, 345 insertions(+), 577 deletions(-)
 create mode 100755 ci/check-unignored-build-artifacts.sh
 delete mode 100755 ci/install-docker-dependencies.sh
 create mode 100644 ci/lib-ci-type.sh
 create mode 100644 ci/lib-tput.sh
 delete mode 100755 ci/make-test-artifacts.sh
 delete mode 100755 ci/mount-fileshare.sh
 delete mode 100755 ci/run-build-and-tests.sh
 delete mode 100755 ci/run-docker-build.sh
 delete mode 100755 ci/run-docker.sh
 delete mode 100755 ci/run-static-analysis.sh
 delete mode 100755 ci/run-test-slice.sh
 create mode 100755 ci/select-test-slice.sh
 delete mode 100755 ci/util/extract-trash-dirs.sh

-- 
2.35.1.1132.ga1fe46f8690

[PATCH 01/25] CI: run "set -ex" early in ci/lib.sh

[Ævar Arnfjörð Bjarmason]

Change ci/lib.sh to run "set -ex" before it does anything else, before
this we were omitting a few "export" commands in our trace output that
happened before we enabled tracing.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index cbc2f8f1caa..57141d38a85 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,5 +1,11 @@
 # Library of functions shared by all CI scripts
 
+# Set 'exit on error' for all CI scripts to let the caller know that
+# something went wrong.
+# Set tracing executed commands, primarily setting environment variables
+# and installing dependencies.
+set -ex
+
 skip_branch_tip_with_tag () {
 	# Sometimes, a branch is pushed at the same time the tag that points
 	# at the same commit as the tip of the branch is pushed, and building
@@ -85,12 +91,6 @@ export TERM=${TERM:-dumb}
 # Clear MAKEFLAGS that may come from the outside world.
 export MAKEFLAGS=
 
-# Set 'exit on error' for all CI scripts to let the caller know that
-# something went wrong.
-# Set tracing executed commands, primarily setting environment variables
-# and installing dependencies.
-set -ex
-
 if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
 then
 	CI_TYPE=azure-pipelines
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 02/25] CI: make "$jobname" explicit, remove fallback

[Ævar Arnfjörð Bjarmason]

Remove the $jobname fallback last adjusted in b011fabd6e0 (ci/lib.sh:
encapsulate Travis-specific things, 2019-01-27).

This makes all of our CI jobs have an explicit $jobname, and removes
the special-case of the Windows jobs implicitly setting one (that they
don't currently use). In subsequent commits we'll use this new
$jobname explicitly.

This is a follow-up to my 25715419bf4 (CI: don't run "make test" twice
in one job, 2021-11-23), since then all the jobs except the Windows
jobs have had an explicit $jobname.

In the case of the Windows jobs we were using a jobname of windows-gcc
for both, due to picking "gcc" as a default fallback for "$CC" in
ci/lib.sh.

Since we'll need this to run ci/lib.sh it makes sense to move the
check to the top of the file. Subsequent commits will add more such
assertions to it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml |  8 ++++++++
 ci/lib.sh                  | 12 +++++++-----
 2 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index c35200defb9..309b9141249 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -79,6 +79,8 @@ jobs:
 
   windows-build:
     name: win build
+    env:
+      jobname: windows-build
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
     runs-on: windows-latest
@@ -100,6 +102,8 @@ jobs:
         path: artifacts
   windows-test:
     name: win test
+    env:
+      jobname: windows-test
     runs-on: windows-latest
     needs: [windows-build]
     strategy:
@@ -131,6 +135,8 @@ jobs:
         path: ${{env.FAILED_TEST_ARTIFACTS}}
   vs-build:
     name: win+VS build
+    env:
+      jobname: vs-build
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
     env:
@@ -183,6 +189,8 @@ jobs:
         path: artifacts
   vs-test:
     name: win+VS test
+    env:
+      jobname: vs-test
     runs-on: windows-latest
     needs: vs-build
     strategy:
diff --git a/ci/lib.sh b/ci/lib.sh
index 57141d38a85..d2a7c33f536 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -6,6 +6,13 @@
 # and installing dependencies.
 set -ex
 
+# Starting assertions
+if test -z "$jobname"
+then
+	echo "must set a CI jobname" >&2
+	exit 1
+fi
+
 skip_branch_tip_with_tag () {
 	# Sometimes, a branch is pushed at the same time the tag that points
 	# at the same commit as the tip of the branch is pushed, and building
@@ -146,11 +153,6 @@ test -n "${DONT_SKIP_TAGS-}" ||
 skip_branch_tip_with_tag
 skip_good_tree
 
-if test -z "$jobname"
-then
-	jobname="$CI_OS_NAME-$CC"
-fi
-
 export DEVELOPER=1
 export DEFAULT_TEST_TARGET=prove
 export GIT_TEST_CLONE_2GB=true
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 03/25] CI: remove more dead Travis CI support

[Ævar Arnfjörð Bjarmason]

Remove code that I missed in 4a6e4b96026 (CI: remove Travis CI
support, 2021-11-23). This code was only called from or used by the
now-removed .travis.yml, or needed by the Travis CI environment.

For the symlinking in ci/run-build-and-tests.sh: Back when
3c93b829205 (travis-ci: build Git during the 'script' phase,
2018-01-08) and 4b060a4d973 (ci: use a junction on Windows instead of
a symlink, 2019-01-27) adjusted this "ln" command, the Windows build
would use ci/run-build-and-tests.sh.

As seen in 889cacb6897 (ci: configure GitHub Actions for CI/PR,
2020-04-11) the current windows build uses a different entry point
under the GitHub CI, which doesn't use this .prove caching. Namely
"ci/run-test-slice.sh".

We can be certain that it's never used in "ci/run-test-slice.sh"
because to have a ".prove" file we'd need to use "--state=save", which
we only do in the dead Azure codepath in ci/lib.sh. If it were used it
would do the wrong thing, because the different test slices would each
try to clobber the same "t/.prove" file.

If a subsequent run then used the -"-state=failed,slow,save" it would
defeat the purpose of "ci/run-test-slice.sh", since all slices would
then run all tests. I.e. behavior of prove's "--state" options is to
select tests to run from the provided "--state" file, in addition to
those specified on the command-line.

For ci/run-docker{,-build}.sh: It was likewise last referenced in the
.travis.yml removed in my 4a6e4b96026. The current "dockerized" run in
".github/workflows/main.yml" calls the same entry points as the main
"regular" job.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                     | 10 ++----
 ci/print-test-failures.sh     |  3 --
 ci/run-build-and-tests.sh     |  5 ---
 ci/run-docker-build.sh        | 66 -----------------------------------
 ci/run-docker.sh              | 47 -------------------------
 ci/run-test-slice.sh          |  5 ---
 ci/util/extract-trash-dirs.sh | 50 --------------------------
 7 files changed, 2 insertions(+), 184 deletions(-)
 delete mode 100755 ci/run-docker-build.sh
 delete mode 100755 ci/run-docker.sh
 delete mode 100755 ci/util/extract-trash-dirs.sh

diff --git a/ci/lib.sh b/ci/lib.sh
index d2a7c33f536..c3c06d66862 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -106,9 +106,6 @@ then
 	CI_COMMIT="$BUILD_SOURCEVERSION"
 	CI_JOB_ID="$BUILD_BUILDID"
 	CI_JOB_NUMBER="$BUILD_BUILDNUMBER"
-	CI_OS_NAME="$(echo "$AGENT_OS" | tr A-Z a-z)"
-	test darwin != "$CI_OS_NAME" || CI_OS_NAME=osx
-	CI_REPO_SLUG="$(expr "$BUILD_REPOSITORY_URI" : '.*/\([^/]*/[^/]*\)$')"
 	CC="${CC:-gcc}"
 
 	# use a subdirectory of the cache dir (because the file share is shared
@@ -118,16 +115,13 @@ then
 	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
-	test windows_nt != "$CI_OS_NAME" ||
+	test Windows_NT != "$AGENT_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 elif test true = "$GITHUB_ACTIONS"
 then
 	CI_TYPE=github-actions
 	CI_BRANCH="$GITHUB_REF"
 	CI_COMMIT="$GITHUB_SHA"
-	CI_OS_NAME="$(echo "$RUNNER_OS" | tr A-Z a-z)"
-	test macos != "$CI_OS_NAME" || CI_OS_NAME=osx
-	CI_REPO_SLUG="$GITHUB_REPOSITORY"
 	CI_JOB_ID="$GITHUB_RUN_ID"
 	CC="${CC:-gcc}"
 	DONT_SKIP_TAGS=t
@@ -137,7 +131,7 @@ then
 	export GIT_PROVE_OPTS="--timer --jobs 10"
 	export GIT_TEST_OPTS="--verbose-log -x"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
-	test windows != "$CI_OS_NAME" ||
+	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 else
 	echo "Could not identify CI type" >&2
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 57277eefcd0..00fcf27b1b2 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -77,9 +77,6 @@ do
 			fi
 			combined_trash_size=$new_combined_trash_size
 
-			# DO NOT modify these two 'echo'-ed strings below
-			# without updating 'ci/util/extract-trash-dirs.sh'
-			# as well.
 			echo "$(tput setaf 1)Start of trash directory of '$test_name':$(tput sgr0)"
 			cat "$trash_tgz_b64"
 			echo "$(tput setaf 1)End of trash directory of '$test_name'$(tput sgr0)"
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 280dda7d285..0a1ec8c2bae 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -5,11 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-case "$CI_OS_NAME" in
-windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
-*) ln -s "$cache_dir/.prove" t/.prove;;
-esac
-
 export MAKE_TARGETS="all test"
 
 case "$jobname" in
diff --git a/ci/run-docker-build.sh b/ci/run-docker-build.sh
deleted file mode 100755
index 6cd832efb9c..00000000000
--- a/ci/run-docker-build.sh
+++ /dev/null
@@ -1,66 +0,0 @@
-#!/bin/sh
-#
-# Build and test Git inside container
-#
-# Usage:
-#   run-docker-build.sh <host-user-id>
-#
-
-set -ex
-
-if test $# -ne 1 || test -z "$1"
-then
-	echo >&2 "usage: run-docker-build.sh <host-user-id>"
-	exit 1
-fi
-
-case "$jobname" in
-linux32)
-	switch_cmd="linux32 --32bit i386"
-	;;
-linux-musl)
-	switch_cmd=
-	useradd () { adduser -D "$@"; }
-	;;
-*)
-	exit 1
-	;;
-esac
-
-"${0%/*}/install-docker-dependencies.sh"
-
-# If this script runs inside a docker container, then all commands are
-# usually executed as root. Consequently, the host user might not be
-# able to access the test output files.
-# If a non 0 host user id is given, then create a user "ci" with that
-# user id to make everything accessible to the host user.
-HOST_UID=$1
-if test $HOST_UID -eq 0
-then
-	# Just in case someone does want to run the test suite as root.
-	CI_USER=root
-else
-	CI_USER=ci
-	if test "$(id -u $CI_USER 2>/dev/null)" = $HOST_UID
-	then
-		echo "user '$CI_USER' already exists with the requested ID $HOST_UID"
-	else
-		useradd -u $HOST_UID $CI_USER
-	fi
-fi
-
-# Build and test
-command $switch_cmd su -m -l $CI_USER -c "
-	set -ex
-	export DEVELOPER='$DEVELOPER'
-	export DEFAULT_TEST_TARGET='$DEFAULT_TEST_TARGET'
-	export GIT_PROVE_OPTS='$GIT_PROVE_OPTS'
-	export GIT_TEST_OPTS='$GIT_TEST_OPTS'
-	export GIT_TEST_CLONE_2GB='$GIT_TEST_CLONE_2GB'
-	export MAKEFLAGS='$MAKEFLAGS'
-	export cache_dir='$cache_dir'
-	cd /usr/src/git
-	test -n '$cache_dir' && ln -s '$cache_dir/.prove' t/.prove
-	make
-	make test
-"
diff --git a/ci/run-docker.sh b/ci/run-docker.sh
deleted file mode 100755
index af89d1624a4..00000000000
--- a/ci/run-docker.sh
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/bin/sh
-#
-# Download and run Docker image to build and test Git
-#
-
-. ${0%/*}/lib.sh
-
-case "$jobname" in
-linux32)
-	CI_CONTAINER="daald/ubuntu32:xenial"
-	;;
-linux-musl)
-	CI_CONTAINER=alpine
-	;;
-*)
-	exit 1
-	;;
-esac
-
-docker pull "$CI_CONTAINER"
-
-# Use the following command to debug the docker build locally:
-# <host-user-id> must be 0 if podman is used as drop-in replacement for docker
-# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/sh "$CI_CONTAINER"
-# root@container:/# export jobname=<jobname>
-# root@container:/# /usr/src/git/ci/run-docker-build.sh <host-user-id>
-
-container_cache_dir=/tmp/container-cache
-
-docker run \
-	--interactive \
-	--env DEVELOPER \
-	--env DEFAULT_TEST_TARGET \
-	--env GIT_PROVE_OPTS \
-	--env GIT_TEST_OPTS \
-	--env GIT_TEST_CLONE_2GB \
-	--env MAKEFLAGS \
-	--env jobname \
-	--env cache_dir="$container_cache_dir" \
-	--volume "${PWD}:/usr/src/git" \
-	--volume "$cache_dir:$container_cache_dir" \
-	"$CI_CONTAINER" \
-	/usr/src/git/ci/run-docker-build.sh $(id -u $USER)
-
-check_unignored_build_artifacts
-
-save_good_tree
diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index f8c2c3106a2..b9a682b4bcd 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -5,11 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-case "$CI_OS_NAME" in
-windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
-*) ln -s "$cache_dir/.prove" t/.prove;;
-esac
-
 make --quiet -C t T="$(cd t &&
 	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
 	tr '\n' ' ')"
diff --git a/ci/util/extract-trash-dirs.sh b/ci/util/extract-trash-dirs.sh
deleted file mode 100755
index 8e67bec21a2..00000000000
--- a/ci/util/extract-trash-dirs.sh
+++ /dev/null
@@ -1,50 +0,0 @@
-#!/bin/sh
-
-error () {
-	echo >&2 "error: $@"
-	exit 1
-}
-
-find_embedded_trash () {
-	while read -r line
-	do
-		case "$line" in
-		*Start\ of\ trash\ directory\ of\ \'t[0-9][0-9][0-9][0-9]-*\':*)
-			test_name="${line#*\'}"
-			test_name="${test_name%\'*}"
-
-			return 0
-		esac
-	done
-
-	return 1
-}
-
-extract_embedded_trash () {
-	while read -r line
-	do
-		case "$line" in
-		*End\ of\ trash\ directory\ of\ \'$test_name\'*)
-			return
-			;;
-		*)
-			printf '%s\n' "$line"
-			;;
-		esac
-	done
-
-	error "unexpected end of input"
-}
-
-# Raw logs from Linux build jobs have CRLF line endings, while OSX
-# build jobs mostly have CRCRLF, except an odd line every now and
-# then that has CRCRCRLF.  'base64 -d' from 'coreutils' doesn't like
-# CRs and complains about "invalid input", so remove all CRs at the
-# end of lines.
-sed -e 's/\r*$//' | \
-while find_embedded_trash
-do
-	echo "Extracting trash directory of '$test_name'"
-
-	extract_embedded_trash |base64 -d |tar xzp
-done
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 04/25] CI: remove dead "tree skipping" code

[Ævar Arnfjörð Bjarmason]

Remove the code related to skipping CI runs if the tree was
successfully tested before. With the GitHub CI this is done with the
"ci-config" step, see e76eec35540 (ci: allow per-branch config for
GitHub Actions, 2020-05-07) and 7d78d5fc1a9 (ci: skip GitHub workflow
runs for already-tested commits/trees, 2020-10-08).

This code hasn't been used since 4a6e4b96026 (CI: remove Travis CI
support, 2021-11-23), and before that for the retired Azure pipeline
support removed in 6081d3898fe (ci: retire the Azure Pipelines
definition, 2020-04-11).

This change is needed because this and subsequent commits are turning
"ci/lib.sh" into a dumber library that'll only be tasked with setting
variables for CI jobs, or for specific steps within those jobs. Now we
don't need to worry about it potentially skipping the run on its own.

This change also removes a subtle potential logic error introduced in
0e7696c64db (ci: disallow directional formatting, 2021-11-04). The
"ci/check-directional-formatting.bash" script would have been made to
run after the "save_good_tree" in invoked in
"ci/run-static-analysis.sh". So if e.g. the "make coccicheck" failed
we'd still mark the tree as good.

I.e. the addition of "ci/check-directional-formatting.bash" didn't
take into account that the various users of the "save_good_tree"
function in ci/*.sh made a hard assumption that they're the only
ci/*.sh script that's being run (and they use "set -e").

This is entirely academic since we weren't actually running this code,
but is something to be careful of if anyone ever needs to resurrect
parts of this.

Finally, the cache_dir="$HOME/none" added in a3f2eec862b (ci/lib:
allow running in GitHub Actions, 2020-04-08) for the GitHub CI can be
removed. It appears to have been added to appease the subsequent
'mkdir -p "$cache_dir"', which wasn't needed in GitHub CI.

1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                 | 91 ---------------------------------------
 ci/run-build-and-tests.sh |  2 -
 ci/run-static-analysis.sh |  2 -
 ci/test-documentation.sh  |  2 -
 4 files changed, 97 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index c3c06d66862..5a8938d4b3c 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -13,75 +13,6 @@ then
 	exit 1
 fi
 
-skip_branch_tip_with_tag () {
-	# Sometimes, a branch is pushed at the same time the tag that points
-	# at the same commit as the tip of the branch is pushed, and building
-	# both at the same time is a waste.
-	#
-	# When the build is triggered by a push to a tag, $CI_BRANCH will
-	# have that tagname, e.g. v2.14.0.  Let's see if $CI_BRANCH is
-	# exactly at a tag, and if so, if it is different from $CI_BRANCH.
-	# That way, we can tell if we are building the tip of a branch that
-	# is tagged and we can skip the build because we won't be skipping a
-	# build of a tag.
-
-	if TAG=$(git describe --exact-match "$CI_BRANCH" 2>/dev/null) &&
-		test "$TAG" != "$CI_BRANCH"
-	then
-		echo "$(tput setaf 2)Tip of $CI_BRANCH is exactly at $TAG$(tput sgr0)"
-		exit 0
-	fi
-}
-
-# Save some info about the current commit's tree, so we can skip the build
-# job if we encounter the same tree again and can provide a useful info
-# message.
-save_good_tree () {
-	echo "$(git rev-parse $CI_COMMIT^{tree}) $CI_COMMIT $CI_JOB_NUMBER $CI_JOB_ID" >>"$good_trees_file"
-	# limit the file size
-	tail -1000 "$good_trees_file" >"$good_trees_file".tmp
-	mv "$good_trees_file".tmp "$good_trees_file"
-}
-
-# Skip the build job if the same tree has already been built and tested
-# successfully before (e.g. because the branch got rebased, changing only
-# the commit messages).
-skip_good_tree () {
-	if test true = "$GITHUB_ACTIONS"
-	then
-		return
-	fi
-
-	if ! good_tree_info="$(grep "^$(git rev-parse $CI_COMMIT^{tree}) " "$good_trees_file")"
-	then
-		# Haven't seen this tree yet, or no cached good trees file yet.
-		# Continue the build job.
-		return
-	fi
-
-	echo "$good_tree_info" | {
-		read tree prev_good_commit prev_good_job_number prev_good_job_id
-
-		if test "$CI_JOB_ID" = "$prev_good_job_id"
-		then
-			cat <<-EOF
-			$(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0)
-			This commit has already been built and tested successfully by this build job.
-			To force a re-build delete the branch's cache and then hit 'Restart job'.
-			EOF
-		else
-			cat <<-EOF
-			$(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0)
-			This commit's tree has already been built and tested successfully in build job $prev_good_job_number for commit $prev_good_commit.
-			The log of that build job is available at $SYSTEM_TASKDEFINITIONSURI$SYSTEM_TEAMPROJECT/_build/results?buildId=$prev_good_job_id
-			To force a re-build delete the branch's cache and then hit 'Restart job'.
-			EOF
-		fi
-	}
-
-	exit 0
-}
-
 check_unignored_build_artifacts ()
 {
 	! git ls-files --other --exclude-standard --error-unmatch \
@@ -102,16 +33,8 @@ if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
 then
 	CI_TYPE=azure-pipelines
 	# We are running in Azure Pipelines
-	CI_BRANCH="$BUILD_SOURCEBRANCH"
-	CI_COMMIT="$BUILD_SOURCEVERSION"
-	CI_JOB_ID="$BUILD_BUILDID"
-	CI_JOB_NUMBER="$BUILD_BUILDNUMBER"
 	CC="${CC:-gcc}"
 
-	# use a subdirectory of the cache dir (because the file share is shared
-	# among *all* phases)
-	cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME"
-
 	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
@@ -120,13 +43,7 @@ then
 elif test true = "$GITHUB_ACTIONS"
 then
 	CI_TYPE=github-actions
-	CI_BRANCH="$GITHUB_REF"
-	CI_COMMIT="$GITHUB_SHA"
-	CI_JOB_ID="$GITHUB_RUN_ID"
 	CC="${CC:-gcc}"
-	DONT_SKIP_TAGS=t
-
-	cache_dir="$HOME/none"
 
 	export GIT_PROVE_OPTS="--timer --jobs 10"
 	export GIT_TEST_OPTS="--verbose-log -x"
@@ -139,14 +56,6 @@ else
 	exit 1
 fi
 
-good_trees_file="$cache_dir/good-trees"
-
-mkdir -p "$cache_dir"
-
-test -n "${DONT_SKIP_TAGS-}" ||
-skip_branch_tip_with_tag
-skip_good_tree
-
 export DEVELOPER=1
 export DEFAULT_TEST_TARGET=prove
 export GIT_TEST_CLONE_2GB=true
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 0a1ec8c2bae..e5a21985b44 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -45,5 +45,3 @@ esac
 # start running tests.
 make $MAKE_TARGETS
 check_unignored_build_artifacts
-
-save_good_tree
diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh
index 65bcebda41a..5a87b1cac96 100755
--- a/ci/run-static-analysis.sh
+++ b/ci/run-static-analysis.sh
@@ -28,5 +28,3 @@ fi
 
 make hdr-check ||
 exit 1
-
-save_good_tree
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index de41888430a..9e0652c30dd 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -41,5 +41,3 @@ grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
 
 rm -f stdout.log stderr.log stderr.raw
 check_unignored_build_artifacts
-
-save_good_tree
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 05/25] CI: remove unused Azure ci/* code

[Ævar Arnfjörð Bjarmason]

Remove Azure-specific code that's been unused since 6081d3898fe (ci:
retire the Azure Pipelines definition, 2020-04-11). As noted in a
larger removal of all of the Azure-supporting code in [1] (although
that missed some of this) there is more of it in-tree, but let's focus
on only the ci/* code for now.

This is needed because in subsequent commits this unused code would
either need to be changed to accommodate changes in ci/*, or be
removed.

As we'll see in those subsequent commits the end-state we're heading
towards will actually make it easier to add new CI types in the
future, even though the only one we're left with now is the GitHub
CI. I.e. the "ci/*" framework will be made to do less, not more. We'll
be offloading more of what it does to our generic build and test
system.

While I'm at it (since the line needs to be touched anyway) change an
odd 'if test true == "$GITHUB_ACTIONS"' to the more usual style used
in other places of 'if test "$GITHUB_ACTIONS" = "true"'.

1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/install-dependencies.sh |  3 ---
 ci/lib.sh                  | 13 +------------
 ci/mount-fileshare.sh      | 25 -------------------------
 ci/print-test-failures.sh  |  7 +------
 4 files changed, 2 insertions(+), 46 deletions(-)
 delete mode 100755 ci/mount-fileshare.sh

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index dbcebad2fb2..e7ea8799411 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -34,8 +34,6 @@ macos-latest)
 	export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
 	# Uncomment this if you want to run perf tests:
 	# brew install gnu-time
-	test -z "$BREW_INSTALL_PACKAGES" ||
-	brew install $BREW_INSTALL_PACKAGES
 	brew link --force gettext
 	brew install --cask --no-quarantine perforce || {
 		# Update the definitions and try again
@@ -69,7 +67,6 @@ Documentation)
 	sudo apt-get -q update
 	sudo apt-get -q -y install asciidoc xmlto docbook-xsl-ns make
 
-	test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
 	sudo gem install --version 1.5.8 asciidoctor
 	;;
 linux-gcc-default)
diff --git a/ci/lib.sh b/ci/lib.sh
index 5a8938d4b3c..ad1aa59a384 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -29,18 +29,7 @@ export TERM=${TERM:-dumb}
 # Clear MAKEFLAGS that may come from the outside world.
 export MAKEFLAGS=
 
-if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
-then
-	CI_TYPE=azure-pipelines
-	# We are running in Azure Pipelines
-	CC="${CC:-gcc}"
-
-	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
-	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
-	MAKEFLAGS="$MAKEFLAGS --jobs=10"
-	test Windows_NT != "$AGENT_OS" ||
-	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
-elif test true = "$GITHUB_ACTIONS"
+if test "$GITHUB_ACTIONS" = "true"
 then
 	CI_TYPE=github-actions
 	CC="${CC:-gcc}"
diff --git a/ci/mount-fileshare.sh b/ci/mount-fileshare.sh
deleted file mode 100755
index 26b58a80960..00000000000
--- a/ci/mount-fileshare.sh
+++ /dev/null
@@ -1,25 +0,0 @@
-#!/bin/sh
-
-die () {
-	echo "$*" >&2
-	exit 1
-}
-
-test $# = 4 ||
-die "Usage: $0 <share> <username> <password> <mountpoint>"
-
-mkdir -p "$4" || die "Could not create $4"
-
-case "$(uname -s)" in
-Linux)
-	sudo mount -t cifs -o vers=3.0,username="$2",password="$3",dir_mode=0777,file_mode=0777,serverino "$1" "$4"
-	;;
-Darwin)
-	pass="$(echo "$3" | sed -e 's/\//%2F/g' -e 's/+/%2B/g')" &&
-	mount -t smbfs,soft "smb://$2:$pass@${1#//}" "$4"
-	;;
-*)
-	die "No support for $(uname -s)"
-	;;
-esac ||
-die "Could not mount $4"
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 00fcf27b1b2..0c63b6f7962 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -38,14 +38,9 @@ do
 		test_name="${TEST_EXIT%.exit}"
 		test_name="${test_name##*/}"
 		trash_dir="trash directory.$test_name"
+		mkdir -p failed-test-artifacts
 		case "$CI_TYPE" in
-		azure-pipelines)
-			mkdir -p failed-test-artifacts
-			mv "$trash_dir" failed-test-artifacts
-			continue
-			;;
 		github-actions)
-			mkdir -p failed-test-artifacts
 			echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
 			cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
 			tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
-- 
2.35.1.1132.ga1fe46f8690

Re: [PATCH 05/25] CI: remove unused Azure ci/* code

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1700 bytes --]

Hi,


On Mon, 21 Feb 2022, Ævar Arnfjörð Bjarmason wrote:

> Remove Azure-specific code that's been unused since 6081d3898fe (ci:
> retire the Azure Pipelines definition, 2020-04-11). As noted in a
> larger removal of all of the Azure-supporting code in [1] (although
> that missed some of this) there is more of it in-tree, but let's focus
> on only the ci/* code for now.
>
> This is needed because in subsequent commits this unused code would
> either need to be changed to accommodate changes in ci/*, or be
> removed.
>
> As we'll see in those subsequent commits the end-state we're heading
> towards will actually make it easier to add new CI types in the
> future, even though the only one we're left with now is the GitHub
> CI. I.e. the "ci/*" framework will be made to do less, not more. We'll
> be offloading more of what it does to our generic build and test
> system.
>
> While I'm at it (since the line needs to be touched anyway) change an
> odd 'if test true == "$GITHUB_ACTIONS"' to the more usual style used
> in other places of 'if test "$GITHUB_ACTIONS" = "true"'.
>
> 1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/

This has been discussed before, and I already gave my NAK.

It is sad that I have to repeat myself: it is a good thing to have the
Azure Pipelines definition as a fall-back. In the past, this has served us
very well especially when we had to run a barrage of security fixes, for a
slew of backports to previous release trains.

You seem to have fun to just remove this code, under some assumption that
it is not needed, despite me pointing out that it is needed.

Ciao,
Johannes

Re: [PATCH 05/25] CI: remove unused Azure ci/* code

[Ævar Arnfjörð Bjarmason]

On Tue, Feb 22 2022, Johannes Schindelin wrote:

> Hi,
>
>
> On Mon, 21 Feb 2022, Ævar Arnfjörð Bjarmason wrote:
>
>> Remove Azure-specific code that's been unused since 6081d3898fe (ci:
>> retire the Azure Pipelines definition, 2020-04-11). As noted in a
>> larger removal of all of the Azure-supporting code in [1] (although
>> that missed some of this) there is more of it in-tree, but let's focus
>> on only the ci/* code for now.
>>
>> This is needed because in subsequent commits this unused code would
>> either need to be changed to accommodate changes in ci/*, or be
>> removed.
>>
>> As we'll see in those subsequent commits the end-state we're heading
>> towards will actually make it easier to add new CI types in the
>> future, even though the only one we're left with now is the GitHub
>> CI. I.e. the "ci/*" framework will be made to do less, not more. We'll
>> be offloading more of what it does to our generic build and test
>> system.
>>
>> While I'm at it (since the line needs to be touched anyway) change an
>> odd 'if test true == "$GITHUB_ACTIONS"' to the more usual style used
>> in other places of 'if test "$GITHUB_ACTIONS" = "true"'.
>>
>> 1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/
>
> This has been discussed before, and I already gave my NAK.
>
> It is sad that I have to repeat myself: it is a good thing to have the
> Azure Pipelines definition as a fall-back. In the past, this has served us
> very well especially when we had to run a barrage of security fixes, for a
> slew of backports to previous release trains.
>
> You seem to have fun to just remove this code, under some assumption that
> it is not needed, despite me pointing out that it is needed.

I previously submitted a stand-alone patch to remove it[1] after the
removal of (most of) the Travis CI was merged in f9b889dd67b (Merge
branch 'ab/ci-updates', 2021-12-15).

I submitted that not for "fun", but because I had other CI/test-lib.sh
changes depending on that.

I.e. I needed to either patch code unused in-tree I couldn't test
without reverting your 6081d3898fe (ci: retire the Azure Pipelines
definition, 2020-04-11) (and presumably set up some Azure CI account
etc.), or remove it.

It's still unclear to me how Azure CI is being used as a "fall-back",
can you explain that? I.e. we don't have the azure-pipelines.yml anymore
(and neither does git-for-windows/git). Is this in-tree code being used
by some out-of-tree fork of git.git, or do you mean you'd like to keep
it in case we find a reason to revert your 6081d3898fe?

In this case (and as I think this series makes clear), we can't easily
keep certain Travis + Azure assumptions around *and* simplify how
ci/lib.sh works in the same way. At the tip of this series it's turned
into a very dumb variable setting helper with the CI recipe itself
driving the test.

Whereas Travis and Azure were using a feature where a "job" once running
would attempt to skip the rest of the CI run.

And in your [1] you mentioned the reasons for keeping it around being:
    
    The reason is that there are still some things that Azure Pipelines can do
    that GitHub workflows cannot, for example:
    
    - present the logs of failed tests in an intuitive manner,
    
    - re-run _only_ failed jobs.

It seems to me that in your parallel series you're working on 1/2 of
those, and with/without those changes we could otherwise improve the
presentation of the failed test runs enough with the "grouping"
etc. feature.

And as for 2/2 that any such support would be mostly orthogonal to
keeping Azure code that's currently unused around. I.e. a goal of this
series is also to make the CI less of a special snowflake, because I'm
not only interested in running the CI code on GitHub CI, but also
locally.

Which means that for any future port to Azure, GitLab CI etc. we'd
presumably just have a thin recipe that ran "make" followed by "make
test", and for a re-run of only that job it would just use the CI itself
to do that, without us needing to carry any special-cases in ci/*.

I'm not at all opposed to keeping Azure support in-tree. I specifically
have a problem with unused code holding other improvements
hostage.

I.e. neither I nor anyone else can easily change anything surrounding it
while being assured that we haven't broken that unused code. As noted in
04/25 I think you've also had that issue, i.e. your recent 0e7696c64db
(it seems to me) introduced what would have been a bug if it were
combined with the azure-pipelines.yml code removed in your 6081d3898fe.

1. https://lore.kernel.org/git/nycvar.QRO.7.76.6.2112201834050.347@tvgsbejvaqbjf.bet/

Re: [PATCH 05/25] CI: remove unused Azure ci/* code

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 272 bytes --]

Hi Ævar,

On Tue, 22 Feb 2022, Ævar Arnfjörð Bjarmason wrote:

> It's still unclear to me how Azure CI is being used as a "fall-back",
> can you explain that?

By reinstating the `azure-pipelines.yml` file from the last known-good
version.

Ciao,
Johannes

Re: [PATCH 05/25] CI: remove unused Azure ci/* code

[Ævar Arnfjörð Bjarmason]

On Tue, Feb 22 2022, Johannes Schindelin wrote:

> Hi Ævar,
>
> On Tue, 22 Feb 2022, Ævar Arnfjörð Bjarmason wrote:
>
>> It's still unclear to me how Azure CI is being used as a "fall-back",
>> can you explain that?
>
> By reinstating the `azure-pipelines.yml` file from the last known-good
> version.

Okey, despite what you might thing I'm not on the war path to remove all
of this at all costs.

In particular I don't mind much keeping the 05/25 parts around, but the
04/25 would be a hassle, depending on how you reply to the below.

So what do you expect me or someone who overtly touches code related to
this & the similar (and related) in-tree JUnit support to do? Presumably
one of:

 1. Don't work on any such code at all, in case you'd one day like to
    resurrect Azure support.

 2. Test my patches locally by reverting azure-pipelines.yml and make
    sure any overt changes to Azure-related code still work with that
    reverted commit.

 3. Just "YOLO hack it" but leave it in place-as is. E.g. for 04/25 and
    05/25 in combination with 11/25 (the later s/export/setenv/g change)
    leave a known-bad-but-looking-like-it-was-before Azure branch in-place.

 4. A variant of #2 where I attempt to patch the Azure code branches, but
    the "testing" is just eyeballing that they look reasonable, but I haven't
    *really* tested them even once (and I'd note as much in a commit message).

I'm not willing to go for #1 and #2. I could do #3 or #4 if Junio/others
chime and agree with you, but I really don't think those are worth it
either.

I could understand your view in your reply back in December when I sent
the stand-alone Azure removal patch[1]. Even though I noted that it was
needed for subsequent changes it was a stand-alone patch, so it wasn't
easy to evaluate the trade-off of whether removing it was worth it.

I think here it is quite easy. The end-state of this series
significantly improves the UX for the CI we actually use, and I think
actually makes it easier to get Azure support back up & running should
you ever want that, since the whole structure of it is making CI less
complex and less of a special-case.

It's been almost 2 years (just around 2 months short of it..) since
azure-pipelines.yml was removed from "master". It would really be quite
easy to get it or any other new CI target off the ground, particularly
after this series.

Insisting that any effort to fix actual CI issues in the actual CI we do
use needs to be hamstrung by any change in the area needing to carefully
eyeball:

    git show 6081d3898fe^:azure-pipelines.yml

And for each change carefully consider them *if* we still ran that just
seems unreasonably obstructionist.

Sure, Azure CI had some neat features, so did Travis CI. If we ever want
to run either of them again let's just consider that if and when that
happens.

Maybe you have relevant feedback queued up, but so far you haven't had
any meaningful comments on the goals end end-state of this series as
compared to yours[2] (to the extent that their approaches differ).

Can we try to focus on that instead? I.e. the actual visible CI changes
that'll either make the CI workflow we actually use better or worse?

1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/
2. https://lore.kernel.org/git/pull.1117.git.1643050574.gitgitgadget@gmail.com/

[PATCH 06/25] CI: don't have "git grep" invoke a pager in tree content check

[Ævar Arnfjörð Bjarmason]

Fix the omission of not adding "-P" (a.k.a. "--no-pager") to the "git"
invocation that spaws "git grep", added in 0e7696c64db (ci: disallow
directional formatting, 2021-11-04).

This didn't matter for the GitHub CI, but does when running the check
locally. A subsequent commit will start invoking it via a new check
from the top-level Makefile, and without this we'd visibly spawn a
pager when doing so.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/check-directional-formatting.bash | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ci/check-directional-formatting.bash b/ci/check-directional-formatting.bash
index e6211b141aa..de8e6e486f0 100755
--- a/ci/check-directional-formatting.bash
+++ b/ci/check-directional-formatting.bash
@@ -23,5 +23,5 @@
 # U+2066..U+2069: LRI, RLI, FSI and PDI
 regex='(\u202a|\u202b|\u202c|\u202d|\u202e|\u2066|\u2067|\u2068|\u2069)'
 
-! LC_CTYPE=C git grep -El "$(LC_CTYPE=C.UTF-8 printf "$regex")" \
+! LC_CTYPE=C git -P grep -El "$(LC_CTYPE=C.UTF-8 printf "$regex")" \
 	-- ':(exclude,attr:binary)' ':(exclude)*.po'
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 07/25] CI: have "static-analysis" run a "make ci-static-analysis" target

[Ævar Arnfjörð Bjarmason]

Make it easier to run the CI logic locally by creating
"ci-static-analysis" target.

Now the "check-coccicheck" logic added in 0860a7641ba (travis-ci: fail
if Coccinelle static analysis found something to transform,
2018-07-23) can be easily tested outside of CI, and likewise the
sanity check added in 0e7696c64db (ci: disallow directional
formatting, 2021-11-04).

These targets could be improved, the "hdr-check" target will re-do all
of its work every time it's run, and could with a change similar to my
c234e8a0ecf (Makefile: make the "sparse" target non-.PHONY,
2021-09-23). The same goes for the new
"ci-check-directional-formatting" target.

But without those improvements being able to easily run a 1=1
equivalent of the checks we do in CI during a local build is already
an improvement. Subsequent commits will expand on this ability.

This change will also make the CI check faster, since we can take
advantage of parallelism. The "make coccicheck" takes a very long
time, and at the tail end we might only have one job pegging 100% of
one CPU core for a long time. Now any extra cores will be free to run
the rest of the targets under "ci-static-analysis".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml |  3 +--
 Makefile                   | 29 +++++++++++++++++++++++++++++
 ci/run-static-analysis.sh  | 30 ------------------------------
 3 files changed, 30 insertions(+), 32 deletions(-)
 delete mode 100755 ci/run-static-analysis.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 309b9141249..ad752010102 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -317,8 +317,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/run-static-analysis.sh
-    - run: ci/check-directional-formatting.bash
+    - run: make ci-static-analysis
   sparse:
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
diff --git a/Makefile b/Makefile
index 6f0b4b775fe..02cd563e216 100644
--- a/Makefile
+++ b/Makefile
@@ -2005,6 +2005,7 @@ ifndef V
 	QUIET_HDR      = @echo '   ' HDR $(<:hcc=h);
 	QUIET_RC       = @echo '   ' RC $@;
 	QUIET_SPATCH   = @echo '   ' SPATCH $<;
+	QUIET_CHECK    = @echo '   ' CHECK $@;
 	QUIET_SUBDIR0  = +@subdir=
 	QUIET_SUBDIR1  = ;$(NO_SUBDIR) echo '   ' SUBDIR $$subdir; \
 			 $(MAKE) $(PRINT_DIR) -C $$subdir
@@ -3052,6 +3053,20 @@ coccicheck: $(addsuffix .patch,$(filter-out %.pending.cocci,$(wildcard contrib/c
 # See contrib/coccinelle/README
 coccicheck-pending: $(addsuffix .patch,$(wildcard contrib/coccinelle/*.pending.cocci))
 
+.PHONY: check-coccicheck
+check-coccicheck: coccicheck
+	$(QUIET_CHECK)for cocci_patch in contrib/coccinelle/*.patch; do \
+		if test -s "$$cocci_patch"; then \
+			echo "Coccinelle suggests the following changes in '$$cocci_patch':"; \
+			cat "$$cocci_patch"; \
+			fail=UnfortunatelyYes; \
+		fi \
+	done; \
+	if test -n "$$fail"; then \
+		echo "error: Coccinelle suggested some changes"; \
+		exit 1; \
+	fi
+
 .PHONY: coccicheck coccicheck-pending
 
 ### Installation rules
@@ -3487,3 +3502,17 @@ $(FUZZ_PROGRAMS): all
 		$(XDIFF_OBJS) $(EXTLIBS) git.o $@.o $(LIB_FUZZING_ENGINE) -o $@
 
 fuzz-all: $(FUZZ_PROGRAMS)
+
+### CI "check" targets
+#
+# These targets are run from the CI, see .github/workflows/main.yml,
+# but can also be run manually to run the same assertions locally.
+
+.PHONY: ci-check-directional-formatting
+ci-check-directional-formatting:
+	$(QUIET_CHECK)ci/check-directional-formatting.bash
+
+.PHONY: ci-static-analysis
+ci-static-analysis: ci-check-directional-formatting
+ci-static-analysis: check-coccicheck
+ci-static-analysis: hdr-check
diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh
deleted file mode 100755
index 5a87b1cac96..00000000000
--- a/ci/run-static-analysis.sh
+++ /dev/null
@@ -1,30 +0,0 @@
-#!/bin/sh
-#
-# Perform various static code analysis checks
-#
-
-. ${0%/*}/lib.sh
-
-make coccicheck
-
-set +x
-
-fail=
-for cocci_patch in contrib/coccinelle/*.patch
-do
-	if test -s "$cocci_patch"
-	then
-		echo "$(tput setaf 1)Coccinelle suggests the following changes in '$cocci_patch':$(tput sgr0)"
-		cat "$cocci_patch"
-		fail=UnfortunatelyYes
-	fi
-done
-
-if test -n "$fail"
-then
-	echo "$(tput setaf 1)error: Coccinelle suggested some changes$(tput sgr0)"
-	exit 1
-fi
-
-make hdr-check ||
-exit 1
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 08/25] CI: have "static-analysis" run "check-builtins", not "documentation"

[Ævar Arnfjörð Bjarmason]

Move the "make check-builtins" check from the "documentation" job to
the "static-analysis" job.

The "check-builtins" target added in c74390e4a1d (cherry is built-in,
do not ship git-cherry.sh, 2006-11-05) is unrelated to the
documentation, so it's odd that b98712b9aa9 (travis-ci: build
documentation, 2016-05-04) added it to the "documentation" job.

Let's just move it to the "ci-static-analysis" target, and while we're
at it improve the output with $(QUIET_CHECK).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile                 | 3 ++-
 ci/test-documentation.sh | 1 -
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 02cd563e216..0c117d32f48 100644
--- a/Makefile
+++ b/Makefile
@@ -3424,7 +3424,7 @@ check-docs::
 ### Make sure built-ins do not have dups and listed in git.c
 #
 check-builtins::
-	./check-builtins.sh
+	$(QUIET_CHECK)./check-builtins.sh
 
 ### Test suite coverage testing
 #
@@ -3514,5 +3514,6 @@ ci-check-directional-formatting:
 
 .PHONY: ci-static-analysis
 ci-static-analysis: ci-check-directional-formatting
+ci-static-analysis: check-builtins
 ci-static-analysis: check-coccicheck
 ci-static-analysis: hdr-check
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 9e0652c30dd..41e2b126311 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -14,7 +14,6 @@ filter_log () {
 	    "$1"
 }
 
-make check-builtins
 make check-docs
 
 # Build docs with AsciiDoc
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 09/25] CI: move p4 and git-lfs variables to ci/install-dependencies.sh

[Ævar Arnfjörð Bjarmason]

Move the declaration of variables that are only used by the
"ubuntu-latest" block in "ci/install-dependencies.sh" there from
"ci/lib.sh".

This makes the code easier to follow, and changes "ci/lib.sh" to a
library that only exports CI variables for general use, we didn't need
to export these $P4_PATH and $GIT_LFS_PATH variables.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/install-dependencies.sh | 17 +++++++++++++++--
 ci/lib.sh                  | 12 ------------
 2 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index e7ea8799411..34b4400f38b 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -5,14 +5,27 @@
 
 . ${0%/*}/lib.sh
 
-P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
-LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
 UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
  tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
  libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl"
 
 case "$runs_on_pool" in
 ubuntu-latest)
+	# The Linux build installs the defined dependency versions below.
+	# The OS X build installs much more recent versions, whichever
+	# were recorded in the Homebrew database upon creating the OS X
+	# image.
+	# Keep that in mind when you encounter a broken OS X build!
+	LINUX_P4_VERSION="16.2"
+	LINUX_GIT_LFS_VERSION="1.5.2"
+
+	P4_PATH="$HOME/custom/p4"
+	GIT_LFS_PATH="$HOME/custom/git-lfs"
+	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
+
+	P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
+	LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
+
 	sudo apt-get -q update
 	sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \
 		$UBUNTU_COMMON_PKGS $CC_PACKAGE
diff --git a/ci/lib.sh b/ci/lib.sh
index ad1aa59a384..6b37b10d750 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -65,18 +65,6 @@ ubuntu-latest)
 	fi
 
 	export GIT_TEST_HTTPD=true
-
-	# The Linux build installs the defined dependency versions below.
-	# The OS X build installs much more recent versions, whichever
-	# were recorded in the Homebrew database upon creating the OS X
-	# image.
-	# Keep that in mind when you encounter a broken OS X build!
-	export LINUX_P4_VERSION="16.2"
-	export LINUX_GIT_LFS_VERSION="1.5.2"
-
-	P4_PATH="$HOME/custom/p4"
-	GIT_LFS_PATH="$HOME/custom/git-lfs"
-	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
 	;;
 macos-latest)
 	if [ "$jobname" = osx-gcc ]
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 10/25] CI: consistently use "export" in ci/lib.sh

[Ævar Arnfjörð Bjarmason]

Change the "ci/lib.sh" script co consistently use "export", for
e.g. MAKEFLAGS we were exporting it, and then assigning to it, let's
do it the other way around.

Right now this doesn't matter, since we in
e.g. "ci/install-dependencies.sh" source this file, and don't use
something like "env(1)" to retrieve these variables. But in a
subsequent commit we'll "export" these variables through a wrapper (to
additionally write them to a GitHub CI-specific $GITHUB_ENV
file). This change makes that subsequent change easier to read, as it
won't need to do any control flow refactoring.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 6b37b10d750..31fe3a476a9 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -27,7 +27,7 @@ check_unignored_build_artifacts ()
 export TERM=${TERM:-dumb}
 
 # Clear MAKEFLAGS that may come from the outside world.
-export MAKEFLAGS=
+MAKEFLAGS=
 
 if test "$GITHUB_ACTIONS" = "true"
 then
@@ -35,10 +35,12 @@ then
 	CC="${CC:-gcc}"
 
 	export GIT_PROVE_OPTS="--timer --jobs 10"
-	export GIT_TEST_OPTS="--verbose-log -x"
+	GIT_TEST_OPTS="--verbose-log -x"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
+
+	export GIT_TEST_OPTS
 else
 	echo "Could not identify CI type" >&2
 	env >&2
@@ -92,4 +94,4 @@ linux-leaks)
 	;;
 esac
 
-MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
+export MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 11/25] CI: export variables via a wrapper

[Ævar Arnfjörð Bjarmason]

Add and use a "setenv" wrapper to export the variables set in
ci/lib.sh.

For now this merely adds debugging information about what variables
are set, but in a subsequent commit we'll expand this to set variables
in $GITHUB_ENV instead of the environment, and thus carry variable
settings between steps in a job.

We'll also eventually be interested in what type of variable we're
setting, and when it needs to be used, because we'll prominently
expose what variables we set in what CI "step", so having only the
ones we actually use will improve readability.

To that end call the "setenv" function with optional "--build",
"--test" and "--all" parameters to indicate whether they're needed for
building, testing or both.

We currently ignore this parameter, but will make use of it in a
subsequent commit. Adding it here reduces the needless churn, as we
won't need to change all these lines again for that subsequent change.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                 | 39 ++++++++++++++++++++++++++++-----------
 ci/run-build-and-tests.sh | 34 +++++++++++++++++-----------------
 2 files changed, 45 insertions(+), 28 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 31fe3a476a9..1a1fa0e4bb2 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -13,6 +13,23 @@ then
 	exit 1
 fi
 
+# Helper functions
+setenv () {
+	varmode=
+	case "$1" in
+	--*)
+		varmode=$1
+		shift
+		;;
+	esac
+
+	key=$1
+	val=$2
+	shift 2
+
+	eval "export $key=\"$val\""
+}
+
 check_unignored_build_artifacts ()
 {
 	! git ls-files --other --exclude-standard --error-unmatch \
@@ -24,7 +41,7 @@ check_unignored_build_artifacts ()
 }
 
 # GitHub Action doesn't set TERM, which is required by tput
-export TERM=${TERM:-dumb}
+setenv TERM ${TERM:-dumb}
 
 # Clear MAKEFLAGS that may come from the outside world.
 MAKEFLAGS=
@@ -34,23 +51,23 @@ then
 	CI_TYPE=github-actions
 	CC="${CC:-gcc}"
 
-	export GIT_PROVE_OPTS="--timer --jobs 10"
+	setenv --test GIT_PROVE_OPTS "--timer --jobs 10"
 	GIT_TEST_OPTS="--verbose-log -x"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 
-	export GIT_TEST_OPTS
+	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
 else
 	echo "Could not identify CI type" >&2
 	env >&2
 	exit 1
 fi
 
-export DEVELOPER=1
-export DEFAULT_TEST_TARGET=prove
-export GIT_TEST_CLONE_2GB=true
-export SKIP_DASHED_BUILT_INS=YesPlease
+setenv --build DEVELOPER 1
+setenv --test DEFAULT_TEST_TARGET prove
+setenv --test GIT_TEST_CLONE_2GB true
+setenv --build SKIP_DASHED_BUILT_INS YesPlease
 
 case "$runs_on_pool" in
 ubuntu-latest)
@@ -66,7 +83,7 @@ ubuntu-latest)
 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python2"
 	fi
 
-	export GIT_TEST_HTTPD=true
+	setenv --test GIT_TEST_HTTPD true
 	;;
 macos-latest)
 	if [ "$jobname" = osx-gcc ]
@@ -89,9 +106,9 @@ linux-musl)
 	MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8"
 	;;
 linux-leaks)
-	export SANITIZE=leak
-	export GIT_TEST_PASSING_SANITIZE_LEAK=true
+	setenv --build SANITIZE leak
+	setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true
 	;;
 esac
 
-export MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
+setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index e5a21985b44..35d45a9373e 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -9,33 +9,33 @@ export MAKE_TARGETS="all test"
 
 case "$jobname" in
 linux-gcc)
-	export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
 	;;
 linux-TEST-vars)
-	export GIT_TEST_SPLIT_INDEX=yes
-	export GIT_TEST_MERGE_ALGORITHM=recursive
-	export GIT_TEST_FULL_IN_PACK_ARRAY=true
-	export GIT_TEST_OE_SIZE=10
-	export GIT_TEST_OE_DELTA_SIZE=5
-	export GIT_TEST_COMMIT_GRAPH=1
-	export GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS=1
-	export GIT_TEST_MULTI_PACK_INDEX=1
-	export GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP=1
-	export GIT_TEST_ADD_I_USE_BUILTIN=1
-	export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master
-	export GIT_TEST_WRITE_REV_INDEX=1
-	export GIT_TEST_CHECKOUT_WORKERS=2
+	setenv --test GIT_TEST_SPLIT_INDEX yes
+	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
+	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
+	setenv --test GIT_TEST_OE_SIZE 10
+	setenv --test GIT_TEST_OE_DELTA_SIZE 5
+	setenv --test GIT_TEST_COMMIT_GRAPH 1
+	setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
+	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
+	setenv --test GIT_TEST_WRITE_REV_INDEX 1
+	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
 	;;
 linux-clang)
-	export GIT_TEST_DEFAULT_HASH=sha1
+	setenv --test GIT_TEST_DEFAULT_HASH sha1
 	;;
 linux-sha256)
-	export GIT_TEST_DEFAULT_HASH=sha256
+	setenv --test GIT_TEST_DEFAULT_HASH sha256
 	;;
 pedantic)
 	# Don't run the tests; we only care about whether Git can be
 	# built.
-	export DEVOPTS=pedantic
+	setenv --build DEVOPTS pedantic
 	export MAKE_TARGETS=all
 	;;
 esac
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 12/25] CI: remove "run-build-and-tests.sh", run "make [test]" directly

[Ævar Arnfjörð Bjarmason]

Remove the already thin "ci/run-build-and-tests.sh" wrapper and
instead make the CI run "make" or "make test" directly. By doing this
we'll be able to easily see at a glance whether our failure was in the
compilation or testing, whether that's via human eyes or improve
machine readability.

We also need to run our new "ci/check-unignored-build-artifacts.sh" on
success() in the CI now, just like we already had a step conditional
on failure() running ci/print-test-failures.sh.

The reason we used a "ci/run-build-and-tests.sh" wrapper in the first
place had to do with Travis CI-specific constraints that no longer
apply to us, as the Travis CI support has been removed.

Instead we can configure the CI in an earlier step by running
"ci/lib.sh", which under GitHub CI will write the environment
variables we need to the "$GITHUB_ENV" file.

We'll then have access to them in subsequent steps, and crucially
those variables will be prominently visible at the start of each step
via an expandable drop-down in the UI.drop-do.

I.e. this changes the CI run from a top-down flow like (pseudocode):

 - job:
   - step1:
     - use ci/lib.sh to set env vars
     - run a script like ci/run-build-and-tests.sh
   - step2:
     - if: failure()
     - use ci/lib.sh to set env vars
     - run ci/print-test-failures.sh

To:

 - job:
   - step1:
     - set variables in $GITHUB_ENV using ci/lib.sh
   - step2:
     - make
   - step3:
     - make test
   - step4:
     - if: failure()
     - run ci/print-test-failures.sh
   - step5:
     - if: success()
     - run ci/check-unignored-build-artifacts.sh

There is a proposal[2] to get some of the benefits of this approach by
not re-arranging our variable setup in this way, but to instead use
the GitHub CI grouping syntax to focus on the relevant parts of "make"
or "make test" when we have failures.

Doing it this way makes for better looking GitHub CI UI, and lays much
better ground work for our CI going forward. Because:

 * The CI logic will be more portable to a future CI system, since a
   common feature of them is to run various commands in sequence, but
   a future system won't necessarily support the GitHub-specifics
   syntax of "grouping" output within a "step".

   Even if those systems don't support a "$GITHUB_ENV" emulating will
   be much easier than to deal with some CI-specific grouping syntax.

 * At the start of every step the GitHub CI presents an expandable
   list of environment variables from "$GITHUB_ENV". We'll now see
   exactly what variables affected that step (although we currently
   overshoot that a bit, and always define all variables).

 * CI failures will be easier to reproduce locally, as this makes the
   relevant ci/* scripts something that sets up our environment, but
   leaves "make" and "make test" working as they do locally.

   To reproduce a run the user only needs to set the variables
   discussed in the drop-down above, either manually or by running
   "ci/lib.sh".

 * The output will be less verbose. The "ci/lib.sh" script uses "set
   -x", and before this e.g. "ci/static-analysis.sh" would start with
   40 lines of trace output, culminating in using "export" to export
   the relevant environment variables.

   Now that verbosity is in the earlier "ci/lib.sh" step, and not in
   any subsequent one. The "make" targets then start out with the
   relevant output non-trace output right away.

 * If we do want to use the grouping syntax within a "step" it'll now
   be easier to do so. It doesn't support nesting, so we'd have to
   make a choice between using it for e.g. "make" v.s. "make test", or
   individual test failures. See "sadly" in [3].

1. https://lore.kernel.org/git/211120.86k0h30zuw.gmgdl@evledraar.gmail.com/
2. https://lore.kernel.org/git/pull.1117.git.1643050574.gitgitgadget@gmail.com/
3. https://lore.kernel.org/git/9333ba781b8240f704e739b00d274f8c3d887e39.1643050574.git.gitgitgadget@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml            | 14 ++++++--
 ci/check-unignored-build-artifacts.sh |  8 +++++
 ci/install-dependencies.sh            |  4 +++
 ci/lib.sh                             | 40 ++++++++++++++++++++++-
 ci/make-test-artifacts.sh             |  2 --
 ci/run-build-and-tests.sh             | 47 ---------------------------
 ci/run-test-slice.sh                  |  2 --
 ci/test-documentation.sh              |  1 -
 8 files changed, 62 insertions(+), 56 deletions(-)
 create mode 100755 ci/check-unignored-build-artifacts.sh
 delete mode 100755 ci/run-build-and-tests.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index ad752010102..c83f6a21ca3 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -268,7 +268,10 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/run-build-and-tests.sh
+    - run: ci/lib.sh
+    - run: make
+    - run: make test
+      if: success()
     - run: ci/print-test-failures.sh
       if: failure()
     - name: Upload failed tests' directories
@@ -292,6 +295,7 @@ jobs:
           image: daald/ubuntu32:xenial
         - jobname: pedantic
           image: fedora
+          skip-tests: no
     env:
       jobname: ${{matrix.vector.jobname}}
     runs-on: ubuntu-latest
@@ -299,9 +303,12 @@ jobs:
     steps:
     - uses: actions/checkout@v1
     - run: ci/install-docker-dependencies.sh
-    - run: ci/run-build-and-tests.sh
+    - run: ci/lib.sh
+    - run: make
+    - run: make test
+      if: success() && matrix.vector.skip-tests != 'no'
     - run: ci/print-test-failures.sh
-      if: failure()
+      if: failure() && matrix.vector.skip-tests != 'no'
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v1
@@ -317,6 +324,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
+    - run: ci/lib.sh
     - run: make ci-static-analysis
   sparse:
     needs: ci-config
diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh
new file mode 100755
index 00000000000..56d04b0db9a
--- /dev/null
+++ b/ci/check-unignored-build-artifacts.sh
@@ -0,0 +1,8 @@
+#!/bin/sh
+#
+# Check whether the build created anything not in our .gitignore
+#
+
+. ${0%/*}/lib.sh
+
+check_unignored_build_artifacts
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 34b4400f38b..01006379fd0 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -22,6 +22,10 @@ ubuntu-latest)
 	P4_PATH="$HOME/custom/p4"
 	GIT_LFS_PATH="$HOME/custom/git-lfs"
 	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
+	if test -n "$GITHUB_PATH"
+	then
+		echo "$PATH" >>"$GITHUB_PATH"
+	fi
 
 	P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
 	LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
diff --git a/ci/lib.sh b/ci/lib.sh
index 1a1fa0e4bb2..05f3dd15e27 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,3 +1,5 @@
+#!/bin/sh
+
 # Library of functions shared by all CI scripts
 
 # Set 'exit on error' for all CI scripts to let the caller know that
@@ -27,7 +29,14 @@ setenv () {
 	val=$2
 	shift 2
 
-	eval "export $key=\"$val\""
+	if test -n "$GITHUB_ENV"
+	then
+		echo "$key=$val" >>"$GITHUB_ENV"
+	else
+		# For local debugging. Not used by the GitHub CI
+		# itself.
+		eval "export $key=\"$val\""
+	fi
 }
 
 check_unignored_build_artifacts ()
@@ -96,6 +105,35 @@ macos-latest)
 esac
 
 case "$jobname" in
+linux-gcc)
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
+	;;
+linux-TEST-vars)
+	setenv --test GIT_TEST_SPLIT_INDEX yes
+	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
+	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
+	setenv --test GIT_TEST_OE_SIZE 10
+	setenv --test GIT_TEST_OE_DELTA_SIZE 5
+	setenv --test GIT_TEST_COMMIT_GRAPH 1
+	setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
+	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
+	setenv --test GIT_TEST_WRITE_REV_INDEX 1
+	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
+	;;
+linux-clang)
+	setenv --test GIT_TEST_DEFAULT_HASH sha1
+	;;
+linux-sha256)
+	setenv --test GIT_TEST_DEFAULT_HASH sha256
+	;;
+pedantic)
+	# Don't run the tests; we only care about whether Git can be
+	# built.
+	setenv --build DEVOPTS pedantic
+	;;
 linux32)
 	CC=gcc
 	;;
diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh
index 646967481f6..45298562982 100755
--- a/ci/make-test-artifacts.sh
+++ b/ci/make-test-artifacts.sh
@@ -8,5 +8,3 @@ mkdir -p "$1" # in case ci/lib.sh decides to quit early
 . ${0%/*}/lib.sh
 
 make artifacts-tar ARTIFACTS_DIRECTORY="$1"
-
-check_unignored_build_artifacts
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
deleted file mode 100755
index 35d45a9373e..00000000000
--- a/ci/run-build-and-tests.sh
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/bin/sh
-#
-# Build and test Git
-#
-
-. ${0%/*}/lib.sh
-
-export MAKE_TARGETS="all test"
-
-case "$jobname" in
-linux-gcc)
-	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
-	;;
-linux-TEST-vars)
-	setenv --test GIT_TEST_SPLIT_INDEX yes
-	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
-	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
-	setenv --test GIT_TEST_OE_SIZE 10
-	setenv --test GIT_TEST_OE_DELTA_SIZE 5
-	setenv --test GIT_TEST_COMMIT_GRAPH 1
-	setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
-	setenv --test GIT_TEST_MULTI_PACK_INDEX 1
-	setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
-	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
-	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
-	setenv --test GIT_TEST_WRITE_REV_INDEX 1
-	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
-	;;
-linux-clang)
-	setenv --test GIT_TEST_DEFAULT_HASH sha1
-	;;
-linux-sha256)
-	setenv --test GIT_TEST_DEFAULT_HASH sha256
-	;;
-pedantic)
-	# Don't run the tests; we only care about whether Git can be
-	# built.
-	setenv --build DEVOPTS pedantic
-	export MAKE_TARGETS=all
-	;;
-esac
-
-# Any new "test" targets should not go after this "make", but should
-# adjust $MAKE_TARGETS. Otherwise compilation-only targets above will
-# start running tests.
-make $MAKE_TARGETS
-check_unignored_build_artifacts
diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index b9a682b4bcd..70326961454 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -8,5 +8,3 @@
 make --quiet -C t T="$(cd t &&
 	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
 	tr '\n' ' ')"
-
-check_unignored_build_artifacts
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 41e2b126311..6b5cce03bd7 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -39,4 +39,3 @@ test -s Documentation/git.html
 grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
 
 rm -f stdout.log stderr.log stderr.raw
-check_unignored_build_artifacts
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 13/25] CI: check ignored unignored build artifacts in "win[+VS] build" too

[Ævar Arnfjörð Bjarmason]

Use the newly created "ci/check-unignored-build-artifacts.sh" script
in the "win build" and "win+VS build" steps too. These steps weren't
doing this sanity check of untracked files against .gitignore before,
but as the check passes for them let's keep doing that extra sanity
check.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index c83f6a21ca3..05c5771e7f6 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -93,6 +93,9 @@ jobs:
         HOME: ${{runner.workspace}}
         NO_PERL: 1
       run: . /etc/profile && ci/make-test-artifacts.sh artifacts
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
+      shell: bash
     - name: zip up tracked files
       run: git archive -o artifacts/tracked.tar.gz HEAD
     - name: upload tracked files and build artifacts
@@ -180,6 +183,9 @@ jobs:
       run: |
         mkdir -p artifacts &&
         eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts NO_GETTEXT=YesPlease 2>&1 | grep ^tar)"
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
+      shell: bash
     - name: zip up tracked files
       run: git archive -o artifacts/tracked.tar.gz HEAD
     - name: upload tracked files and build artifacts
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 14/25] CI: invoke "make artifacts-tar" directly in windows-build

[Ævar Arnfjörð Bjarmason]

Change the windows-build job to invoke the "make artifacts-tar" step
directly, instead of calling a "ci/make-test-artifacts.sh" script.

The script was needed because "ci/lib.sh" would set up various
environment variables for us, but now we can instead use the
"ci/lib.sh" in its script mode.

The "mkdir -p" added in b819f1d2cec (ci: parallelize testing on
Windows, 2019-01-29) isn't needed, the same commit added that "mkdir
-p" to the "artifacts-tar" rule itself, so we can have "make" create
the directory for us.

This also has the benefit of making the "build" step less chatty,
since it won't start with the verbose "set -x" output, that's now
contained in the "ci/lib.sh" step.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml |  5 ++++-
 ci/make-test-artifacts.sh  | 10 ----------
 2 files changed, 4 insertions(+), 11 deletions(-)
 delete mode 100755 ci/make-test-artifacts.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 05c5771e7f6..2c23a19eac2 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -87,12 +87,15 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
+    - run: ci/lib.sh
+      shell: bash
     - name: build
       shell: bash
       env:
         HOME: ${{runner.workspace}}
         NO_PERL: 1
-      run: . /etc/profile && ci/make-test-artifacts.sh artifacts
+      run: . /etc/profile && make artifacts-tar ARTIFACTS_DIRECTORY=artifacts
+      if: success()
     - run: ci/check-unignored-build-artifacts.sh
       if: success()
       shell: bash
diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh
deleted file mode 100755
index 45298562982..00000000000
--- a/ci/make-test-artifacts.sh
+++ /dev/null
@@ -1,10 +0,0 @@
-#!/bin/sh
-#
-# Build Git and store artifacts for testing
-#
-
-mkdir -p "$1" # in case ci/lib.sh decides to quit early
-
-. ${0%/*}/lib.sh
-
-make artifacts-tar ARTIFACTS_DIRECTORY="$1"
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 15/25] CI: split up and reduce "ci/test-documentation.sh"

[Ævar Arnfjörð Bjarmason]

Change the "ci/test-documentation.sh" script to run the bash-specific
parts in as one command in the CI job itself, and to run the two "make
doc" commands at the top-level.

It'll now be obvious from the title of the step if if we failed in the
asciidoc or asciidoctor step.

Since the "check_unignored_build_artifacts()" function is now only
used in "ci/check-unignored-build-artifacts.sh" move that function
there.

The recipe for the job in ".github/workflows/main.yml" is now a bit
verbose because it's effectively the same job twice, with a "make
clean" in-between. It would be better for the verbosity to run it via
a matrix as done in the alternate approach in [1] does, but then we'd
sacrifice overall CPU time for the brevity. It might still be worth
doing, but let's go for this simpler approach for now.

1. https://lore.kernel.org/git/patch-v2-6.6-7c423c8283d-20211120T030848Z-avarab@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml            | 16 +++++++++++++-
 ci/check-unignored-build-artifacts.sh | 10 +++++++++
 ci/lib.sh                             | 10 ---------
 ci/test-documentation.sh              | 31 +++++++++------------------
 4 files changed, 35 insertions(+), 32 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 2c23a19eac2..92b914f16fd 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -365,4 +365,18 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/test-documentation.sh
+    - run: ci/lib.sh
+    - run: make check-docs
+    - run: "make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)"
+      shell: bash
+    - run: ci/test-documentation.sh AsciiDoc
+      if: success()
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
+    - run: make clean
+    - run: "make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)"
+      shell: bash
+    - run: ci/test-documentation.sh Asciidoctor
+      if: success()
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh
index 56d04b0db9a..0bc04f32804 100755
--- a/ci/check-unignored-build-artifacts.sh
+++ b/ci/check-unignored-build-artifacts.sh
@@ -5,4 +5,14 @@
 
 . ${0%/*}/lib.sh
 
+check_unignored_build_artifacts ()
+{
+	! git ls-files --other --exclude-standard --error-unmatch \
+		-- ':/*' 2>/dev/null ||
+	{
+		echo "$(tput setaf 1)error: found unignored build artifacts$(tput sgr0)"
+		false
+	}
+}
+
 check_unignored_build_artifacts
diff --git a/ci/lib.sh b/ci/lib.sh
index 05f3dd15e27..fc6ce4d3e04 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -39,16 +39,6 @@ setenv () {
 	fi
 }
 
-check_unignored_build_artifacts ()
-{
-	! git ls-files --other --exclude-standard --error-unmatch \
-		-- ':/*' 2>/dev/null ||
-	{
-		echo "$(tput setaf 1)error: found unignored build artifacts$(tput sgr0)"
-		false
-	}
-}
-
 # GitHub Action doesn't set TERM, which is required by tput
 setenv TERM ${TERM:-dumb}
 
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 6b5cce03bd7..b8a6a6f664e 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -1,10 +1,12 @@
-#!/usr/bin/env bash
+#!/bin/sh
 #
-# Perform sanity checks on documentation and build it.
+# Perform sanity checks on "make doc" output and built documentation
 #
 
 . ${0%/*}/lib.sh
 
+generator=$1
+
 filter_log () {
 	sed -e '/^GIT_VERSION = /d' \
 	    -e "/constant Gem::ConfigMap is deprecated/d" \
@@ -14,28 +16,15 @@ filter_log () {
 	    "$1"
 }
 
-make check-docs
-
-# Build docs with AsciiDoc
-make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
-cat stderr.raw
-filter_log stderr.raw >stderr.log
-test ! -s stderr.log
-test -s Documentation/git.html
-test -s Documentation/git.xml
-test -s Documentation/git.1
-grep '<meta name="generator" content="AsciiDoc ' Documentation/git.html
-
-rm -f stdout.log stderr.log stderr.raw
-check_unignored_build_artifacts
-
-# Build docs with AsciiDoctor
-make clean
-make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
 cat stderr.raw
 filter_log stderr.raw >stderr.log
 test ! -s stderr.log
 test -s Documentation/git.html
-grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
+if test "$generator" = "Asciidoctor"
+then
+	test -s Documentation/git.xml
+	test -s Documentation/git.1
+fi
+grep "<meta name=\"generator\" content=\"$generator " Documentation/git.html
 
 rm -f stdout.log stderr.log stderr.raw
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 16/25] CI: combine ci/install{,-docker}-dependencies.sh

[Ævar Arnfjörð Bjarmason]

Combine the "ci/install-dependencies.sh" and
"ci/install-docker-dependencies.sh" into one script. The "case"
statement in the latter only cared about "$jobname", and can be folded
into the same "case" statement in the former.

The reason they split up is historical, and because the
"ci/install-dependencies.sh" used "ci/lib.sh", which requires
"bash". At least one of the docker containers doesn't have "bash". To
make the existing code POSIX-compatible we need to replace pushd/popd
with a sub-shell, but no other changes were needed.

This also fixes issue in "ci/install-docker-dependencies.sh" where
we'd hide errors due to not using "set -e". Now that we include
"ci/lib.sh" we won't have that potential issue.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml        |  2 +-
 ci/install-dependencies.sh        | 27 ++++++++++++++++++++++-----
 ci/install-docker-dependencies.sh | 22 ----------------------
 3 files changed, 23 insertions(+), 28 deletions(-)
 delete mode 100755 ci/install-docker-dependencies.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 92b914f16fd..3fd3e4f873c 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -311,7 +311,7 @@ jobs:
     container: ${{matrix.vector.image}}
     steps:
     - uses: actions/checkout@v1
-    - run: ci/install-docker-dependencies.sh
+    - run: ci/install-dependencies.sh
     - run: ci/lib.sh
     - run: make
     - run: make test
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 01006379fd0..9388289d0ed 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -1,4 +1,4 @@
-#!/usr/bin/env bash
+#!/bin/sh
 #
 # Install dependencies required to build and test Git on Linux and macOS
 #
@@ -34,18 +34,20 @@ ubuntu-latest)
 	sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \
 		$UBUNTU_COMMON_PKGS $CC_PACKAGE
 	mkdir --parents "$P4_PATH"
-	pushd "$P4_PATH"
+	(
+		cd "$P4_PATH"
 		wget --quiet "$P4WHENCE/bin.linux26x86_64/p4d"
 		wget --quiet "$P4WHENCE/bin.linux26x86_64/p4"
 		chmod u+x p4d
 		chmod u+x p4
-	popd
+	)
 	mkdir --parents "$GIT_LFS_PATH"
-	pushd "$GIT_LFS_PATH"
+	(
+		cd "$GIT_LFS_PATH"
 		wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
 		tar --extract --gunzip --file "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
 		cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs .
-	popd
+	)
 	;;
 macos-latest)
 	export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
@@ -90,6 +92,21 @@ linux-gcc-default)
 	sudo apt-get -q update
 	sudo apt-get -q -y install $UBUNTU_COMMON_PKGS
 	;;
+linux32)
+	linux32 --32bit i386 sh -c '
+		apt update >/dev/null &&
+		apt install -y build-essential libcurl4-openssl-dev \
+			libssl-dev libexpat-dev gettext python >/dev/null
+	'
+	;;
+linux-musl)
+	apk add --update build-base curl-dev openssl-dev expat-dev gettext \
+		pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null
+	;;
+pedantic)
+	dnf -yq update >/dev/null &&
+	dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null
+	;;
 esac
 
 if type p4d >/dev/null && type p4 >/dev/null
diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh
deleted file mode 100755
index 78b7e326da6..00000000000
--- a/ci/install-docker-dependencies.sh
+++ /dev/null
@@ -1,22 +0,0 @@
-#!/bin/sh
-#
-# Install dependencies required to build and test Git inside container
-#
-
-case "$jobname" in
-linux32)
-	linux32 --32bit i386 sh -c '
-		apt update >/dev/null &&
-		apt install -y build-essential libcurl4-openssl-dev \
-			libssl-dev libexpat-dev gettext python >/dev/null
-	'
-	;;
-linux-musl)
-	apk add --update build-base curl-dev openssl-dev expat-dev gettext \
-		pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null
-	;;
-pedantic)
-	dnf -yq update >/dev/null &&
-	dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null
-	;;
-esac
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 17/25] CI: move "env" definitions into ci/lib.sh

[Ævar Arnfjörð Bjarmason]

Have the GitHub CI jobs use the "ci/lib".sh as a source of truth for
environment variables that affect their builds and tests. This brings
them in line with the rest of the jobs, and make it easier to see the
entire CI configuration per jobname at a glance.

To do this we need to add a ci/lib.sh "step" to the remaining jobs
that didn't have it added in preceding commits.

The Makefile parameters are luckily all accepted via the environment,
so we can export these instead of adding them to MAKEFLAGS.

Let's also use the documented GIT_CONIFG_* mechanism for setting
config instead of the internal GIT_CONFIG_PARAMETERS variable. This
adjusts code added in 889cacb6897 (ci: configure GitHub Actions for
CI/PR, 2020-04-11), we could probably use the GIT_AUTHOR_NAME etc, but
let's keep setting this via config, just with the documented
mechanism.

By setting "ARTIFACTS_DIRECTORY=artifacts" once we don't need to
repeat it in various places, let's get it from the environment
instead.

Finally, the "DEVELOPER: 1" was only needed by the "sparse" job, which
wasn't using "ci/lib.sh" (and which sets DEVELOPER=1). Let's instead
have the "sparse" job use the "ci/lib.sh" and remove DEVELOPER=1 from
".github/workflows/main.yml".

This substantially speeds up the "sparse" job, since it'll now pick up
the "--jobs" setting in MAKEFLAGS that we use everywhere else. Before
it ran in around 4m30s, now in around 2m30s.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 29 +++++++++++------------------
 ci/lib.sh                  | 21 +++++++++++++++++++++
 2 files changed, 32 insertions(+), 18 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 3fd3e4f873c..fe37e0f1b36 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -2,9 +2,6 @@ name: CI
 
 on: [push, pull_request]
 
-env:
-  DEVELOPER: 1
-
 jobs:
   ci-config:
     name: config
@@ -93,8 +90,7 @@ jobs:
       shell: bash
       env:
         HOME: ${{runner.workspace}}
-        NO_PERL: 1
-      run: . /etc/profile && make artifacts-tar ARTIFACTS_DIRECTORY=artifacts
+      run: . /etc/profile && make artifacts-tar
       if: success()
     - run: ci/check-unignored-build-artifacts.sh
       if: success()
@@ -105,7 +101,7 @@ jobs:
       uses: actions/upload-artifact@v2
       with:
         name: windows-artifacts
-        path: artifacts
+        path: ${{env.ARTIFACTS_DIRECTORY}}
   windows-test:
     name: win test
     env:
@@ -126,6 +122,8 @@ jobs:
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
+    - run: ci/lib.sh
+      shell: bash
     - name: test
       shell: bash
       run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
@@ -145,9 +143,6 @@ jobs:
       jobname: vs-build
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
-    env:
-      NO_PERL: 1
-      GIT_CONFIG_PARAMETERS: "'user.name=CI' 'user.email=ci@git'"
     runs-on: windows-latest
     steps:
     - uses: actions/checkout@v2
@@ -171,6 +166,8 @@ jobs:
     - name: copy dlls to root
       shell: cmd
       run: compat\vcbuild\vcpkg_copy_dlls.bat release
+    - run: ci/lib.sh
+      shell: bash
     - name: generate Visual Studio solution
       shell: bash
       run: |
@@ -181,21 +178,16 @@ jobs:
     - name: bundle artifact tar
       shell: bash
       env:
-        MSVC: 1
         VCPKG_ROOT: ${{github.workspace}}\compat\vcbuild\vcpkg
       run: |
-        mkdir -p artifacts &&
-        eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts NO_GETTEXT=YesPlease 2>&1 | grep ^tar)"
-    - run: ci/check-unignored-build-artifacts.sh
-      if: success()
-      shell: bash
+        eval "$(make -n artifacts-tar 2>&1 | grep -e ^mkdir -e ^tar)"
     - name: zip up tracked files
       run: git archive -o artifacts/tracked.tar.gz HEAD
     - name: upload tracked files and build artifacts
       uses: actions/upload-artifact@v2
       with:
         name: vs-artifacts
-        path: artifacts
+        path: ${{env.ARTIFACTS_DIRECTORY}}
   vs-test:
     name: win+VS test
     env:
@@ -216,10 +208,10 @@ jobs:
     - name: extract tracked files and build artifacts
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
+    - run: ci/lib.sh
+      shell: bash
     - name: test
       shell: bash
-      env:
-        NO_SVN_TESTS: 1
       run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
     - name: ci/print-test-failures.sh
       if: failure()
@@ -354,6 +346,7 @@ jobs:
     - uses: actions/checkout@v2
     - name: Install other dependencies
       run: ci/install-dependencies.sh
+    - run: ci/lib.sh
     - run: make sparse
   documentation:
     name: documentation
diff --git a/ci/lib.sh b/ci/lib.sh
index fc6ce4d3e04..b882849ed5d 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -95,6 +95,27 @@ macos-latest)
 esac
 
 case "$jobname" in
+windows-build)
+	setenv --build NO_PERL NoThanks
+	setenv --build ARTIFACTS_DIRECTORY artifacts
+	;;
+vs-build)
+	setenv --build NO_PERL NoThanks
+	setenv --build NO_GETTEXT NoThanks
+	setenv --build ARTIFACTS_DIRECTORY artifacts
+	setenv --build INCLUDE_DLLS_IN_ARTIFACTS YesPlease
+	setenv --build MSVC YesPlease
+
+	setenv --build GIT_CONFIG_COUNT 2
+	setenv --build GIT_CONFIG_KEY_0 user.name
+	setenv --build GIT_CONFIG_VALUE_0 CI
+	setenv --build GIT_CONFIG_KEY_1 user.emailname
+	setenv --build GIT_CONFIG_VALUE_1 ci@git
+	setenv --build GIT_CONFIG_VALUE_1 ci@git
+	;;
+vs-test)
+	setenv --test NO_SVN_TESTS YesPlease
+	;;
 linux-gcc)
 	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
 	;;
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 18/25] ci/run-test-slice.sh: replace shelling out with "echo"

[Ævar Arnfjörð Bjarmason]

Amend this code added in b819f1d2cec (ci: parallelize testing on
Windows, 2019-01-29) to stop shelling out to "tr", all we're trying to
do here is to turn '\n'-delimited output into a ' '-delimited list. To
do that we can just use "echo".

None of these files will contain the odd characters using "echo" might
choke on in the general case (i.e. '\'-escapes and the like).

This has the added benefit of not adding a redundant ' ' to the end of
the T-parameter, which didn't matter in practice, but as we'll see in
a subsequent commit makes for slightly nicer output, as we'll show
this variable using a GitHub CI-native feature.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/run-test-slice.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index 70326961454..1de6a18ca47 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -5,6 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-make --quiet -C t T="$(cd t &&
-	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
-	tr '\n' ' ')"
+tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \
+	t[0-9]*.sh))
+make --quiet -C t T="$tests"
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 19/25] CI: pre-select test slice in Windows & VS tests

[Ævar Arnfjörð Bjarmason]

In preceding commits the tests have been changed to do their setup via
$GITHUB_ENV in one step, and to then have subsequent steps that re-use
that environment.

Let's change the "test slice" tests added in b819f1d2cec (ci:
parallelize testing on Windows, 2019-01-29) to do the same. These
tests select 10% of the tests to run in 10 "test slices". Now we'll
select those in a step that immediately precedes the testing step, and
then simply invoke "make -C t -e".

This has the advantage that the tests to be run are now listed in the
standard "Run" drop-down at the start of the "test" step.

Since the "T" variable in "t/Makefile" doesn't normally accept
overrides from the environment we need to invoke "make" with the "-e"
option (a.k.a. "--environment-overrides"). We could also make $(T) in
t/Makefile be a "?=" assigned variable, but this way works, and is
arguably clearer as it's more obvious that we're injecting a special
list of tests that override the normal behavior of that Makefile.

Note that we cannot run the top-level "make test" here, because of how
the Windows CI builds git, i.e. either via CMake or some option that
would cause "make test" to recompile git itself. Instead we run "make
-C t [...]".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml                     | 10 ++++++++--
 ci/{run-test-slice.sh => select-test-slice.sh} |  4 ++--
 2 files changed, 10 insertions(+), 4 deletions(-)
 rename ci/{run-test-slice.sh => select-test-slice.sh} (57%)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index fe37e0f1b36..1e13718b989 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -124,9 +124,12 @@ jobs:
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
     - run: ci/lib.sh
       shell: bash
+    - name: select tests
+      run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
+      shell: bash
     - name: test
       shell: bash
-      run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
+      run: . /etc/profile && make -C t -e
     - name: ci/print-test-failures.sh
       if: failure()
       shell: bash
@@ -210,9 +213,12 @@ jobs:
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
     - run: ci/lib.sh
       shell: bash
+    - name: select tests
+      run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
+      shell: bash
     - name: test
       shell: bash
-      run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
+      run: . /etc/profile && make -C t -e
     - name: ci/print-test-failures.sh
       if: failure()
       shell: bash
diff --git a/ci/run-test-slice.sh b/ci/select-test-slice.sh
similarity index 57%
rename from ci/run-test-slice.sh
rename to ci/select-test-slice.sh
index 1de6a18ca47..ec602f8a053 100755
--- a/ci/run-test-slice.sh
+++ b/ci/select-test-slice.sh
@@ -1,10 +1,10 @@
 #!/bin/sh
 #
-# Test Git in parallel
+# Select a portion of the tests for testing Git in parallel
 #
 
 . ${0%/*}/lib.sh
 
 tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \
 	t[0-9]*.sh))
-make --quiet -C t T="$tests"
+echo T="$tests" >>$GITHUB_ENV
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 20/25] CI: only invoke ci/lib.sh as "steps" in main.yml

[Ævar Arnfjörð Bjarmason]

Change the scripts in ci/ to stop using ci/lib.sh as a library, now
that the only thing it did for them was to "set -ex" and possibly set
TERM=dumb.

Let's create a ci/lib-tput.sh for those that need to use "tput"
instead, and have these scripts invoke "set -ex" themselves.

This makes their invocation a lot less verbose, since they'll be
relying on an earlier step in the CI job to have set the variables in
$GITHUB_ENV, and won't be spewing their own trace output to set those
variables again.

Let's also create a ci/lib-ci-type.sh, and have ci/lib.sh and
ci/print-test-failures.sh share the logic to discover the CI type. We
could have set the CI_TYPE in the environment with "setenv", but let's
avoid that verbosity for this purely internal variable.

The "ci/lib.sh" is now no longer a "Library of functions shared by all
CI scripts", so let's remove that commentary, and the misleading
comment about "set -ex" being for "installing dependencies", we're now
no longer using it in "ci/install-dependencies.sh" (but it does its
own "set -ex").

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/check-unignored-build-artifacts.sh |  4 +++-
 ci/install-dependencies.sh            |  2 +-
 ci/lib-ci-type.sh                     |  8 ++++++++
 ci/lib-tput.sh                        |  2 ++
 ci/lib.sh                             | 27 ++++++++++-----------------
 ci/print-test-failures.sh             |  6 +++---
 ci/select-test-slice.sh               |  2 +-
 ci/test-documentation.sh              |  2 +-
 8 files changed, 29 insertions(+), 24 deletions(-)
 create mode 100644 ci/lib-ci-type.sh
 create mode 100644 ci/lib-tput.sh

diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh
index 0bc04f32804..c27d6a97f45 100755
--- a/ci/check-unignored-build-artifacts.sh
+++ b/ci/check-unignored-build-artifacts.sh
@@ -3,7 +3,9 @@
 # Check whether the build created anything not in our .gitignore
 #
 
-. ${0%/*}/lib.sh
+set -ex
+
+. ${0%/*}/lib-tput.sh
 
 check_unignored_build_artifacts ()
 {
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 9388289d0ed..ee9af62fc92 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -3,7 +3,7 @@
 # Install dependencies required to build and test Git on Linux and macOS
 #
 
-. ${0%/*}/lib.sh
+set -ex
 
 UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
  tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
diff --git a/ci/lib-ci-type.sh b/ci/lib-ci-type.sh
new file mode 100644
index 00000000000..bd6e093c8f4
--- /dev/null
+++ b/ci/lib-ci-type.sh
@@ -0,0 +1,8 @@
+if test "$GITHUB_ACTIONS" = "true"
+then
+	CI_TYPE=github-actions
+else
+	echo "Could not identify CI type" >&2
+	env >&2
+	exit 1
+fi
diff --git a/ci/lib-tput.sh b/ci/lib-tput.sh
new file mode 100644
index 00000000000..baed1892f69
--- /dev/null
+++ b/ci/lib-tput.sh
@@ -0,0 +1,2 @@
+# GitHub Action doesn't set TERM, which is required by tput
+export TERM=${TERM:-dumb}
diff --git a/ci/lib.sh b/ci/lib.sh
index b882849ed5d..5eadc96de29 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,13 +1,9 @@
 #!/bin/sh
-
-# Library of functions shared by all CI scripts
-
-# Set 'exit on error' for all CI scripts to let the caller know that
-# something went wrong.
-# Set tracing executed commands, primarily setting environment variables
-# and installing dependencies.
 set -ex
 
+# Helper libraries
+. ${0%/*}/lib-ci-type.sh
+
 # Starting assertions
 if test -z "$jobname"
 then
@@ -39,15 +35,11 @@ setenv () {
 	fi
 }
 
-# GitHub Action doesn't set TERM, which is required by tput
-setenv TERM ${TERM:-dumb}
-
 # Clear MAKEFLAGS that may come from the outside world.
 MAKEFLAGS=
 
-if test "$GITHUB_ACTIONS" = "true"
-then
-	CI_TYPE=github-actions
+case "$CI_TYPE" in
+github-actions)
 	CC="${CC:-gcc}"
 
 	setenv --test GIT_PROVE_OPTS "--timer --jobs 10"
@@ -57,11 +49,12 @@ then
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 
 	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
-else
-	echo "Could not identify CI type" >&2
-	env >&2
+	;;
+*)
+	echo "Unhandled CI type: $CI_TYPE" >&2
 	exit 1
-fi
+	;;
+esac
 
 setenv --build DEVELOPER 1
 setenv --test DEFAULT_TEST_TARGET prove
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 0c63b6f7962..452aff35d74 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -3,10 +3,10 @@
 # Print output of failing tests
 #
 
-. ${0%/*}/lib.sh
+set -e
 
-# Tracing executed commands would produce too much noise in the loop below.
-set +x
+. ${0%/*}/lib-ci-type.sh
+. ${0%/*}/lib-tput.sh
 
 cd t/
 
diff --git a/ci/select-test-slice.sh b/ci/select-test-slice.sh
index ec602f8a053..a0332095bb3 100755
--- a/ci/select-test-slice.sh
+++ b/ci/select-test-slice.sh
@@ -3,7 +3,7 @@
 # Select a portion of the tests for testing Git in parallel
 #
 
-. ${0%/*}/lib.sh
+set -ex
 
 tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \
 	t[0-9]*.sh))
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index b8a6a6f664e..64ff212cdaa 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -3,7 +3,7 @@
 # Perform sanity checks on "make doc" output and built documentation
 #
 
-. ${0%/*}/lib.sh
+set -ex
 
 generator=$1
 
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 21/25] CI: narrow down variable definitions in --build and --test

[Ævar Arnfjörð Bjarmason]

In a preceding step the "setvar" function was made to take a
"--build", "--test" or "--all" argument to indicate where the
variables it sets were used.

Let's make use of that by having the relevant parts of
".github/workflows/main.yml" invoke "ci/lib.sh" with those options.

By doing this the set of variables shown in build-only steps will be
fewer, which makes diagnosing anything going on there easier, as we
won't have to look at a deluge of e.g. GIT_TEST_* variables.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 20 +++++++++++---------
 ci/lib.sh                  | 17 +++++++++++++++++
 2 files changed, 28 insertions(+), 9 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 1e13718b989..0787cadc76b 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -84,7 +84,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
       shell: bash
     - name: build
       shell: bash
@@ -122,7 +122,7 @@ jobs:
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
-    - run: ci/lib.sh
+    - run: ci/lib.sh --test
       shell: bash
     - name: select tests
       run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
@@ -169,7 +169,7 @@ jobs:
     - name: copy dlls to root
       shell: cmd
       run: compat\vcbuild\vcpkg_copy_dlls.bat release
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
       shell: bash
     - name: generate Visual Studio solution
       shell: bash
@@ -211,7 +211,7 @@ jobs:
     - name: extract tracked files and build artifacts
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
-    - run: ci/lib.sh
+    - run: ci/lib.sh --test
       shell: bash
     - name: select tests
       run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
@@ -275,8 +275,9 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make
+    - run: ci/lib.sh --test
     - run: make test
       if: success()
     - run: ci/print-test-failures.sh
@@ -310,8 +311,9 @@ jobs:
     steps:
     - uses: actions/checkout@v1
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make
+    - run: ci/lib.sh --test
     - run: make test
       if: success() && matrix.vector.skip-tests != 'no'
     - run: ci/print-test-failures.sh
@@ -331,7 +333,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make ci-static-analysis
   sparse:
     needs: ci-config
@@ -352,7 +354,7 @@ jobs:
     - uses: actions/checkout@v2
     - name: Install other dependencies
       run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make sparse
   documentation:
     name: documentation
@@ -364,7 +366,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make check-docs
     - run: "make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)"
       shell: bash
diff --git a/ci/lib.sh b/ci/lib.sh
index 5eadc96de29..367d1ee05d5 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -5,6 +5,13 @@ set -ex
 . ${0%/*}/lib-ci-type.sh
 
 # Starting assertions
+mode=$1
+if test -z "$mode"
+then
+	echo "need a $0 mode, e.g. --build or --test"
+	exit 1
+fi
+
 if test -z "$jobname"
 then
 	echo "must set a CI jobname" >&2
@@ -13,9 +20,14 @@ fi
 
 # Helper functions
 setenv () {
+	skip=
 	varmode=
 	case "$1" in
 	--*)
+		if test "$1" != "$mode" && test "$1" != "--all"
+		then
+			skip=t
+		fi
 		varmode=$1
 		shift
 		;;
@@ -25,6 +37,11 @@ setenv () {
 	val=$2
 	shift 2
 
+	if test -n "$skip"
+	then
+		return 0
+	fi
+
 	if test -n "$GITHUB_ENV"
 	then
 		echo "$key=$val" >>"$GITHUB_ENV"
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 22/25] CI: add more variables to MAKEFLAGS, except under vs-build

[Ævar Arnfjörð Bjarmason]

It's clearer that "DEVELOPER" is a flag that affects the Makefile
itself in particular if it's put into "MAKEFLAGS than" if it
generically sits in the environment. Let's move both it and
"SKIP_DASHED_BUILT_INS" to "MAKEFLAGS".

We can't do this under vs-build, since that invokes cmake. Let's have
only that job set these in the environment.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 367d1ee05d5..475e9f63a74 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -55,6 +55,13 @@ setenv () {
 # Clear MAKEFLAGS that may come from the outside world.
 MAKEFLAGS=
 
+# Common make and cmake build options
+DEVELOPER=1
+SKIP_DASHED_BUILT_INS=YesPlease
+
+# Use common options for "make" (cmake in "vs-build" below)
+MAKEFLAGS="DEVELOPER=$DEVELOPER SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
+
 case "$CI_TYPE" in
 github-actions)
 	CC="${CC:-gcc}"
@@ -73,10 +80,8 @@ github-actions)
 	;;
 esac
 
-setenv --build DEVELOPER 1
 setenv --test DEFAULT_TEST_TARGET prove
 setenv --test GIT_TEST_CLONE_2GB true
-setenv --build SKIP_DASHED_BUILT_INS YesPlease
 
 case "$runs_on_pool" in
 ubuntu-latest)
@@ -110,6 +115,9 @@ windows-build)
 	setenv --build ARTIFACTS_DIRECTORY artifacts
 	;;
 vs-build)
+	setenv --build DEVELOPER $DEVELOPER
+	setenv --build SKIP_DASHED_BUILT_INS $SKIP_DASHED_BUILT_INS
+
 	setenv --build NO_PERL NoThanks
 	setenv --build NO_GETTEXT NoThanks
 	setenv --build ARTIFACTS_DIRECTORY artifacts
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 23/25] CI: stop over-setting the $CC variable

[Ævar Arnfjörð Bjarmason]

As detailed in 2c8921db2b8 (travis-ci: build with the right compiler,
2019-01-17) the reason we started using $CC in $MAKEFLAGS as opposed
to setting it in the environment was due to Travis CI clobbering $CC
in the environment.

We don't need to set it unconditionally to accomplish that, but rather
just have it set for those jobs that need them. E.g. the "win+VS
build" job confusingly has CC=gcc set, even though it builds with
MSVC.

This partially reverts my 707d2f2fe86 (CI: use "$runs_on_pool", not
"$jobname" to select packages & config, 2021-11-23), i.e. we're now
aiming to only set those variables specific jobs need.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 13 -------------
 ci/lib.sh                  | 26 ++++++++++++++++++++++----
 2 files changed, 22 insertions(+), 17 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 0787cadc76b..6d25ec4ae3b 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -238,37 +238,24 @@ jobs:
       matrix:
         vector:
           - jobname: linux-clang
-            cc: clang
             pool: ubuntu-latest
           - jobname: linux-sha256
-            cc: clang
             os: ubuntu
             pool: ubuntu-latest
           - jobname: linux-gcc
-            cc: gcc
-            cc_package: gcc-8
             pool: ubuntu-latest
           - jobname: linux-TEST-vars
-            cc: gcc
             os: ubuntu
-            cc_package: gcc-8
             pool: ubuntu-latest
           - jobname: osx-clang
-            cc: clang
             pool: macos-latest
           - jobname: osx-gcc
-            cc: gcc
-            cc_package: gcc-9
             pool: macos-latest
           - jobname: linux-gcc-default
-            cc: gcc
             pool: ubuntu-latest
           - jobname: linux-leaks
-            cc: gcc
             pool: ubuntu-latest
     env:
-      CC: ${{matrix.vector.cc}}
-      CC_PACKAGE: ${{matrix.vector.cc_package}}
       jobname: ${{matrix.vector.jobname}}
       runs_on_pool: ${{matrix.vector.pool}}
     runs-on: ${{matrix.vector.pool}}
diff --git a/ci/lib.sh b/ci/lib.sh
index 475e9f63a74..7064a17cfeb 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -52,7 +52,9 @@ setenv () {
 	fi
 }
 
-# Clear MAKEFLAGS that may come from the outside world.
+# Clear variables that may come from the outside world.
+CC=
+CC_PACKAGE=
 MAKEFLAGS=
 
 # Common make and cmake build options
@@ -64,8 +66,6 @@ MAKEFLAGS="DEVELOPER=$DEVELOPER SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
 
 case "$CI_TYPE" in
 github-actions)
-	CC="${CC:-gcc}"
-
 	setenv --test GIT_PROVE_OPTS "--timer --jobs 10"
 	GIT_TEST_OPTS="--verbose-log -x"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
@@ -135,9 +135,16 @@ vs-test)
 	setenv --test NO_SVN_TESTS YesPlease
 	;;
 linux-gcc)
+	CC=gcc
+	CC_PACKAGE=gcc-8
 	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
 	;;
+linux-gcc-default)
+	CC=gcc
+	;;
 linux-TEST-vars)
+	CC=gcc
+	CC_PACKAGE=gcc-8
 	setenv --test GIT_TEST_SPLIT_INDEX yes
 	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
 	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
@@ -152,10 +159,19 @@ linux-TEST-vars)
 	setenv --test GIT_TEST_WRITE_REV_INDEX 1
 	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
 	;;
+osx-gcc)
+	CC=gcc
+	CC_PACKAGE=gcc-9
+	;;
+osx-clang)
+	CC=clang
+	;;
 linux-clang)
+	CC=clang
 	setenv --test GIT_TEST_DEFAULT_HASH sha1
 	;;
 linux-sha256)
+	CC=clang
 	setenv --test GIT_TEST_DEFAULT_HASH sha256
 	;;
 pedantic)
@@ -173,9 +189,11 @@ linux-musl)
 	MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8"
 	;;
 linux-leaks)
+	CC=gcc
 	setenv --build SANITIZE leak
 	setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true
 	;;
 esac
 
-setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
+MAKEFLAGS="$MAKEFLAGS${CC:+ CC=$CC}"
+setenv --all MAKEFLAGS "$MAKEFLAGS"
-- 
2.35.1.1132.ga1fe46f8690

Re: [PATCH 23/25] CI: stop over-setting the $CC variable

[SZEDER Gábor]

On Mon, Feb 21, 2022 at 03:46:35PM +0100, Ævar Arnfjörð Bjarmason wrote:
> As detailed in 2c8921db2b8 (travis-ci: build with the right compiler,
> 2019-01-17) the reason we started using $CC in $MAKEFLAGS as opposed
> to setting it in the environment was due to Travis CI clobbering $CC
> in the environment.

This is not what 2c8921db2b8 detailed; in fact 2c8921db2b8 detailed
the exact opposite.  Travis CI did not clobber $CC, it set $CC to what
we asked it to.  We started using $CC in $MAKEFLAGS as opposed to
setting it in the environment, because our Makefile _ignores_ $CC in
the environment.

Re: [PATCH 23/25] CI: stop over-setting the $CC variable

[Ævar Arnfjörð Bjarmason]

On Sat, Mar 05 2022, SZEDER Gábor wrote:

> On Mon, Feb 21, 2022 at 03:46:35PM +0100, Ævar Arnfjörð Bjarmason wrote:
>> As detailed in 2c8921db2b8 (travis-ci: build with the right compiler,
>> 2019-01-17) the reason we started using $CC in $MAKEFLAGS as opposed
>> to setting it in the environment was due to Travis CI clobbering $CC
>> in the environment.
>
> This is not what 2c8921db2b8 detailed; in fact 2c8921db2b8 detailed
> the exact opposite.  Travis CI did not clobber $CC, it set $CC to what
> we asked it to.  We started using $CC in $MAKEFLAGS as opposed to
> setting it in the environment, because our Makefile _ignores_ $CC in
> the environment.

Thanks. You're completely right, I don't know how I got that wrong when
writing this summary.

I'll fix it.

But as for the body of the change I believe it looks good & is
consistent with avoiding that issue. I.e. we'll set it in MAKEFLAGS both
before and after this change.

We just won't have a separate "CC" variable in the environment as far as
GitHub's environment variable dump is concerned, which IMO makes this a
lot easier to follow, i.e. we only need CC as a "make" argument, so
let's add it and other such things to MAKEFLAGS only.

[PATCH 24/25] CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case

[Ævar Arnfjörð Bjarmason]

Move the setting of "PYTHON_PATH=[...]" to the "$jobname" case
statement. This partially backs out of my 707d2f2fe86 (CI: use
"$runs_on_pool", not "$jobname" to select packages & config,
2021-11-23), now that we have a "osx-{clang,gcc}" anyway for setting
"$CC" we might as well do away with this part of the "$runs_on_pool"
case.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 7064a17cfeb..36f79eb6a65 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -99,14 +99,6 @@ ubuntu-latest)
 
 	setenv --test GIT_TEST_HTTPD true
 	;;
-macos-latest)
-	if [ "$jobname" = osx-gcc ]
-	then
-		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
-	else
-		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
-	fi
-	;;
 esac
 
 case "$jobname" in
@@ -160,10 +152,12 @@ linux-TEST-vars)
 	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
 	;;
 osx-gcc)
+	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
 	CC=gcc
 	CC_PACKAGE=gcc-9
 	;;
 osx-clang)
+	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
 	CC=clang
 	;;
 linux-clang)
-- 
2.35.1.1132.ga1fe46f8690

[PATCH 25/25] CI: don't use "set -x" in "ci/lib.sh" output

[Ævar Arnfjörð Bjarmason]

Remove the "set -x" from the "ci/lib.sh" output. Before preceding
commits the logic in that file was much more complex, and likely to
fail in some scenarios.

Now we only task "ci/lib.sh" with setting various variables for
subsequent steps in our jobs, so we can start emitting more tailored
debugging output, which makes what it's doing easier to read.

This change also changes the output of the "ci/print-test-failures.sh"
script, since it's the only other user of "ci/lib-ci-type.sh". In that
case it's also helpful to know what "$CI_TYPE" we're using, as that
script doesn't "set -x" and will act differently depending on the
$CI_TYPE.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib-ci-type.sh | 2 ++
 ci/lib.sh         | 9 ++++++++-
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/ci/lib-ci-type.sh b/ci/lib-ci-type.sh
index bd6e093c8f4..6cfe58596d2 100644
--- a/ci/lib-ci-type.sh
+++ b/ci/lib-ci-type.sh
@@ -6,3 +6,5 @@ else
 	env >&2
 	exit 1
 fi
+
+echo "CONFIG: CI_TYPE=$CI_TYPE" >&2
diff --git a/ci/lib.sh b/ci/lib.sh
index 36f79eb6a65..f4323a3dd36 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,5 +1,5 @@
 #!/bin/sh
-set -ex
+set -e
 
 # Helper libraries
 . ${0%/*}/lib-ci-type.sh
@@ -11,12 +11,16 @@ then
 	echo "need a $0 mode, e.g. --build or --test"
 	exit 1
 fi
+echo "CONFIG: mode=$mode" >&2
 
 if test -z "$jobname"
 then
 	echo "must set a CI jobname" >&2
 	exit 1
 fi
+echo "CONFIG: jobname=$jobname" >&2
+echo "CONFIG: runs_on_pool=$runs_on_pool" >&2
+echo "CONFIG: GITHUB_ENV=$GITHUB_ENV" >&2
 
 # Helper functions
 setenv () {
@@ -39,6 +43,7 @@ setenv () {
 
 	if test -n "$skip"
 	then
+		echo "SKIP '$key=$val'" >&2
 		return 0
 	fi
 
@@ -50,6 +55,8 @@ setenv () {
 		# itself.
 		eval "export $key=\"$val\""
 	fi
+
+	echo "SET: '$key=$val'" >&2
 }
 
 # Clear variables that may come from the outside world.
-- 
2.35.1.1132.ga1fe46f8690

Re: [PATCH 00/25] CI: run "make [test]" directly, use $GITHUB_ENV

[Phillip Wood]

Hi Ævar

On 21/02/2022 14:46, Ævar Arnfjörð Bjarmason wrote:
 > [...]
> == Details
> 
> This series conflicts with and is a proposed alternative to Johannes's
> proposed changes to teach "ci/lib.sh" GitHub specific syntax to
> "group" certain parts of the output[2]. That series proposes to keep
> the "ci/run-build-and-tests.sh" and similar scripts, but to teach them
> to emit markers indicating when the "build" part commences, then
> "test" etc.
> 
> I think the approach offered here is better as elaborated on in
> 12/25[1], especially because as Johannes notes[3] we only have one
> level of "grouping" within a "step" available to us. In his version
> using it for "build" v.s. "test" precludes being able to group parts
> of that "build" or "test" output in the future.
> 
> The following compares CI output between the three with a
> change-on-top which caused a compilation error ("this" is this series,
> "js" is Johannes's) when clicking on the "linux-gcc" failure:
> 
>    master: https://github.com/avar/git/runs/5274251909?check_suite_focus=true
>    this: https://github.com/avar/git/runs/5274464670?check_suite_focus=true
>    js: https://github.com/avar/git/runs/5272239403?check_suite_focus=true
> 
>    Here we went from 93 lines of output on "master" to 47 in "this"
>    (107 in "js"). Note also how in "this" we can:
> 
>    - Expand the "Run make" to get the "MAKEFLAGS" and other variables
>      used in the step. That's now a 3-line summary instead of the
>      first 50 lines being the old "ci/lib.sh" "set -x" output.
> 
>    - Because "make" failed, we have an elided "make test" step that's
>      not being run below. We can thus see what steps our failure caused
>      us to skip.
> 
>    - Unlike the "js" version we'll show the compilation error by
>      default (the "js" version is grouped and needs to be expanded),
>      but our output is now brief enough that that's no longer
>      surrounded by other irrelevant output.
> 
>    - Unlike the "js" version there is no "CI setup" group within each
>      step. That work is in the earlier "ci/lib.sh --build" step
>      instead, which sets the config for all subsequent steps.

I can see that showing compilation errors without having to expand 
anything is useful but in my experience they are relatively rare 
compared to test failures. If I understand the rest of the message 
correctly we're left with having to expand print-test-failures and 
searching for "not ok" to find out what went wrong with this series.

> For other output changes look at the difference between "master" and
> "this" at:
> 
>    master: https://github.com/git/git/actions/runs/1866786595
>    this: https://github.com/avar/git/actions/runs/1876270588
> 
> Not explicitly covered in the summary above is that various other
> parts of ci/* were doing the same sort of within-step setup, but now
> do so via cross-step passing of variables via $GITHUB_ENV. E.g. on
> "master" the test slices for the Windows tests have a lot of verbosity
> before they get to running the test itself:
> 
>      https://github.com/git/git/runs/5254267914?check_suite_focus=true#step:5:56
> 
> In the "js" version the test output is hidden ("grouped"), but we've
> got the same amount of verbosity by default:
> 
>      https://github.com/gitgitgadget/git/runs/4937491771?check_suite_focus=true#step:5:67
> 
> Whereas in "this" version that verbosity is in the preceding "select
> tests" step, with the "test" step showing only the relevant end-state
> of the "$T" variable we'll use in the Makefile (hidden by default,
> expanded in this link):
> 
>      https://github.com/avar/git/runs/5274558869?check_suite_focus=true#step:7:13

All I can see in the "$T" variable I have to scroll to get the prove 
output on screen


> This series does not attempt to replace the use of the "group" output
> used for the "ok" or "not ok" portion of tests in Johannes's
> series. When a test fails the output in this series (sans config
> discovery not being repeated, and now summarized in the $GITHUB_ENV
> drop-down) is substantially the same as on "master".
> 
> My summary at [4] goes into other overlap & non-overlap between the
> two. I think using the "group" output for those purposes might be
> useful, although I left some feedback on [5] with problems in the
> current "js" implementation.
> 
> I do think any such changes should follow behind this series, as doing
> that sort of grouping once we can effectively free up an extra "group"
> level (by peeling those off into "steps", as is done here) would be
> much more useful.

I can see the attraction of being able to use make directly in the ci 
from an implementation point of view but from the point of view of 
someone trying to investigate a test failure then unless I've 
misunderstood I don't think this series improves the current situation. 
Why can't you build on top of Dscho's series that makes it easier to see 
the output of the failed tests?

Best Wishes

Phillip

> 1. https://lore.kernel.org/git/patch-12.25-dfd823f2e7d-20220221T122605Z-avarab@gmail.com
> 2. https://lore.kernel.org/git/pull.1117.git.1643050574.gitgitgadget@gmail.com/
> 3. https://lore.kernel.org/git/9333ba781b8240f704e739b00d274f8c3d887e39.1643050574.git.gitgitgadget@gmail.com/
> 4. https://lore.kernel.org/git/220127.86ilu5cdnf.gmgdl@evledraar.gmail.com/
> 5. https://lore.kernel.org/git/220126.86sftbfjl4.gmgdl@evledraar.gmail.com/
> 
> Ævar Arnfjörð Bjarmason (25):
>    CI: run "set -ex" early in ci/lib.sh
>    CI: make "$jobname" explicit, remove fallback
>    CI: remove more dead Travis CI support
>    CI: remove dead "tree skipping" code
>    CI: remove unused Azure ci/* code
>    CI: don't have "git grep" invoke a pager in tree content check
>    CI: have "static-analysis" run a "make ci-static-analysis" target
>    CI: have "static-analysis" run "check-builtins", not "documentation"
>    CI: move p4 and git-lfs variables to ci/install-dependencies.sh
>    CI: consistently use "export" in ci/lib.sh
>    CI: export variables via a wrapper
>    CI: remove "run-build-and-tests.sh", run "make [test]" directly
>    CI: check ignored unignored build artifacts in "win[+VS] build" too
>    CI: invoke "make artifacts-tar" directly in windows-build
>    CI: split up and reduce "ci/test-documentation.sh"
>    CI: combine ci/install{,-docker}-dependencies.sh
>    CI: move "env" definitions into ci/lib.sh
>    ci/run-test-slice.sh: replace shelling out with "echo"
>    CI: pre-select test slice in Windows & VS tests
>    CI: only invoke ci/lib.sh as "steps" in main.yml
>    CI: narrow down variable definitions in --build and --test
>    CI: add more variables to MAKEFLAGS, except under vs-build
>    CI: stop over-setting the $CC variable
>    CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case
>    CI: don't use "set -x" in "ci/lib.sh" output
> 
>   .github/workflows/main.yml            | 100 +++++---
>   Makefile                              |  32 ++-
>   ci/check-directional-formatting.bash  |   2 +-
>   ci/check-unignored-build-artifacts.sh |  20 ++
>   ci/install-dependencies.sh            |  53 ++++-
>   ci/install-docker-dependencies.sh     |  22 --
>   ci/lib-ci-type.sh                     |  10 +
>   ci/lib-tput.sh                        |   2 +
>   ci/lib.sh                             | 315 ++++++++++++--------------
>   ci/make-test-artifacts.sh             |  12 -
>   ci/mount-fileshare.sh                 |  25 --
>   ci/print-test-failures.sh             |  16 +-
>   ci/run-build-and-tests.sh             |  54 -----
>   ci/run-docker-build.sh                |  66 ------
>   ci/run-docker.sh                      |  47 ----
>   ci/run-static-analysis.sh             |  32 ---
>   ci/run-test-slice.sh                  |  17 --
>   ci/select-test-slice.sh               |  10 +
>   ci/test-documentation.sh              |  37 +--
>   ci/util/extract-trash-dirs.sh         |  50 ----
>   20 files changed, 345 insertions(+), 577 deletions(-)
>   create mode 100755 ci/check-unignored-build-artifacts.sh
>   delete mode 100755 ci/install-docker-dependencies.sh
>   create mode 100644 ci/lib-ci-type.sh
>   create mode 100644 ci/lib-tput.sh
>   delete mode 100755 ci/make-test-artifacts.sh
>   delete mode 100755 ci/mount-fileshare.sh
>   delete mode 100755 ci/run-build-and-tests.sh
>   delete mode 100755 ci/run-docker-build.sh
>   delete mode 100755 ci/run-docker.sh
>   delete mode 100755 ci/run-static-analysis.sh
>   delete mode 100755 ci/run-test-slice.sh
>   create mode 100755 ci/select-test-slice.sh
>   delete mode 100755 ci/util/extract-trash-dirs.sh
> 

Re: [PATCH 00/25] CI: run "make [test]" directly, use $GITHUB_ENV

[Junio C Hamano]

Phillip Wood <phillip.wood123@gmail.com> writes:

> I can see that showing compilation errors without having to expand
> anything is useful but in my experience they are relatively rare 
> compared to test failures. If I understand the rest of the message
> correctly we're left with having to expand print-test-failures and 
> searching for "not ok" to find out what went wrong with this series.

It matches my experience that most of the time I have to look for
test errors, not compilation errors.  Having to expand the other
group and looking for "not ok" is something I've got accustomed to,
so if that part improves, that would be great, but if that part does
not become worse, then that is OK, at least to me ;-)

Re: [PATCH 00/25] CI: run "make [test]" directly, use $GITHUB_ENV

[Ævar Arnfjörð Bjarmason]

On Wed, Feb 23 2022, Phillip Wood wrote:

> On 21/02/2022 14:46, Ævar Arnfjörð Bjarmason wrote:
>> [...]
>> == Details
>> This series conflicts with and is a proposed alternative to
>> Johannes's
>> proposed changes to teach "ci/lib.sh" GitHub specific syntax to
>> "group" certain parts of the output[2]. That series proposes to keep
>> the "ci/run-build-and-tests.sh" and similar scripts, but to teach them
>> to emit markers indicating when the "build" part commences, then
>> "test" etc.
>> I think the approach offered here is better as elaborated on in
>> 12/25[1], especially because as Johannes notes[3] we only have one
>> level of "grouping" within a "step" available to us. In his version
>> using it for "build" v.s. "test" precludes being able to group parts
>> of that "build" or "test" output in the future.
>> The following compares CI output between the three with a
>> change-on-top which caused a compilation error ("this" is this series,
>> "js" is Johannes's) when clicking on the "linux-gcc" failure:
>>    master:
>> https://github.com/avar/git/runs/5274251909?check_suite_focus=true
>>    this: https://github.com/avar/git/runs/5274464670?check_suite_focus=true
>>    js: https://github.com/avar/git/runs/5272239403?check_suite_focus=true
>>    Here we went from 93 lines of output on "master" to 47 in "this"
>>    (107 in "js"). Note also how in "this" we can:
>>    - Expand the "Run make" to get the "MAKEFLAGS" and other
>> variables
>>      used in the step. That's now a 3-line summary instead of the
>>      first 50 lines being the old "ci/lib.sh" "set -x" output.
>>    - Because "make" failed, we have an elided "make test" step
>> that's
>>      not being run below. We can thus see what steps our failure caused
>>      us to skip.
>>    - Unlike the "js" version we'll show the compilation error by
>>      default (the "js" version is grouped and needs to be expanded),
>>      but our output is now brief enough that that's no longer
>>      surrounded by other irrelevant output.
>>    - Unlike the "js" version there is no "CI setup" group within
>> each
>>      step. That work is in the earlier "ci/lib.sh --build" step
>>      instead, which sets the config for all subsequent steps.
>
> I can see that showing compilation errors without having to expand
> anything is useful but in my experience they are relatively rare 
> compared to test failures. If I understand the rest of the message
> correctly we're left with having to expand print-test-failures and 
> searching for "not ok" to find out what went wrong with this series.

Yes, it's a non-goal of this series to directly improve or change the
"prove" output, or to replace or change the functioning of
ci/print-test-failures.sh.

I think that's also a thing worth doing, it's just not what I'm aiming
for here. I.e. the goals of this & Johannes's series is only partial.

The above summary is comparing the parts of the two where those goals do
overlap, i.e. it's rougly comparable to patches 1-4/9 of Johannes's
series. His 5-9/9 are (mostly) orthagonal modifications to t/test-lib.sh
etc.

>> For other output changes look at the difference between "master" and
>> "this" at:
>>    master: https://github.com/git/git/actions/runs/1866786595
>>    this: https://github.com/avar/git/actions/runs/1876270588
>> Not explicitly covered in the summary above is that various other
>> parts of ci/* were doing the same sort of within-step setup, but now
>> do so via cross-step passing of variables via $GITHUB_ENV. E.g. on
>> "master" the test slices for the Windows tests have a lot of verbosity
>> before they get to running the test itself:
>>      https://github.com/git/git/runs/5254267914?check_suite_focus=true#step:5:56
>> In the "js" version the test output is hidden ("grouped"), but we've
>> got the same amount of verbosity by default:
>>      https://github.com/gitgitgadget/git/runs/4937491771?check_suite_focus=true#step:5:67
>> Whereas in "this" version that verbosity is in the preceding "select
>> tests" step, with the "test" step showing only the relevant end-state
>> of the "$T" variable we'll use in the Makefile (hidden by default,
>> expanded in this link):
>>      https://github.com/avar/git/runs/5274558869?check_suite_focus=true#step:7:13
>
> All I can see in the "$T" variable I have to scroll to get the prove
> output on screen

Yes, what that link is showing you is that for the test run the
variables that control the full set of variables that control the test
run are now prominently displayed.

All of the output etc. after that isn't changed.

The point of those comparisons is to draw your eyes to the part that
*is* changed v.s. master, and how using that variable passing mechanism
is making things related to that simpler & less verbose.

>> This series does not attempt to replace the use of the "group" output
>> used for the "ok" or "not ok" portion of tests in Johannes's
>> series. When a test fails the output in this series (sans config
>> discovery not being repeated, and now summarized in the $GITHUB_ENV
>> drop-down) is substantially the same as on "master".
>> My summary at [4] goes into other overlap & non-overlap between the
>> two. I think using the "group" output for those purposes might be
>> useful, although I left some feedback on [5] with problems in the
>> current "js" implementation.
>> I do think any such changes should follow behind this series, as
>> doing
>> that sort of grouping once we can effectively free up an extra "group"
>> level (by peeling those off into "steps", as is done here) would be
>> much more useful.
>
> I can see the attraction of being able to use make directly in the ci
> from an implementation point of view but from the point of view of 
> someone trying to investigate a test failure then unless I've
> misunderstood I don't think this series improves the current
> situation.

Johannes's series is making use of GitHub's group output with changes to
ci/lib.sh to e.g. distinguish the "make" phase from "make test".

This series shows that's not something that's needed to "group" those
two.

Instead we can set set the variables and other context that ci/lib.sh
currently sets at the start of every "step" in an earlier "step" in a
way that spans later steps.

Those later steps are then a light "make" or "make test", and because
steps are their own "group" it accomplishes the same goals for grouping
that part of the output, without needing any CI-specific syntax in ci/*.

Further, if we ever wanted to make use of such syntax for the
"ci/lib.sh" part we could do that on top of this series for
e.g. "grouping" individual errors that "make" emits.

Doing that wouldn't be possible with Johannes's approach, since there's
only one "group" level, you can't nest them. So using it for "build" and
"test" means you've already used up your one-level.

It's also a lot simpler, the resulting code size for the two in terms of
*.sh code in ci/ is:

    $ git grep . avar/ci-unroll-make-commands-to-ci-recipe 'ci/**.sh'|wc -l
    422
    $ git grep . pr-1117/dscho/use-grouping-in-ci-v1 'ci/**.sh'|wc -l
    721

> Why can't you build on top of Dscho's series that makes it
> easier to see the output of the failed tests?

Because building on top of it would mean adding a bunch of code and
complexity to ci/* to accomplish a goal that this series shows that we
can equally get without that complexity and added code, as the diffstat
etc. shows.

What we do about t/test-lib.sh, "prove" etc. emitting summary output
about tests is then a mostly orthagonal change on top of that.

"Mostly" except insofar that Johannes's series notes that he's left with
the limitation[3] that "sadly, workflow output currently cannot contain
any nested groups".

Of course that's also true after this series (that GitHub CI feature
will be the same).

But the difference is that since we won't have used the "group" for
"make" and "make test" (as it's "kicked up" to the "step" level) a
series to add "group"-ing to the test output wouldn't have the group for
an individual test failure at the same level as "make test".

>> 1. https://lore.kernel.org/git/patch-12.25-dfd823f2e7d-20220221T122605Z-avarab@gmail.com
>> 2. https://lore.kernel.org/git/pull.1117.git.1643050574.gitgitgadget@gmail.com/
>> 3. https://lore.kernel.org/git/9333ba781b8240f704e739b00d274f8c3d887e39.1643050574.git.gitgitgadget@gmail.com/
>> 4. https://lore.kernel.org/git/220127.86ilu5cdnf.gmgdl@evledraar.gmail.com/
>> 5. https://lore.kernel.org/git/220126.86sftbfjl4.gmgdl@evledraar.gmail.com/
>> Ævar Arnfjörð Bjarmason (25):
>>    CI: run "set -ex" early in ci/lib.sh
>>    CI: make "$jobname" explicit, remove fallback
>>    CI: remove more dead Travis CI support
>>    CI: remove dead "tree skipping" code
>>    CI: remove unused Azure ci/* code
>>    CI: don't have "git grep" invoke a pager in tree content check
>>    CI: have "static-analysis" run a "make ci-static-analysis" target
>>    CI: have "static-analysis" run "check-builtins", not "documentation"
>>    CI: move p4 and git-lfs variables to ci/install-dependencies.sh
>>    CI: consistently use "export" in ci/lib.sh
>>    CI: export variables via a wrapper
>>    CI: remove "run-build-and-tests.sh", run "make [test]" directly
>>    CI: check ignored unignored build artifacts in "win[+VS] build" too
>>    CI: invoke "make artifacts-tar" directly in windows-build
>>    CI: split up and reduce "ci/test-documentation.sh"
>>    CI: combine ci/install{,-docker}-dependencies.sh
>>    CI: move "env" definitions into ci/lib.sh
>>    ci/run-test-slice.sh: replace shelling out with "echo"
>>    CI: pre-select test slice in Windows & VS tests
>>    CI: only invoke ci/lib.sh as "steps" in main.yml
>>    CI: narrow down variable definitions in --build and --test
>>    CI: add more variables to MAKEFLAGS, except under vs-build
>>    CI: stop over-setting the $CC variable
>>    CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case
>>    CI: don't use "set -x" in "ci/lib.sh" output
>>   .github/workflows/main.yml            | 100 +++++---
>>   Makefile                              |  32 ++-
>>   ci/check-directional-formatting.bash  |   2 +-
>>   ci/check-unignored-build-artifacts.sh |  20 ++
>>   ci/install-dependencies.sh            |  53 ++++-
>>   ci/install-docker-dependencies.sh     |  22 --
>>   ci/lib-ci-type.sh                     |  10 +
>>   ci/lib-tput.sh                        |   2 +
>>   ci/lib.sh                             | 315 ++++++++++++--------------
>>   ci/make-test-artifacts.sh             |  12 -
>>   ci/mount-fileshare.sh                 |  25 --
>>   ci/print-test-failures.sh             |  16 +-
>>   ci/run-build-and-tests.sh             |  54 -----
>>   ci/run-docker-build.sh                |  66 ------
>>   ci/run-docker.sh                      |  47 ----
>>   ci/run-static-analysis.sh             |  32 ---
>>   ci/run-test-slice.sh                  |  17 --
>>   ci/select-test-slice.sh               |  10 +
>>   ci/test-documentation.sh              |  37 +--
>>   ci/util/extract-trash-dirs.sh         |  50 ----
>>   20 files changed, 345 insertions(+), 577 deletions(-)
>>   create mode 100755 ci/check-unignored-build-artifacts.sh
>>   delete mode 100755 ci/install-docker-dependencies.sh
>>   create mode 100644 ci/lib-ci-type.sh
>>   create mode 100644 ci/lib-tput.sh
>>   delete mode 100755 ci/make-test-artifacts.sh
>>   delete mode 100755 ci/mount-fileshare.sh
>>   delete mode 100755 ci/run-build-and-tests.sh
>>   delete mode 100755 ci/run-docker-build.sh
>>   delete mode 100755 ci/run-docker.sh
>>   delete mode 100755 ci/run-static-analysis.sh
>>   delete mode 100755 ci/run-test-slice.sh
>>   create mode 100755 ci/select-test-slice.sh
>>   delete mode 100755 ci/util/extract-trash-dirs.sh
>> 

Re: [PATCH 00/25] CI: run "make [test]" directly, use $GITHUB_ENV

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1016 bytes --]

Hi Ævar,

On Wed, 23 Feb 2022, Ævar Arnfjörð Bjarmason wrote:

> On Wed, Feb 23 2022, Phillip Wood wrote:
>
> > If I understand the rest of the message correctly we're left with
> > having to expand print-test-failures and searching for "not ok" to
> > find out what went wrong with this series.
>
> Yes, it's a non-goal of this series to directly improve or change the
> "prove" output, or to replace or change the functioning of
> ci/print-test-failures.sh.

Since it is explicitly _not_ the goal of this series, but of the series I
offered for review in
https://lore.kernel.org/git/pull.1117.v2.git.1646130289.gitgitgadget@gmail.com/
(with which this here patch series conflicts rather intentionally), I
suggest that your patch series can wait a little longer and build on top
of mine.

We do have the rule in the Git project to try to avoid conflicting with
patch series that are already in flight. It just makes the lives of all
involved parties so much easier.

Thanks,
Johannes

[PATCH v2 00/25] CI: run "make [test]" directly, use $GITHUB_ENV

[Ævar Arnfjörð Bjarmason]

A re-roll of my improvements my series to simplify the CI setup a lot
(see diffstat), much of it was dealing with constraints that went away
with Travis et al. CI for this series (OSX runners failing for
unrelated reasons):

    https://github.com/avar/git/actions/runs/2040223909

For a much more detailed summary of how the output looks before/after
see v1[].

This series heavily conflicts with Johannes's
js/ci-github-workflow-markup in "seen", but in the v1 I suggested
basing that series on top of this one, because it can benefit a lot
from these simplifications.

I'll reply to this series with a proposed rebasing of that series on
top of this one, which allows for removing almost all of its changes
to "ci/" with no harm to its end-goals, i.e. the splitting up of
"make" and "make test" output is something it'll get for free from
this series.

Junio: Since that series has been stalled on still-outstanding
performance issues for a couple of months I was hoping we could queue
this instead, and perhaps in addition if Johannes approves of the
proposed re-roll on top of his.

There's some forward progress on the performance issues (this[2] reply
of Victoria Dye's from yesterday), but fully resolving those will
probably take a bit...

Whereas even though this one is relatively large I don't think there's
anything controversial here. The one concern that's been raised has
been Johannes's objection to removing some of the dead Azure code
(which was needed to move forward here). I asked how he'd prefer to
move forward with that in [3], but there hasn't been a reply to that
in >1 month.

I think just removing it is OK, we can always bring it back if needed,
and doing so is actually going to be simpler on top of this since the
CI is now less special, and leans more heavily on the logic of our
normal build process.

1. https://lore.kernel.org/git/cover-00.25-00000000000-20220221T143936Z-avarab@gmail.com
2. https://lore.kernel.org/git/6b83bb83-32b9-20c9-fa02-c1c3170351c3@github.com/
3. https://lore.kernel.org/git/220222.86y2236ndp.gmgdl@evledraar.gmail.com/

Ævar Arnfjörð Bjarmason (25):
  CI: run "set -ex" early in ci/lib.sh
  CI: make "$jobname" explicit, remove fallback
  CI: remove more dead Travis CI support
  CI: remove dead "tree skipping" code
  CI: remove unused Azure ci/* code
  CI: don't have "git grep" invoke a pager in tree content check
  CI: have "static-analysis" run a "make ci-static-analysis" target
  CI: have "static-analysis" run "check-builtins", not "documentation"
  CI: move p4 and git-lfs variables to ci/install-dependencies.sh
  CI: consistently use "export" in ci/lib.sh
  CI: export variables via a wrapper
  CI: remove "run-build-and-tests.sh", run "make [test]" directly
  CI: check ignored unignored build artifacts in "win[+VS] build" too
  CI: invoke "make artifacts-tar" directly in windows-build
  CI: split up and reduce "ci/test-documentation.sh"
  CI: combine ci/install{,-docker}-dependencies.sh
  CI: move "env" definitions into ci/lib.sh
  ci/run-test-slice.sh: replace shelling out with "echo"
  CI: pre-select test slice in Windows & VS tests
  CI: only invoke ci/lib.sh as "steps" in main.yml
  CI: narrow down variable definitions in --build and --test
  CI: add more variables to MAKEFLAGS, except under vs-build
  CI: set CC in MAKEFLAGS directly, don't add it to the environment
  CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case
  CI: don't use "set -x" in "ci/lib.sh" output

 .github/workflows/main.yml            | 100 +++++---
 Makefile                              |  31 ++-
 ci/check-directional-formatting.bash  |   2 +-
 ci/check-unignored-build-artifacts.sh |  20 ++
 ci/install-dependencies.sh            |  53 ++++-
 ci/install-docker-dependencies.sh     |  22 --
 ci/lib-ci-type.sh                     |  10 +
 ci/lib-tput.sh                        |   2 +
 ci/lib.sh                             | 316 +++++++++++++-------------
 ci/make-test-artifacts.sh             |  12 -
 ci/mount-fileshare.sh                 |  25 --
 ci/print-test-failures.sh             |  16 +-
 ci/run-build-and-tests.sh             |  54 -----
 ci/run-docker-build.sh                |  66 ------
 ci/run-docker.sh                      |  47 ----
 ci/run-static-analysis.sh             |  32 ---
 ci/run-test-slice.sh                  |  17 --
 ci/select-test-slice.sh               |  10 +
 ci/test-documentation.sh              |  37 +--
 ci/util/extract-trash-dirs.sh         |  50 ----
 shared.mak                            |   1 +
 21 files changed, 346 insertions(+), 577 deletions(-)
 create mode 100755 ci/check-unignored-build-artifacts.sh
 delete mode 100755 ci/install-docker-dependencies.sh
 create mode 100644 ci/lib-ci-type.sh
 create mode 100644 ci/lib-tput.sh
 delete mode 100755 ci/make-test-artifacts.sh
 delete mode 100755 ci/mount-fileshare.sh
 delete mode 100755 ci/run-build-and-tests.sh
 delete mode 100755 ci/run-docker-build.sh
 delete mode 100755 ci/run-docker.sh
 delete mode 100755 ci/run-static-analysis.sh
 delete mode 100755 ci/run-test-slice.sh
 create mode 100755 ci/select-test-slice.sh
 delete mode 100755 ci/util/extract-trash-dirs.sh

Range-diff against v1:
 1:  970849a227f =  1:  4addbd70213 CI: run "set -ex" early in ci/lib.sh
 2:  eda7fb18064 =  2:  b23aa99fd37 CI: make "$jobname" explicit, remove fallback
 3:  3ee32815cf3 =  3:  eec15a95879 CI: remove more dead Travis CI support
 4:  c81c1b3f504 =  4:  73c894f1665 CI: remove dead "tree skipping" code
 5:  4738a22a36d =  5:  b5e6d538554 CI: remove unused Azure ci/* code
 6:  59e4f41e86c =  6:  a4b9febbdca CI: don't have "git grep" invoke a pager in tree content check
 7:  836ef20fdcc !  7:  5d46d5b34c9 CI: have "static-analysis" run a "make ci-static-analysis" target
    @@ .github/workflows/main.yml: jobs:
          if: needs.ci-config.outputs.enabled == 'yes'
     
      ## Makefile ##
    -@@ Makefile: ifndef V
    - 	QUIET_HDR      = @echo '   ' HDR $(<:hcc=h);
    - 	QUIET_RC       = @echo '   ' RC $@;
    - 	QUIET_SPATCH   = @echo '   ' SPATCH $<;
    -+	QUIET_CHECK    = @echo '   ' CHECK $@;
    - 	QUIET_SUBDIR0  = +@subdir=
    - 	QUIET_SUBDIR1  = ;$(NO_SUBDIR) echo '   ' SUBDIR $$subdir; \
    - 			 $(MAKE) $(PRINT_DIR) -C $$subdir
     @@ Makefile: coccicheck: $(addsuffix .patch,$(filter-out %.pending.cocci,$(wildcard contrib/c
      # See contrib/coccinelle/README
      coccicheck-pending: $(addsuffix .patch,$(wildcard contrib/coccinelle/*.pending.cocci))
    @@ ci/run-static-analysis.sh (deleted)
     -
     -make hdr-check ||
     -exit 1
    +
    + ## shared.mak ##
    +@@ shared.mak: ifndef V
    + 	QUIET_HDR      = @echo '   ' HDR $(<:hcc=h);
    + 	QUIET_RC       = @echo '   ' RC $@;
    + 	QUIET_SPATCH   = @echo '   ' SPATCH $<;
    ++	QUIET_CHECK    = @echo '   ' CHECK $@;
    + 
    + ## Used in "Documentation/Makefile"
    + 	QUIET_ASCIIDOC	= @echo '   ' ASCIIDOC $@;
 8:  95cd496868e =  8:  81e06f13d84 CI: have "static-analysis" run "check-builtins", not "documentation"
 9:  a1d0796259e =  9:  3be91c26d44 CI: move p4 and git-lfs variables to ci/install-dependencies.sh
10:  b6a61a786c5 = 10:  9dc148341ba CI: consistently use "export" in ci/lib.sh
11:  f2fcee5d6e4 = 11:  e9c7ba492e8 CI: export variables via a wrapper
12:  dfd823f2e7d = 12:  86d5a48d59a CI: remove "run-build-and-tests.sh", run "make [test]" directly
13:  46459fff296 = 13:  649ad1ae249 CI: check ignored unignored build artifacts in "win[+VS] build" too
14:  aecd3ebaafe = 14:  b1b5b083389 CI: invoke "make artifacts-tar" directly in windows-build
15:  4f1564af70f = 15:  dfa91ac8938 CI: split up and reduce "ci/test-documentation.sh"
16:  868613a5b06 = 16:  ba4ed216769 CI: combine ci/install{,-docker}-dependencies.sh
17:  5d854e8ff36 = 17:  b5e89a33340 CI: move "env" definitions into ci/lib.sh
18:  a6106525b7f = 18:  571f4d0f441 ci/run-test-slice.sh: replace shelling out with "echo"
19:  e9c6c4dd293 = 19:  2edea06ee4d CI: pre-select test slice in Windows & VS tests
20:  40d86e8c1dc = 20:  ef9daa6882f CI: only invoke ci/lib.sh as "steps" in main.yml
21:  abf9c504740 = 21:  44e3ace5fbe CI: narrow down variable definitions in --build and --test
22:  9f4c2798a82 = 22:  5f56b922e08 CI: add more variables to MAKEFLAGS, except under vs-build
23:  8a8b7ecf16b ! 23:  b45b7cec94e CI: stop over-setting the $CC variable
    @@ Metadata
     Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## Commit message ##
    -    CI: stop over-setting the $CC variable
    +    CI: set CC in MAKEFLAGS directly, don't add it to the environment
     
    -    As detailed in 2c8921db2b8 (travis-ci: build with the right compiler,
    -    2019-01-17) the reason we started using $CC in $MAKEFLAGS as opposed
    -    to setting it in the environment was due to Travis CI clobbering $CC
    -    in the environment.
    +    Rather than pass a "$CC" and "$CC_PACKAGE" in the environment to be
    +    picked up in ci/lib.sh let's instead have ci/lib.sh itself add it
    +    directly to MAKEFLAGS.
     
    -    We don't need to set it unconditionally to accomplish that, but rather
    -    just have it set for those jobs that need them. E.g. the "win+VS
    -    build" job confusingly has CC=gcc set, even though it builds with
    -    MSVC.
    +    Setting CC=gcc by default made for confusing trace output, and since a
    +    preceding change to carry it and others over across "steps" in the
    +    GitHub CI it's been even more misleading.  E.g. the "win+VS build" job
    +    confusingly has CC=gcc set, even though it builds with MSVC.
    +
    +    Let's instead reply on the Makefile default of CC=cc, and only
    +    override it for those jobs where it's needed. This does mean that
    +    we'll need to set it for the "pedantic" job, which previously relied
    +    on the default CC=gcc in case "clang" become the default on that
    +    platform.
     
         This partially reverts my 707d2f2fe86 (CI: use "$runs_on_pool", not
         "$jobname" to select packages & config, 2021-11-23), i.e. we're now
    @@ ci/lib.sh: linux-TEST-vars)
      	setenv --test GIT_TEST_DEFAULT_HASH sha256
      	;;
      pedantic)
    ++	CC=gcc
    + 	# Don't run the tests; we only care about whether Git can be
    + 	# built.
    + 	setenv --build DEVOPTS pedantic
     @@ ci/lib.sh: linux-musl)
      	MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8"
      	;;
24:  d7b472b4a52 = 24:  06bf8d9f61b CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case
25:  08a9776c259 = 25:  6dc96ba8b82 CI: don't use "set -x" in "ci/lib.sh" output
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 01/25] CI: run "set -ex" early in ci/lib.sh

[Ævar Arnfjörð Bjarmason]

Change ci/lib.sh to run "set -ex" before it does anything else, before
this we were omitting a few "export" commands in our trace output that
happened before we enabled tracing.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index cbc2f8f1caa..57141d38a85 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,5 +1,11 @@
 # Library of functions shared by all CI scripts
 
+# Set 'exit on error' for all CI scripts to let the caller know that
+# something went wrong.
+# Set tracing executed commands, primarily setting environment variables
+# and installing dependencies.
+set -ex
+
 skip_branch_tip_with_tag () {
 	# Sometimes, a branch is pushed at the same time the tag that points
 	# at the same commit as the tip of the branch is pushed, and building
@@ -85,12 +91,6 @@ export TERM=${TERM:-dumb}
 # Clear MAKEFLAGS that may come from the outside world.
 export MAKEFLAGS=
 
-# Set 'exit on error' for all CI scripts to let the caller know that
-# something went wrong.
-# Set tracing executed commands, primarily setting environment variables
-# and installing dependencies.
-set -ex
-
 if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
 then
 	CI_TYPE=azure-pipelines
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 02/25] CI: make "$jobname" explicit, remove fallback

[Ævar Arnfjörð Bjarmason]

Remove the $jobname fallback last adjusted in b011fabd6e0 (ci/lib.sh:
encapsulate Travis-specific things, 2019-01-27).

This makes all of our CI jobs have an explicit $jobname, and removes
the special-case of the Windows jobs implicitly setting one (that they
don't currently use). In subsequent commits we'll use this new
$jobname explicitly.

This is a follow-up to my 25715419bf4 (CI: don't run "make test" twice
in one job, 2021-11-23), since then all the jobs except the Windows
jobs have had an explicit $jobname.

In the case of the Windows jobs we were using a jobname of windows-gcc
for both, due to picking "gcc" as a default fallback for "$CC" in
ci/lib.sh.

Since we'll need this to run ci/lib.sh it makes sense to move the
check to the top of the file. Subsequent commits will add more such
assertions to it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml |  8 ++++++++
 ci/lib.sh                  | 12 +++++++-----
 2 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index c35200defb9..309b9141249 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -79,6 +79,8 @@ jobs:
 
   windows-build:
     name: win build
+    env:
+      jobname: windows-build
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
     runs-on: windows-latest
@@ -100,6 +102,8 @@ jobs:
         path: artifacts
   windows-test:
     name: win test
+    env:
+      jobname: windows-test
     runs-on: windows-latest
     needs: [windows-build]
     strategy:
@@ -131,6 +135,8 @@ jobs:
         path: ${{env.FAILED_TEST_ARTIFACTS}}
   vs-build:
     name: win+VS build
+    env:
+      jobname: vs-build
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
     env:
@@ -183,6 +189,8 @@ jobs:
         path: artifacts
   vs-test:
     name: win+VS test
+    env:
+      jobname: vs-test
     runs-on: windows-latest
     needs: vs-build
     strategy:
diff --git a/ci/lib.sh b/ci/lib.sh
index 57141d38a85..d2a7c33f536 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -6,6 +6,13 @@
 # and installing dependencies.
 set -ex
 
+# Starting assertions
+if test -z "$jobname"
+then
+	echo "must set a CI jobname" >&2
+	exit 1
+fi
+
 skip_branch_tip_with_tag () {
 	# Sometimes, a branch is pushed at the same time the tag that points
 	# at the same commit as the tip of the branch is pushed, and building
@@ -146,11 +153,6 @@ test -n "${DONT_SKIP_TAGS-}" ||
 skip_branch_tip_with_tag
 skip_good_tree
 
-if test -z "$jobname"
-then
-	jobname="$CI_OS_NAME-$CC"
-fi
-
 export DEVELOPER=1
 export DEFAULT_TEST_TARGET=prove
 export GIT_TEST_CLONE_2GB=true
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 03/25] CI: remove more dead Travis CI support

[Ævar Arnfjörð Bjarmason]

Remove code that I missed in 4a6e4b96026 (CI: remove Travis CI
support, 2021-11-23). This code was only called from or used by the
now-removed .travis.yml, or needed by the Travis CI environment.

For the symlinking in ci/run-build-and-tests.sh: Back when
3c93b829205 (travis-ci: build Git during the 'script' phase,
2018-01-08) and 4b060a4d973 (ci: use a junction on Windows instead of
a symlink, 2019-01-27) adjusted this "ln" command, the Windows build
would use ci/run-build-and-tests.sh.

As seen in 889cacb6897 (ci: configure GitHub Actions for CI/PR,
2020-04-11) the current windows build uses a different entry point
under the GitHub CI, which doesn't use this .prove caching. Namely
"ci/run-test-slice.sh".

We can be certain that it's never used in "ci/run-test-slice.sh"
because to have a ".prove" file we'd need to use "--state=save", which
we only do in the dead Azure codepath in ci/lib.sh. If it were used it
would do the wrong thing, because the different test slices would each
try to clobber the same "t/.prove" file.

If a subsequent run then used the -"-state=failed,slow,save" it would
defeat the purpose of "ci/run-test-slice.sh", since all slices would
then run all tests. I.e. behavior of prove's "--state" options is to
select tests to run from the provided "--state" file, in addition to
those specified on the command-line.

For ci/run-docker{,-build}.sh: It was likewise last referenced in the
.travis.yml removed in my 4a6e4b96026. The current "dockerized" run in
".github/workflows/main.yml" calls the same entry points as the main
"regular" job.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                     | 10 ++----
 ci/print-test-failures.sh     |  3 --
 ci/run-build-and-tests.sh     |  5 ---
 ci/run-docker-build.sh        | 66 -----------------------------------
 ci/run-docker.sh              | 47 -------------------------
 ci/run-test-slice.sh          |  5 ---
 ci/util/extract-trash-dirs.sh | 50 --------------------------
 7 files changed, 2 insertions(+), 184 deletions(-)
 delete mode 100755 ci/run-docker-build.sh
 delete mode 100755 ci/run-docker.sh
 delete mode 100755 ci/util/extract-trash-dirs.sh

diff --git a/ci/lib.sh b/ci/lib.sh
index d2a7c33f536..c3c06d66862 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -106,9 +106,6 @@ then
 	CI_COMMIT="$BUILD_SOURCEVERSION"
 	CI_JOB_ID="$BUILD_BUILDID"
 	CI_JOB_NUMBER="$BUILD_BUILDNUMBER"
-	CI_OS_NAME="$(echo "$AGENT_OS" | tr A-Z a-z)"
-	test darwin != "$CI_OS_NAME" || CI_OS_NAME=osx
-	CI_REPO_SLUG="$(expr "$BUILD_REPOSITORY_URI" : '.*/\([^/]*/[^/]*\)$')"
 	CC="${CC:-gcc}"
 
 	# use a subdirectory of the cache dir (because the file share is shared
@@ -118,16 +115,13 @@ then
 	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
-	test windows_nt != "$CI_OS_NAME" ||
+	test Windows_NT != "$AGENT_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 elif test true = "$GITHUB_ACTIONS"
 then
 	CI_TYPE=github-actions
 	CI_BRANCH="$GITHUB_REF"
 	CI_COMMIT="$GITHUB_SHA"
-	CI_OS_NAME="$(echo "$RUNNER_OS" | tr A-Z a-z)"
-	test macos != "$CI_OS_NAME" || CI_OS_NAME=osx
-	CI_REPO_SLUG="$GITHUB_REPOSITORY"
 	CI_JOB_ID="$GITHUB_RUN_ID"
 	CC="${CC:-gcc}"
 	DONT_SKIP_TAGS=t
@@ -137,7 +131,7 @@ then
 	export GIT_PROVE_OPTS="--timer --jobs 10"
 	export GIT_TEST_OPTS="--verbose-log -x"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
-	test windows != "$CI_OS_NAME" ||
+	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 else
 	echo "Could not identify CI type" >&2
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 57277eefcd0..00fcf27b1b2 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -77,9 +77,6 @@ do
 			fi
 			combined_trash_size=$new_combined_trash_size
 
-			# DO NOT modify these two 'echo'-ed strings below
-			# without updating 'ci/util/extract-trash-dirs.sh'
-			# as well.
 			echo "$(tput setaf 1)Start of trash directory of '$test_name':$(tput sgr0)"
 			cat "$trash_tgz_b64"
 			echo "$(tput setaf 1)End of trash directory of '$test_name'$(tput sgr0)"
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 280dda7d285..0a1ec8c2bae 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -5,11 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-case "$CI_OS_NAME" in
-windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
-*) ln -s "$cache_dir/.prove" t/.prove;;
-esac
-
 export MAKE_TARGETS="all test"
 
 case "$jobname" in
diff --git a/ci/run-docker-build.sh b/ci/run-docker-build.sh
deleted file mode 100755
index 6cd832efb9c..00000000000
--- a/ci/run-docker-build.sh
+++ /dev/null
@@ -1,66 +0,0 @@
-#!/bin/sh
-#
-# Build and test Git inside container
-#
-# Usage:
-#   run-docker-build.sh <host-user-id>
-#
-
-set -ex
-
-if test $# -ne 1 || test -z "$1"
-then
-	echo >&2 "usage: run-docker-build.sh <host-user-id>"
-	exit 1
-fi
-
-case "$jobname" in
-linux32)
-	switch_cmd="linux32 --32bit i386"
-	;;
-linux-musl)
-	switch_cmd=
-	useradd () { adduser -D "$@"; }
-	;;
-*)
-	exit 1
-	;;
-esac
-
-"${0%/*}/install-docker-dependencies.sh"
-
-# If this script runs inside a docker container, then all commands are
-# usually executed as root. Consequently, the host user might not be
-# able to access the test output files.
-# If a non 0 host user id is given, then create a user "ci" with that
-# user id to make everything accessible to the host user.
-HOST_UID=$1
-if test $HOST_UID -eq 0
-then
-	# Just in case someone does want to run the test suite as root.
-	CI_USER=root
-else
-	CI_USER=ci
-	if test "$(id -u $CI_USER 2>/dev/null)" = $HOST_UID
-	then
-		echo "user '$CI_USER' already exists with the requested ID $HOST_UID"
-	else
-		useradd -u $HOST_UID $CI_USER
-	fi
-fi
-
-# Build and test
-command $switch_cmd su -m -l $CI_USER -c "
-	set -ex
-	export DEVELOPER='$DEVELOPER'
-	export DEFAULT_TEST_TARGET='$DEFAULT_TEST_TARGET'
-	export GIT_PROVE_OPTS='$GIT_PROVE_OPTS'
-	export GIT_TEST_OPTS='$GIT_TEST_OPTS'
-	export GIT_TEST_CLONE_2GB='$GIT_TEST_CLONE_2GB'
-	export MAKEFLAGS='$MAKEFLAGS'
-	export cache_dir='$cache_dir'
-	cd /usr/src/git
-	test -n '$cache_dir' && ln -s '$cache_dir/.prove' t/.prove
-	make
-	make test
-"
diff --git a/ci/run-docker.sh b/ci/run-docker.sh
deleted file mode 100755
index af89d1624a4..00000000000
--- a/ci/run-docker.sh
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/bin/sh
-#
-# Download and run Docker image to build and test Git
-#
-
-. ${0%/*}/lib.sh
-
-case "$jobname" in
-linux32)
-	CI_CONTAINER="daald/ubuntu32:xenial"
-	;;
-linux-musl)
-	CI_CONTAINER=alpine
-	;;
-*)
-	exit 1
-	;;
-esac
-
-docker pull "$CI_CONTAINER"
-
-# Use the following command to debug the docker build locally:
-# <host-user-id> must be 0 if podman is used as drop-in replacement for docker
-# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/sh "$CI_CONTAINER"
-# root@container:/# export jobname=<jobname>
-# root@container:/# /usr/src/git/ci/run-docker-build.sh <host-user-id>
-
-container_cache_dir=/tmp/container-cache
-
-docker run \
-	--interactive \
-	--env DEVELOPER \
-	--env DEFAULT_TEST_TARGET \
-	--env GIT_PROVE_OPTS \
-	--env GIT_TEST_OPTS \
-	--env GIT_TEST_CLONE_2GB \
-	--env MAKEFLAGS \
-	--env jobname \
-	--env cache_dir="$container_cache_dir" \
-	--volume "${PWD}:/usr/src/git" \
-	--volume "$cache_dir:$container_cache_dir" \
-	"$CI_CONTAINER" \
-	/usr/src/git/ci/run-docker-build.sh $(id -u $USER)
-
-check_unignored_build_artifacts
-
-save_good_tree
diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index f8c2c3106a2..b9a682b4bcd 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -5,11 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-case "$CI_OS_NAME" in
-windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
-*) ln -s "$cache_dir/.prove" t/.prove;;
-esac
-
 make --quiet -C t T="$(cd t &&
 	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
 	tr '\n' ' ')"
diff --git a/ci/util/extract-trash-dirs.sh b/ci/util/extract-trash-dirs.sh
deleted file mode 100755
index 8e67bec21a2..00000000000
--- a/ci/util/extract-trash-dirs.sh
+++ /dev/null
@@ -1,50 +0,0 @@
-#!/bin/sh
-
-error () {
-	echo >&2 "error: $@"
-	exit 1
-}
-
-find_embedded_trash () {
-	while read -r line
-	do
-		case "$line" in
-		*Start\ of\ trash\ directory\ of\ \'t[0-9][0-9][0-9][0-9]-*\':*)
-			test_name="${line#*\'}"
-			test_name="${test_name%\'*}"
-
-			return 0
-		esac
-	done
-
-	return 1
-}
-
-extract_embedded_trash () {
-	while read -r line
-	do
-		case "$line" in
-		*End\ of\ trash\ directory\ of\ \'$test_name\'*)
-			return
-			;;
-		*)
-			printf '%s\n' "$line"
-			;;
-		esac
-	done
-
-	error "unexpected end of input"
-}
-
-# Raw logs from Linux build jobs have CRLF line endings, while OSX
-# build jobs mostly have CRCRLF, except an odd line every now and
-# then that has CRCRCRLF.  'base64 -d' from 'coreutils' doesn't like
-# CRs and complains about "invalid input", so remove all CRs at the
-# end of lines.
-sed -e 's/\r*$//' | \
-while find_embedded_trash
-do
-	echo "Extracting trash directory of '$test_name'"
-
-	extract_embedded_trash |base64 -d |tar xzp
-done
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 04/25] CI: remove dead "tree skipping" code

[Ævar Arnfjörð Bjarmason]

Remove the code related to skipping CI runs if the tree was
successfully tested before. With the GitHub CI this is done with the
"ci-config" step, see e76eec35540 (ci: allow per-branch config for
GitHub Actions, 2020-05-07) and 7d78d5fc1a9 (ci: skip GitHub workflow
runs for already-tested commits/trees, 2020-10-08).

This code hasn't been used since 4a6e4b96026 (CI: remove Travis CI
support, 2021-11-23), and before that for the retired Azure pipeline
support removed in 6081d3898fe (ci: retire the Azure Pipelines
definition, 2020-04-11).

This change is needed because this and subsequent commits are turning
"ci/lib.sh" into a dumber library that'll only be tasked with setting
variables for CI jobs, or for specific steps within those jobs. Now we
don't need to worry about it potentially skipping the run on its own.

This change also removes a subtle potential logic error introduced in
0e7696c64db (ci: disallow directional formatting, 2021-11-04). The
"ci/check-directional-formatting.bash" script would have been made to
run after the "save_good_tree" in invoked in
"ci/run-static-analysis.sh". So if e.g. the "make coccicheck" failed
we'd still mark the tree as good.

I.e. the addition of "ci/check-directional-formatting.bash" didn't
take into account that the various users of the "save_good_tree"
function in ci/*.sh made a hard assumption that they're the only
ci/*.sh script that's being run (and they use "set -e").

This is entirely academic since we weren't actually running this code,
but is something to be careful of if anyone ever needs to resurrect
parts of this.

Finally, the cache_dir="$HOME/none" added in a3f2eec862b (ci/lib:
allow running in GitHub Actions, 2020-04-08) for the GitHub CI can be
removed. It appears to have been added to appease the subsequent
'mkdir -p "$cache_dir"', which wasn't needed in GitHub CI.

1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                 | 91 ---------------------------------------
 ci/run-build-and-tests.sh |  2 -
 ci/run-static-analysis.sh |  2 -
 ci/test-documentation.sh  |  2 -
 4 files changed, 97 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index c3c06d66862..5a8938d4b3c 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -13,75 +13,6 @@ then
 	exit 1
 fi
 
-skip_branch_tip_with_tag () {
-	# Sometimes, a branch is pushed at the same time the tag that points
-	# at the same commit as the tip of the branch is pushed, and building
-	# both at the same time is a waste.
-	#
-	# When the build is triggered by a push to a tag, $CI_BRANCH will
-	# have that tagname, e.g. v2.14.0.  Let's see if $CI_BRANCH is
-	# exactly at a tag, and if so, if it is different from $CI_BRANCH.
-	# That way, we can tell if we are building the tip of a branch that
-	# is tagged and we can skip the build because we won't be skipping a
-	# build of a tag.
-
-	if TAG=$(git describe --exact-match "$CI_BRANCH" 2>/dev/null) &&
-		test "$TAG" != "$CI_BRANCH"
-	then
-		echo "$(tput setaf 2)Tip of $CI_BRANCH is exactly at $TAG$(tput sgr0)"
-		exit 0
-	fi
-}
-
-# Save some info about the current commit's tree, so we can skip the build
-# job if we encounter the same tree again and can provide a useful info
-# message.
-save_good_tree () {
-	echo "$(git rev-parse $CI_COMMIT^{tree}) $CI_COMMIT $CI_JOB_NUMBER $CI_JOB_ID" >>"$good_trees_file"
-	# limit the file size
-	tail -1000 "$good_trees_file" >"$good_trees_file".tmp
-	mv "$good_trees_file".tmp "$good_trees_file"
-}
-
-# Skip the build job if the same tree has already been built and tested
-# successfully before (e.g. because the branch got rebased, changing only
-# the commit messages).
-skip_good_tree () {
-	if test true = "$GITHUB_ACTIONS"
-	then
-		return
-	fi
-
-	if ! good_tree_info="$(grep "^$(git rev-parse $CI_COMMIT^{tree}) " "$good_trees_file")"
-	then
-		# Haven't seen this tree yet, or no cached good trees file yet.
-		# Continue the build job.
-		return
-	fi
-
-	echo "$good_tree_info" | {
-		read tree prev_good_commit prev_good_job_number prev_good_job_id
-
-		if test "$CI_JOB_ID" = "$prev_good_job_id"
-		then
-			cat <<-EOF
-			$(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0)
-			This commit has already been built and tested successfully by this build job.
-			To force a re-build delete the branch's cache and then hit 'Restart job'.
-			EOF
-		else
-			cat <<-EOF
-			$(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0)
-			This commit's tree has already been built and tested successfully in build job $prev_good_job_number for commit $prev_good_commit.
-			The log of that build job is available at $SYSTEM_TASKDEFINITIONSURI$SYSTEM_TEAMPROJECT/_build/results?buildId=$prev_good_job_id
-			To force a re-build delete the branch's cache and then hit 'Restart job'.
-			EOF
-		fi
-	}
-
-	exit 0
-}
-
 check_unignored_build_artifacts ()
 {
 	! git ls-files --other --exclude-standard --error-unmatch \
@@ -102,16 +33,8 @@ if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
 then
 	CI_TYPE=azure-pipelines
 	# We are running in Azure Pipelines
-	CI_BRANCH="$BUILD_SOURCEBRANCH"
-	CI_COMMIT="$BUILD_SOURCEVERSION"
-	CI_JOB_ID="$BUILD_BUILDID"
-	CI_JOB_NUMBER="$BUILD_BUILDNUMBER"
 	CC="${CC:-gcc}"
 
-	# use a subdirectory of the cache dir (because the file share is shared
-	# among *all* phases)
-	cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME"
-
 	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
@@ -120,13 +43,7 @@ then
 elif test true = "$GITHUB_ACTIONS"
 then
 	CI_TYPE=github-actions
-	CI_BRANCH="$GITHUB_REF"
-	CI_COMMIT="$GITHUB_SHA"
-	CI_JOB_ID="$GITHUB_RUN_ID"
 	CC="${CC:-gcc}"
-	DONT_SKIP_TAGS=t
-
-	cache_dir="$HOME/none"
 
 	export GIT_PROVE_OPTS="--timer --jobs 10"
 	export GIT_TEST_OPTS="--verbose-log -x"
@@ -139,14 +56,6 @@ else
 	exit 1
 fi
 
-good_trees_file="$cache_dir/good-trees"
-
-mkdir -p "$cache_dir"
-
-test -n "${DONT_SKIP_TAGS-}" ||
-skip_branch_tip_with_tag
-skip_good_tree
-
 export DEVELOPER=1
 export DEFAULT_TEST_TARGET=prove
 export GIT_TEST_CLONE_2GB=true
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 0a1ec8c2bae..e5a21985b44 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -45,5 +45,3 @@ esac
 # start running tests.
 make $MAKE_TARGETS
 check_unignored_build_artifacts
-
-save_good_tree
diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh
index 65bcebda41a..5a87b1cac96 100755
--- a/ci/run-static-analysis.sh
+++ b/ci/run-static-analysis.sh
@@ -28,5 +28,3 @@ fi
 
 make hdr-check ||
 exit 1
-
-save_good_tree
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index de41888430a..9e0652c30dd 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -41,5 +41,3 @@ grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
 
 rm -f stdout.log stderr.log stderr.raw
 check_unignored_build_artifacts
-
-save_good_tree
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 05/25] CI: remove unused Azure ci/* code

[Ævar Arnfjörð Bjarmason]

Remove Azure-specific code that's been unused since 6081d3898fe (ci:
retire the Azure Pipelines definition, 2020-04-11). As noted in a
larger removal of all of the Azure-supporting code in [1] (although
that missed some of this) there is more of it in-tree, but let's focus
on only the ci/* code for now.

This is needed because in subsequent commits this unused code would
either need to be changed to accommodate changes in ci/*, or be
removed.

As we'll see in those subsequent commits the end-state we're heading
towards will actually make it easier to add new CI types in the
future, even though the only one we're left with now is the GitHub
CI. I.e. the "ci/*" framework will be made to do less, not more. We'll
be offloading more of what it does to our generic build and test
system.

While I'm at it (since the line needs to be touched anyway) change an
odd 'if test true == "$GITHUB_ACTIONS"' to the more usual style used
in other places of 'if test "$GITHUB_ACTIONS" = "true"'.

1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/install-dependencies.sh |  3 ---
 ci/lib.sh                  | 13 +------------
 ci/mount-fileshare.sh      | 25 -------------------------
 ci/print-test-failures.sh  |  7 +------
 4 files changed, 2 insertions(+), 46 deletions(-)
 delete mode 100755 ci/mount-fileshare.sh

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index dbcebad2fb2..e7ea8799411 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -34,8 +34,6 @@ macos-latest)
 	export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
 	# Uncomment this if you want to run perf tests:
 	# brew install gnu-time
-	test -z "$BREW_INSTALL_PACKAGES" ||
-	brew install $BREW_INSTALL_PACKAGES
 	brew link --force gettext
 	brew install --cask --no-quarantine perforce || {
 		# Update the definitions and try again
@@ -69,7 +67,6 @@ Documentation)
 	sudo apt-get -q update
 	sudo apt-get -q -y install asciidoc xmlto docbook-xsl-ns make
 
-	test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
 	sudo gem install --version 1.5.8 asciidoctor
 	;;
 linux-gcc-default)
diff --git a/ci/lib.sh b/ci/lib.sh
index 5a8938d4b3c..ad1aa59a384 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -29,18 +29,7 @@ export TERM=${TERM:-dumb}
 # Clear MAKEFLAGS that may come from the outside world.
 export MAKEFLAGS=
 
-if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
-then
-	CI_TYPE=azure-pipelines
-	# We are running in Azure Pipelines
-	CC="${CC:-gcc}"
-
-	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
-	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
-	MAKEFLAGS="$MAKEFLAGS --jobs=10"
-	test Windows_NT != "$AGENT_OS" ||
-	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
-elif test true = "$GITHUB_ACTIONS"
+if test "$GITHUB_ACTIONS" = "true"
 then
 	CI_TYPE=github-actions
 	CC="${CC:-gcc}"
diff --git a/ci/mount-fileshare.sh b/ci/mount-fileshare.sh
deleted file mode 100755
index 26b58a80960..00000000000
--- a/ci/mount-fileshare.sh
+++ /dev/null
@@ -1,25 +0,0 @@
-#!/bin/sh
-
-die () {
-	echo "$*" >&2
-	exit 1
-}
-
-test $# = 4 ||
-die "Usage: $0 <share> <username> <password> <mountpoint>"
-
-mkdir -p "$4" || die "Could not create $4"
-
-case "$(uname -s)" in
-Linux)
-	sudo mount -t cifs -o vers=3.0,username="$2",password="$3",dir_mode=0777,file_mode=0777,serverino "$1" "$4"
-	;;
-Darwin)
-	pass="$(echo "$3" | sed -e 's/\//%2F/g' -e 's/+/%2B/g')" &&
-	mount -t smbfs,soft "smb://$2:$pass@${1#//}" "$4"
-	;;
-*)
-	die "No support for $(uname -s)"
-	;;
-esac ||
-die "Could not mount $4"
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 00fcf27b1b2..0c63b6f7962 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -38,14 +38,9 @@ do
 		test_name="${TEST_EXIT%.exit}"
 		test_name="${test_name##*/}"
 		trash_dir="trash directory.$test_name"
+		mkdir -p failed-test-artifacts
 		case "$CI_TYPE" in
-		azure-pipelines)
-			mkdir -p failed-test-artifacts
-			mv "$trash_dir" failed-test-artifacts
-			continue
-			;;
 		github-actions)
-			mkdir -p failed-test-artifacts
 			echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
 			cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
 			tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 06/25] CI: don't have "git grep" invoke a pager in tree content check

[Ævar Arnfjörð Bjarmason]

Fix the omission of not adding "-P" (a.k.a. "--no-pager") to the "git"
invocation that spaws "git grep", added in 0e7696c64db (ci: disallow
directional formatting, 2021-11-04).

This didn't matter for the GitHub CI, but does when running the check
locally. A subsequent commit will start invoking it via a new check
from the top-level Makefile, and without this we'd visibly spawn a
pager when doing so.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/check-directional-formatting.bash | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ci/check-directional-formatting.bash b/ci/check-directional-formatting.bash
index e6211b141aa..de8e6e486f0 100755
--- a/ci/check-directional-formatting.bash
+++ b/ci/check-directional-formatting.bash
@@ -23,5 +23,5 @@
 # U+2066..U+2069: LRI, RLI, FSI and PDI
 regex='(\u202a|\u202b|\u202c|\u202d|\u202e|\u2066|\u2067|\u2068|\u2069)'
 
-! LC_CTYPE=C git grep -El "$(LC_CTYPE=C.UTF-8 printf "$regex")" \
+! LC_CTYPE=C git -P grep -El "$(LC_CTYPE=C.UTF-8 printf "$regex")" \
 	-- ':(exclude,attr:binary)' ':(exclude)*.po'
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 07/25] CI: have "static-analysis" run a "make ci-static-analysis" target

[Ævar Arnfjörð Bjarmason]

Make it easier to run the CI logic locally by creating
"ci-static-analysis" target.

Now the "check-coccicheck" logic added in 0860a7641ba (travis-ci: fail
if Coccinelle static analysis found something to transform,
2018-07-23) can be easily tested outside of CI, and likewise the
sanity check added in 0e7696c64db (ci: disallow directional
formatting, 2021-11-04).

These targets could be improved, the "hdr-check" target will re-do all
of its work every time it's run, and could with a change similar to my
c234e8a0ecf (Makefile: make the "sparse" target non-.PHONY,
2021-09-23). The same goes for the new
"ci-check-directional-formatting" target.

But without those improvements being able to easily run a 1=1
equivalent of the checks we do in CI during a local build is already
an improvement. Subsequent commits will expand on this ability.

This change will also make the CI check faster, since we can take
advantage of parallelism. The "make coccicheck" takes a very long
time, and at the tail end we might only have one job pegging 100% of
one CPU core for a long time. Now any extra cores will be free to run
the rest of the targets under "ci-static-analysis".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml |  3 +--
 Makefile                   | 28 ++++++++++++++++++++++++++++
 ci/run-static-analysis.sh  | 30 ------------------------------
 shared.mak                 |  1 +
 4 files changed, 30 insertions(+), 32 deletions(-)
 delete mode 100755 ci/run-static-analysis.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 309b9141249..ad752010102 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -317,8 +317,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/run-static-analysis.sh
-    - run: ci/check-directional-formatting.bash
+    - run: make ci-static-analysis
   sparse:
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
diff --git a/Makefile b/Makefile
index 70f0a004e75..d30862a23a2 100644
--- a/Makefile
+++ b/Makefile
@@ -3007,6 +3007,20 @@ coccicheck: $(addsuffix .patch,$(filter-out %.pending.cocci,$(wildcard contrib/c
 # See contrib/coccinelle/README
 coccicheck-pending: $(addsuffix .patch,$(wildcard contrib/coccinelle/*.pending.cocci))
 
+.PHONY: check-coccicheck
+check-coccicheck: coccicheck
+	$(QUIET_CHECK)for cocci_patch in contrib/coccinelle/*.patch; do \
+		if test -s "$$cocci_patch"; then \
+			echo "Coccinelle suggests the following changes in '$$cocci_patch':"; \
+			cat "$$cocci_patch"; \
+			fail=UnfortunatelyYes; \
+		fi \
+	done; \
+	if test -n "$$fail"; then \
+		echo "error: Coccinelle suggested some changes"; \
+		exit 1; \
+	fi
+
 .PHONY: coccicheck coccicheck-pending
 
 ### Installation rules
@@ -3442,3 +3456,17 @@ $(FUZZ_PROGRAMS): all
 		$(XDIFF_OBJS) $(EXTLIBS) git.o $@.o $(LIB_FUZZING_ENGINE) -o $@
 
 fuzz-all: $(FUZZ_PROGRAMS)
+
+### CI "check" targets
+#
+# These targets are run from the CI, see .github/workflows/main.yml,
+# but can also be run manually to run the same assertions locally.
+
+.PHONY: ci-check-directional-formatting
+ci-check-directional-formatting:
+	$(QUIET_CHECK)ci/check-directional-formatting.bash
+
+.PHONY: ci-static-analysis
+ci-static-analysis: ci-check-directional-formatting
+ci-static-analysis: check-coccicheck
+ci-static-analysis: hdr-check
diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh
deleted file mode 100755
index 5a87b1cac96..00000000000
--- a/ci/run-static-analysis.sh
+++ /dev/null
@@ -1,30 +0,0 @@
-#!/bin/sh
-#
-# Perform various static code analysis checks
-#
-
-. ${0%/*}/lib.sh
-
-make coccicheck
-
-set +x
-
-fail=
-for cocci_patch in contrib/coccinelle/*.patch
-do
-	if test -s "$cocci_patch"
-	then
-		echo "$(tput setaf 1)Coccinelle suggests the following changes in '$cocci_patch':$(tput sgr0)"
-		cat "$cocci_patch"
-		fail=UnfortunatelyYes
-	fi
-done
-
-if test -n "$fail"
-then
-	echo "$(tput setaf 1)error: Coccinelle suggested some changes$(tput sgr0)"
-	exit 1
-fi
-
-make hdr-check ||
-exit 1
diff --git a/shared.mak b/shared.mak
index 4e1b62ee99e..0593b3c98be 100644
--- a/shared.mak
+++ b/shared.mak
@@ -68,6 +68,7 @@ ifndef V
 	QUIET_HDR      = @echo '   ' HDR $(<:hcc=h);
 	QUIET_RC       = @echo '   ' RC $@;
 	QUIET_SPATCH   = @echo '   ' SPATCH $<;
+	QUIET_CHECK    = @echo '   ' CHECK $@;
 
 ## Used in "Documentation/Makefile"
 	QUIET_ASCIIDOC	= @echo '   ' ASCIIDOC $@;
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 08/25] CI: have "static-analysis" run "check-builtins", not "documentation"

[Ævar Arnfjörð Bjarmason]

Move the "make check-builtins" check from the "documentation" job to
the "static-analysis" job.

The "check-builtins" target added in c74390e4a1d (cherry is built-in,
do not ship git-cherry.sh, 2006-11-05) is unrelated to the
documentation, so it's odd that b98712b9aa9 (travis-ci: build
documentation, 2016-05-04) added it to the "documentation" job.

Let's just move it to the "ci-static-analysis" target, and while we're
at it improve the output with $(QUIET_CHECK).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile                 | 3 ++-
 ci/test-documentation.sh | 1 -
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index d30862a23a2..82f4206188a 100644
--- a/Makefile
+++ b/Makefile
@@ -3378,7 +3378,7 @@ check-docs::
 ### Make sure built-ins do not have dups and listed in git.c
 #
 check-builtins::
-	./check-builtins.sh
+	$(QUIET_CHECK)./check-builtins.sh
 
 ### Test suite coverage testing
 #
@@ -3468,5 +3468,6 @@ ci-check-directional-formatting:
 
 .PHONY: ci-static-analysis
 ci-static-analysis: ci-check-directional-formatting
+ci-static-analysis: check-builtins
 ci-static-analysis: check-coccicheck
 ci-static-analysis: hdr-check
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 9e0652c30dd..41e2b126311 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -14,7 +14,6 @@ filter_log () {
 	    "$1"
 }
 
-make check-builtins
 make check-docs
 
 # Build docs with AsciiDoc
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 09/25] CI: move p4 and git-lfs variables to ci/install-dependencies.sh

[Ævar Arnfjörð Bjarmason]

Move the declaration of variables that are only used by the
"ubuntu-latest" block in "ci/install-dependencies.sh" there from
"ci/lib.sh".

This makes the code easier to follow, and changes "ci/lib.sh" to a
library that only exports CI variables for general use, we didn't need
to export these $P4_PATH and $GIT_LFS_PATH variables.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/install-dependencies.sh | 17 +++++++++++++++--
 ci/lib.sh                  | 12 ------------
 2 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index e7ea8799411..34b4400f38b 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -5,14 +5,27 @@
 
 . ${0%/*}/lib.sh
 
-P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
-LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
 UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
  tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
  libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl"
 
 case "$runs_on_pool" in
 ubuntu-latest)
+	# The Linux build installs the defined dependency versions below.
+	# The OS X build installs much more recent versions, whichever
+	# were recorded in the Homebrew database upon creating the OS X
+	# image.
+	# Keep that in mind when you encounter a broken OS X build!
+	LINUX_P4_VERSION="16.2"
+	LINUX_GIT_LFS_VERSION="1.5.2"
+
+	P4_PATH="$HOME/custom/p4"
+	GIT_LFS_PATH="$HOME/custom/git-lfs"
+	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
+
+	P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
+	LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
+
 	sudo apt-get -q update
 	sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \
 		$UBUNTU_COMMON_PKGS $CC_PACKAGE
diff --git a/ci/lib.sh b/ci/lib.sh
index ad1aa59a384..6b37b10d750 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -65,18 +65,6 @@ ubuntu-latest)
 	fi
 
 	export GIT_TEST_HTTPD=true
-
-	# The Linux build installs the defined dependency versions below.
-	# The OS X build installs much more recent versions, whichever
-	# were recorded in the Homebrew database upon creating the OS X
-	# image.
-	# Keep that in mind when you encounter a broken OS X build!
-	export LINUX_P4_VERSION="16.2"
-	export LINUX_GIT_LFS_VERSION="1.5.2"
-
-	P4_PATH="$HOME/custom/p4"
-	GIT_LFS_PATH="$HOME/custom/git-lfs"
-	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
 	;;
 macos-latest)
 	if [ "$jobname" = osx-gcc ]
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 10/25] CI: consistently use "export" in ci/lib.sh

[Ævar Arnfjörð Bjarmason]

Change the "ci/lib.sh" script co consistently use "export", for
e.g. MAKEFLAGS we were exporting it, and then assigning to it, let's
do it the other way around.

Right now this doesn't matter, since we in
e.g. "ci/install-dependencies.sh" source this file, and don't use
something like "env(1)" to retrieve these variables. But in a
subsequent commit we'll "export" these variables through a wrapper (to
additionally write them to a GitHub CI-specific $GITHUB_ENV
file). This change makes that subsequent change easier to read, as it
won't need to do any control flow refactoring.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 6b37b10d750..31fe3a476a9 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -27,7 +27,7 @@ check_unignored_build_artifacts ()
 export TERM=${TERM:-dumb}
 
 # Clear MAKEFLAGS that may come from the outside world.
-export MAKEFLAGS=
+MAKEFLAGS=
 
 if test "$GITHUB_ACTIONS" = "true"
 then
@@ -35,10 +35,12 @@ then
 	CC="${CC:-gcc}"
 
 	export GIT_PROVE_OPTS="--timer --jobs 10"
-	export GIT_TEST_OPTS="--verbose-log -x"
+	GIT_TEST_OPTS="--verbose-log -x"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
+
+	export GIT_TEST_OPTS
 else
 	echo "Could not identify CI type" >&2
 	env >&2
@@ -92,4 +94,4 @@ linux-leaks)
 	;;
 esac
 
-MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
+export MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 11/25] CI: export variables via a wrapper

[Ævar Arnfjörð Bjarmason]

Add and use a "setenv" wrapper to export the variables set in
ci/lib.sh.

For now this merely adds debugging information about what variables
are set, but in a subsequent commit we'll expand this to set variables
in $GITHUB_ENV instead of the environment, and thus carry variable
settings between steps in a job.

We'll also eventually be interested in what type of variable we're
setting, and when it needs to be used, because we'll prominently
expose what variables we set in what CI "step", so having only the
ones we actually use will improve readability.

To that end call the "setenv" function with optional "--build",
"--test" and "--all" parameters to indicate whether they're needed for
building, testing or both.

We currently ignore this parameter, but will make use of it in a
subsequent commit. Adding it here reduces the needless churn, as we
won't need to change all these lines again for that subsequent change.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                 | 39 ++++++++++++++++++++++++++++-----------
 ci/run-build-and-tests.sh | 34 +++++++++++++++++-----------------
 2 files changed, 45 insertions(+), 28 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 31fe3a476a9..1a1fa0e4bb2 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -13,6 +13,23 @@ then
 	exit 1
 fi
 
+# Helper functions
+setenv () {
+	varmode=
+	case "$1" in
+	--*)
+		varmode=$1
+		shift
+		;;
+	esac
+
+	key=$1
+	val=$2
+	shift 2
+
+	eval "export $key=\"$val\""
+}
+
 check_unignored_build_artifacts ()
 {
 	! git ls-files --other --exclude-standard --error-unmatch \
@@ -24,7 +41,7 @@ check_unignored_build_artifacts ()
 }
 
 # GitHub Action doesn't set TERM, which is required by tput
-export TERM=${TERM:-dumb}
+setenv TERM ${TERM:-dumb}
 
 # Clear MAKEFLAGS that may come from the outside world.
 MAKEFLAGS=
@@ -34,23 +51,23 @@ then
 	CI_TYPE=github-actions
 	CC="${CC:-gcc}"
 
-	export GIT_PROVE_OPTS="--timer --jobs 10"
+	setenv --test GIT_PROVE_OPTS "--timer --jobs 10"
 	GIT_TEST_OPTS="--verbose-log -x"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 
-	export GIT_TEST_OPTS
+	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
 else
 	echo "Could not identify CI type" >&2
 	env >&2
 	exit 1
 fi
 
-export DEVELOPER=1
-export DEFAULT_TEST_TARGET=prove
-export GIT_TEST_CLONE_2GB=true
-export SKIP_DASHED_BUILT_INS=YesPlease
+setenv --build DEVELOPER 1
+setenv --test DEFAULT_TEST_TARGET prove
+setenv --test GIT_TEST_CLONE_2GB true
+setenv --build SKIP_DASHED_BUILT_INS YesPlease
 
 case "$runs_on_pool" in
 ubuntu-latest)
@@ -66,7 +83,7 @@ ubuntu-latest)
 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python2"
 	fi
 
-	export GIT_TEST_HTTPD=true
+	setenv --test GIT_TEST_HTTPD true
 	;;
 macos-latest)
 	if [ "$jobname" = osx-gcc ]
@@ -89,9 +106,9 @@ linux-musl)
 	MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8"
 	;;
 linux-leaks)
-	export SANITIZE=leak
-	export GIT_TEST_PASSING_SANITIZE_LEAK=true
+	setenv --build SANITIZE leak
+	setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true
 	;;
 esac
 
-export MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
+setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index e5a21985b44..35d45a9373e 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -9,33 +9,33 @@ export MAKE_TARGETS="all test"
 
 case "$jobname" in
 linux-gcc)
-	export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
 	;;
 linux-TEST-vars)
-	export GIT_TEST_SPLIT_INDEX=yes
-	export GIT_TEST_MERGE_ALGORITHM=recursive
-	export GIT_TEST_FULL_IN_PACK_ARRAY=true
-	export GIT_TEST_OE_SIZE=10
-	export GIT_TEST_OE_DELTA_SIZE=5
-	export GIT_TEST_COMMIT_GRAPH=1
-	export GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS=1
-	export GIT_TEST_MULTI_PACK_INDEX=1
-	export GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP=1
-	export GIT_TEST_ADD_I_USE_BUILTIN=1
-	export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master
-	export GIT_TEST_WRITE_REV_INDEX=1
-	export GIT_TEST_CHECKOUT_WORKERS=2
+	setenv --test GIT_TEST_SPLIT_INDEX yes
+	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
+	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
+	setenv --test GIT_TEST_OE_SIZE 10
+	setenv --test GIT_TEST_OE_DELTA_SIZE 5
+	setenv --test GIT_TEST_COMMIT_GRAPH 1
+	setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
+	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
+	setenv --test GIT_TEST_WRITE_REV_INDEX 1
+	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
 	;;
 linux-clang)
-	export GIT_TEST_DEFAULT_HASH=sha1
+	setenv --test GIT_TEST_DEFAULT_HASH sha1
 	;;
 linux-sha256)
-	export GIT_TEST_DEFAULT_HASH=sha256
+	setenv --test GIT_TEST_DEFAULT_HASH sha256
 	;;
 pedantic)
 	# Don't run the tests; we only care about whether Git can be
 	# built.
-	export DEVOPTS=pedantic
+	setenv --build DEVOPTS pedantic
 	export MAKE_TARGETS=all
 	;;
 esac
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 12/25] CI: remove "run-build-and-tests.sh", run "make [test]" directly

[Ævar Arnfjörð Bjarmason]

Remove the already thin "ci/run-build-and-tests.sh" wrapper and
instead make the CI run "make" or "make test" directly. By doing this
we'll be able to easily see at a glance whether our failure was in the
compilation or testing, whether that's via human eyes or improve
machine readability.

We also need to run our new "ci/check-unignored-build-artifacts.sh" on
success() in the CI now, just like we already had a step conditional
on failure() running ci/print-test-failures.sh.

The reason we used a "ci/run-build-and-tests.sh" wrapper in the first
place had to do with Travis CI-specific constraints that no longer
apply to us, as the Travis CI support has been removed.

Instead we can configure the CI in an earlier step by running
"ci/lib.sh", which under GitHub CI will write the environment
variables we need to the "$GITHUB_ENV" file.

We'll then have access to them in subsequent steps, and crucially
those variables will be prominently visible at the start of each step
via an expandable drop-down in the UI.drop-do.

I.e. this changes the CI run from a top-down flow like (pseudocode):

 - job:
   - step1:
     - use ci/lib.sh to set env vars
     - run a script like ci/run-build-and-tests.sh
   - step2:
     - if: failure()
     - use ci/lib.sh to set env vars
     - run ci/print-test-failures.sh

To:

 - job:
   - step1:
     - set variables in $GITHUB_ENV using ci/lib.sh
   - step2:
     - make
   - step3:
     - make test
   - step4:
     - if: failure()
     - run ci/print-test-failures.sh
   - step5:
     - if: success()
     - run ci/check-unignored-build-artifacts.sh

There is a proposal[2] to get some of the benefits of this approach by
not re-arranging our variable setup in this way, but to instead use
the GitHub CI grouping syntax to focus on the relevant parts of "make"
or "make test" when we have failures.

Doing it this way makes for better looking GitHub CI UI, and lays much
better ground work for our CI going forward. Because:

 * The CI logic will be more portable to a future CI system, since a
   common feature of them is to run various commands in sequence, but
   a future system won't necessarily support the GitHub-specifics
   syntax of "grouping" output within a "step".

   Even if those systems don't support a "$GITHUB_ENV" emulating will
   be much easier than to deal with some CI-specific grouping syntax.

 * At the start of every step the GitHub CI presents an expandable
   list of environment variables from "$GITHUB_ENV". We'll now see
   exactly what variables affected that step (although we currently
   overshoot that a bit, and always define all variables).

 * CI failures will be easier to reproduce locally, as this makes the
   relevant ci/* scripts something that sets up our environment, but
   leaves "make" and "make test" working as they do locally.

   To reproduce a run the user only needs to set the variables
   discussed in the drop-down above, either manually or by running
   "ci/lib.sh".

 * The output will be less verbose. The "ci/lib.sh" script uses "set
   -x", and before this e.g. "ci/static-analysis.sh" would start with
   40 lines of trace output, culminating in using "export" to export
   the relevant environment variables.

   Now that verbosity is in the earlier "ci/lib.sh" step, and not in
   any subsequent one. The "make" targets then start out with the
   relevant output non-trace output right away.

 * If we do want to use the grouping syntax within a "step" it'll now
   be easier to do so. It doesn't support nesting, so we'd have to
   make a choice between using it for e.g. "make" v.s. "make test", or
   individual test failures. See "sadly" in [3].

1. https://lore.kernel.org/git/211120.86k0h30zuw.gmgdl@evledraar.gmail.com/
2. https://lore.kernel.org/git/pull.1117.git.1643050574.gitgitgadget@gmail.com/
3. https://lore.kernel.org/git/9333ba781b8240f704e739b00d274f8c3d887e39.1643050574.git.gitgitgadget@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml            | 14 ++++++--
 ci/check-unignored-build-artifacts.sh |  8 +++++
 ci/install-dependencies.sh            |  4 +++
 ci/lib.sh                             | 40 ++++++++++++++++++++++-
 ci/make-test-artifacts.sh             |  2 --
 ci/run-build-and-tests.sh             | 47 ---------------------------
 ci/run-test-slice.sh                  |  2 --
 ci/test-documentation.sh              |  1 -
 8 files changed, 62 insertions(+), 56 deletions(-)
 create mode 100755 ci/check-unignored-build-artifacts.sh
 delete mode 100755 ci/run-build-and-tests.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index ad752010102..c83f6a21ca3 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -268,7 +268,10 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/run-build-and-tests.sh
+    - run: ci/lib.sh
+    - run: make
+    - run: make test
+      if: success()
     - run: ci/print-test-failures.sh
       if: failure()
     - name: Upload failed tests' directories
@@ -292,6 +295,7 @@ jobs:
           image: daald/ubuntu32:xenial
         - jobname: pedantic
           image: fedora
+          skip-tests: no
     env:
       jobname: ${{matrix.vector.jobname}}
     runs-on: ubuntu-latest
@@ -299,9 +303,12 @@ jobs:
     steps:
     - uses: actions/checkout@v1
     - run: ci/install-docker-dependencies.sh
-    - run: ci/run-build-and-tests.sh
+    - run: ci/lib.sh
+    - run: make
+    - run: make test
+      if: success() && matrix.vector.skip-tests != 'no'
     - run: ci/print-test-failures.sh
-      if: failure()
+      if: failure() && matrix.vector.skip-tests != 'no'
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v1
@@ -317,6 +324,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
+    - run: ci/lib.sh
     - run: make ci-static-analysis
   sparse:
     needs: ci-config
diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh
new file mode 100755
index 00000000000..56d04b0db9a
--- /dev/null
+++ b/ci/check-unignored-build-artifacts.sh
@@ -0,0 +1,8 @@
+#!/bin/sh
+#
+# Check whether the build created anything not in our .gitignore
+#
+
+. ${0%/*}/lib.sh
+
+check_unignored_build_artifacts
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 34b4400f38b..01006379fd0 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -22,6 +22,10 @@ ubuntu-latest)
 	P4_PATH="$HOME/custom/p4"
 	GIT_LFS_PATH="$HOME/custom/git-lfs"
 	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
+	if test -n "$GITHUB_PATH"
+	then
+		echo "$PATH" >>"$GITHUB_PATH"
+	fi
 
 	P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
 	LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
diff --git a/ci/lib.sh b/ci/lib.sh
index 1a1fa0e4bb2..05f3dd15e27 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,3 +1,5 @@
+#!/bin/sh
+
 # Library of functions shared by all CI scripts
 
 # Set 'exit on error' for all CI scripts to let the caller know that
@@ -27,7 +29,14 @@ setenv () {
 	val=$2
 	shift 2
 
-	eval "export $key=\"$val\""
+	if test -n "$GITHUB_ENV"
+	then
+		echo "$key=$val" >>"$GITHUB_ENV"
+	else
+		# For local debugging. Not used by the GitHub CI
+		# itself.
+		eval "export $key=\"$val\""
+	fi
 }
 
 check_unignored_build_artifacts ()
@@ -96,6 +105,35 @@ macos-latest)
 esac
 
 case "$jobname" in
+linux-gcc)
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
+	;;
+linux-TEST-vars)
+	setenv --test GIT_TEST_SPLIT_INDEX yes
+	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
+	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
+	setenv --test GIT_TEST_OE_SIZE 10
+	setenv --test GIT_TEST_OE_DELTA_SIZE 5
+	setenv --test GIT_TEST_COMMIT_GRAPH 1
+	setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
+	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
+	setenv --test GIT_TEST_WRITE_REV_INDEX 1
+	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
+	;;
+linux-clang)
+	setenv --test GIT_TEST_DEFAULT_HASH sha1
+	;;
+linux-sha256)
+	setenv --test GIT_TEST_DEFAULT_HASH sha256
+	;;
+pedantic)
+	# Don't run the tests; we only care about whether Git can be
+	# built.
+	setenv --build DEVOPTS pedantic
+	;;
 linux32)
 	CC=gcc
 	;;
diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh
index 646967481f6..45298562982 100755
--- a/ci/make-test-artifacts.sh
+++ b/ci/make-test-artifacts.sh
@@ -8,5 +8,3 @@ mkdir -p "$1" # in case ci/lib.sh decides to quit early
 . ${0%/*}/lib.sh
 
 make artifacts-tar ARTIFACTS_DIRECTORY="$1"
-
-check_unignored_build_artifacts
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
deleted file mode 100755
index 35d45a9373e..00000000000
--- a/ci/run-build-and-tests.sh
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/bin/sh
-#
-# Build and test Git
-#
-
-. ${0%/*}/lib.sh
-
-export MAKE_TARGETS="all test"
-
-case "$jobname" in
-linux-gcc)
-	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
-	;;
-linux-TEST-vars)
-	setenv --test GIT_TEST_SPLIT_INDEX yes
-	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
-	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
-	setenv --test GIT_TEST_OE_SIZE 10
-	setenv --test GIT_TEST_OE_DELTA_SIZE 5
-	setenv --test GIT_TEST_COMMIT_GRAPH 1
-	setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
-	setenv --test GIT_TEST_MULTI_PACK_INDEX 1
-	setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
-	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
-	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
-	setenv --test GIT_TEST_WRITE_REV_INDEX 1
-	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
-	;;
-linux-clang)
-	setenv --test GIT_TEST_DEFAULT_HASH sha1
-	;;
-linux-sha256)
-	setenv --test GIT_TEST_DEFAULT_HASH sha256
-	;;
-pedantic)
-	# Don't run the tests; we only care about whether Git can be
-	# built.
-	setenv --build DEVOPTS pedantic
-	export MAKE_TARGETS=all
-	;;
-esac
-
-# Any new "test" targets should not go after this "make", but should
-# adjust $MAKE_TARGETS. Otherwise compilation-only targets above will
-# start running tests.
-make $MAKE_TARGETS
-check_unignored_build_artifacts
diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index b9a682b4bcd..70326961454 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -8,5 +8,3 @@
 make --quiet -C t T="$(cd t &&
 	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
 	tr '\n' ' ')"
-
-check_unignored_build_artifacts
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 41e2b126311..6b5cce03bd7 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -39,4 +39,3 @@ test -s Documentation/git.html
 grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
 
 rm -f stdout.log stderr.log stderr.raw
-check_unignored_build_artifacts
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 13/25] CI: check ignored unignored build artifacts in "win[+VS] build" too

[Ævar Arnfjörð Bjarmason]

Use the newly created "ci/check-unignored-build-artifacts.sh" script
in the "win build" and "win+VS build" steps too. These steps weren't
doing this sanity check of untracked files against .gitignore before,
but as the check passes for them let's keep doing that extra sanity
check.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index c83f6a21ca3..05c5771e7f6 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -93,6 +93,9 @@ jobs:
         HOME: ${{runner.workspace}}
         NO_PERL: 1
       run: . /etc/profile && ci/make-test-artifacts.sh artifacts
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
+      shell: bash
     - name: zip up tracked files
       run: git archive -o artifacts/tracked.tar.gz HEAD
     - name: upload tracked files and build artifacts
@@ -180,6 +183,9 @@ jobs:
       run: |
         mkdir -p artifacts &&
         eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts NO_GETTEXT=YesPlease 2>&1 | grep ^tar)"
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
+      shell: bash
     - name: zip up tracked files
       run: git archive -o artifacts/tracked.tar.gz HEAD
     - name: upload tracked files and build artifacts
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 14/25] CI: invoke "make artifacts-tar" directly in windows-build

[Ævar Arnfjörð Bjarmason]

Change the windows-build job to invoke the "make artifacts-tar" step
directly, instead of calling a "ci/make-test-artifacts.sh" script.

The script was needed because "ci/lib.sh" would set up various
environment variables for us, but now we can instead use the
"ci/lib.sh" in its script mode.

The "mkdir -p" added in b819f1d2cec (ci: parallelize testing on
Windows, 2019-01-29) isn't needed, the same commit added that "mkdir
-p" to the "artifacts-tar" rule itself, so we can have "make" create
the directory for us.

This also has the benefit of making the "build" step less chatty,
since it won't start with the verbose "set -x" output, that's now
contained in the "ci/lib.sh" step.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml |  5 ++++-
 ci/make-test-artifacts.sh  | 10 ----------
 2 files changed, 4 insertions(+), 11 deletions(-)
 delete mode 100755 ci/make-test-artifacts.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 05c5771e7f6..2c23a19eac2 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -87,12 +87,15 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
+    - run: ci/lib.sh
+      shell: bash
     - name: build
       shell: bash
       env:
         HOME: ${{runner.workspace}}
         NO_PERL: 1
-      run: . /etc/profile && ci/make-test-artifacts.sh artifacts
+      run: . /etc/profile && make artifacts-tar ARTIFACTS_DIRECTORY=artifacts
+      if: success()
     - run: ci/check-unignored-build-artifacts.sh
       if: success()
       shell: bash
diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh
deleted file mode 100755
index 45298562982..00000000000
--- a/ci/make-test-artifacts.sh
+++ /dev/null
@@ -1,10 +0,0 @@
-#!/bin/sh
-#
-# Build Git and store artifacts for testing
-#
-
-mkdir -p "$1" # in case ci/lib.sh decides to quit early
-
-. ${0%/*}/lib.sh
-
-make artifacts-tar ARTIFACTS_DIRECTORY="$1"
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 15/25] CI: split up and reduce "ci/test-documentation.sh"

[Ævar Arnfjörð Bjarmason]

Change the "ci/test-documentation.sh" script to run the bash-specific
parts in as one command in the CI job itself, and to run the two "make
doc" commands at the top-level.

It'll now be obvious from the title of the step if if we failed in the
asciidoc or asciidoctor step.

Since the "check_unignored_build_artifacts()" function is now only
used in "ci/check-unignored-build-artifacts.sh" move that function
there.

The recipe for the job in ".github/workflows/main.yml" is now a bit
verbose because it's effectively the same job twice, with a "make
clean" in-between. It would be better for the verbosity to run it via
a matrix as done in the alternate approach in [1] does, but then we'd
sacrifice overall CPU time for the brevity. It might still be worth
doing, but let's go for this simpler approach for now.

1. https://lore.kernel.org/git/patch-v2-6.6-7c423c8283d-20211120T030848Z-avarab@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml            | 16 +++++++++++++-
 ci/check-unignored-build-artifacts.sh | 10 +++++++++
 ci/lib.sh                             | 10 ---------
 ci/test-documentation.sh              | 31 +++++++++------------------
 4 files changed, 35 insertions(+), 32 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 2c23a19eac2..92b914f16fd 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -365,4 +365,18 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/test-documentation.sh
+    - run: ci/lib.sh
+    - run: make check-docs
+    - run: "make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)"
+      shell: bash
+    - run: ci/test-documentation.sh AsciiDoc
+      if: success()
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
+    - run: make clean
+    - run: "make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)"
+      shell: bash
+    - run: ci/test-documentation.sh Asciidoctor
+      if: success()
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh
index 56d04b0db9a..0bc04f32804 100755
--- a/ci/check-unignored-build-artifacts.sh
+++ b/ci/check-unignored-build-artifacts.sh
@@ -5,4 +5,14 @@
 
 . ${0%/*}/lib.sh
 
+check_unignored_build_artifacts ()
+{
+	! git ls-files --other --exclude-standard --error-unmatch \
+		-- ':/*' 2>/dev/null ||
+	{
+		echo "$(tput setaf 1)error: found unignored build artifacts$(tput sgr0)"
+		false
+	}
+}
+
 check_unignored_build_artifacts
diff --git a/ci/lib.sh b/ci/lib.sh
index 05f3dd15e27..fc6ce4d3e04 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -39,16 +39,6 @@ setenv () {
 	fi
 }
 
-check_unignored_build_artifacts ()
-{
-	! git ls-files --other --exclude-standard --error-unmatch \
-		-- ':/*' 2>/dev/null ||
-	{
-		echo "$(tput setaf 1)error: found unignored build artifacts$(tput sgr0)"
-		false
-	}
-}
-
 # GitHub Action doesn't set TERM, which is required by tput
 setenv TERM ${TERM:-dumb}
 
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 6b5cce03bd7..b8a6a6f664e 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -1,10 +1,12 @@
-#!/usr/bin/env bash
+#!/bin/sh
 #
-# Perform sanity checks on documentation and build it.
+# Perform sanity checks on "make doc" output and built documentation
 #
 
 . ${0%/*}/lib.sh
 
+generator=$1
+
 filter_log () {
 	sed -e '/^GIT_VERSION = /d' \
 	    -e "/constant Gem::ConfigMap is deprecated/d" \
@@ -14,28 +16,15 @@ filter_log () {
 	    "$1"
 }
 
-make check-docs
-
-# Build docs with AsciiDoc
-make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
-cat stderr.raw
-filter_log stderr.raw >stderr.log
-test ! -s stderr.log
-test -s Documentation/git.html
-test -s Documentation/git.xml
-test -s Documentation/git.1
-grep '<meta name="generator" content="AsciiDoc ' Documentation/git.html
-
-rm -f stdout.log stderr.log stderr.raw
-check_unignored_build_artifacts
-
-# Build docs with AsciiDoctor
-make clean
-make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
 cat stderr.raw
 filter_log stderr.raw >stderr.log
 test ! -s stderr.log
 test -s Documentation/git.html
-grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
+if test "$generator" = "Asciidoctor"
+then
+	test -s Documentation/git.xml
+	test -s Documentation/git.1
+fi
+grep "<meta name=\"generator\" content=\"$generator " Documentation/git.html
 
 rm -f stdout.log stderr.log stderr.raw
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 16/25] CI: combine ci/install{,-docker}-dependencies.sh

[Ævar Arnfjörð Bjarmason]

Combine the "ci/install-dependencies.sh" and
"ci/install-docker-dependencies.sh" into one script. The "case"
statement in the latter only cared about "$jobname", and can be folded
into the same "case" statement in the former.

The reason they split up is historical, and because the
"ci/install-dependencies.sh" used "ci/lib.sh", which requires
"bash". At least one of the docker containers doesn't have "bash". To
make the existing code POSIX-compatible we need to replace pushd/popd
with a sub-shell, but no other changes were needed.

This also fixes issue in "ci/install-docker-dependencies.sh" where
we'd hide errors due to not using "set -e". Now that we include
"ci/lib.sh" we won't have that potential issue.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml        |  2 +-
 ci/install-dependencies.sh        | 27 ++++++++++++++++++++++-----
 ci/install-docker-dependencies.sh | 22 ----------------------
 3 files changed, 23 insertions(+), 28 deletions(-)
 delete mode 100755 ci/install-docker-dependencies.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 92b914f16fd..3fd3e4f873c 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -311,7 +311,7 @@ jobs:
     container: ${{matrix.vector.image}}
     steps:
     - uses: actions/checkout@v1
-    - run: ci/install-docker-dependencies.sh
+    - run: ci/install-dependencies.sh
     - run: ci/lib.sh
     - run: make
     - run: make test
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 01006379fd0..9388289d0ed 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -1,4 +1,4 @@
-#!/usr/bin/env bash
+#!/bin/sh
 #
 # Install dependencies required to build and test Git on Linux and macOS
 #
@@ -34,18 +34,20 @@ ubuntu-latest)
 	sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \
 		$UBUNTU_COMMON_PKGS $CC_PACKAGE
 	mkdir --parents "$P4_PATH"
-	pushd "$P4_PATH"
+	(
+		cd "$P4_PATH"
 		wget --quiet "$P4WHENCE/bin.linux26x86_64/p4d"
 		wget --quiet "$P4WHENCE/bin.linux26x86_64/p4"
 		chmod u+x p4d
 		chmod u+x p4
-	popd
+	)
 	mkdir --parents "$GIT_LFS_PATH"
-	pushd "$GIT_LFS_PATH"
+	(
+		cd "$GIT_LFS_PATH"
 		wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
 		tar --extract --gunzip --file "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
 		cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs .
-	popd
+	)
 	;;
 macos-latest)
 	export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
@@ -90,6 +92,21 @@ linux-gcc-default)
 	sudo apt-get -q update
 	sudo apt-get -q -y install $UBUNTU_COMMON_PKGS
 	;;
+linux32)
+	linux32 --32bit i386 sh -c '
+		apt update >/dev/null &&
+		apt install -y build-essential libcurl4-openssl-dev \
+			libssl-dev libexpat-dev gettext python >/dev/null
+	'
+	;;
+linux-musl)
+	apk add --update build-base curl-dev openssl-dev expat-dev gettext \
+		pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null
+	;;
+pedantic)
+	dnf -yq update >/dev/null &&
+	dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null
+	;;
 esac
 
 if type p4d >/dev/null && type p4 >/dev/null
diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh
deleted file mode 100755
index 78b7e326da6..00000000000
--- a/ci/install-docker-dependencies.sh
+++ /dev/null
@@ -1,22 +0,0 @@
-#!/bin/sh
-#
-# Install dependencies required to build and test Git inside container
-#
-
-case "$jobname" in
-linux32)
-	linux32 --32bit i386 sh -c '
-		apt update >/dev/null &&
-		apt install -y build-essential libcurl4-openssl-dev \
-			libssl-dev libexpat-dev gettext python >/dev/null
-	'
-	;;
-linux-musl)
-	apk add --update build-base curl-dev openssl-dev expat-dev gettext \
-		pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null
-	;;
-pedantic)
-	dnf -yq update >/dev/null &&
-	dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null
-	;;
-esac
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 17/25] CI: move "env" definitions into ci/lib.sh

[Ævar Arnfjörð Bjarmason]

Have the GitHub CI jobs use the "ci/lib".sh as a source of truth for
environment variables that affect their builds and tests. This brings
them in line with the rest of the jobs, and make it easier to see the
entire CI configuration per jobname at a glance.

To do this we need to add a ci/lib.sh "step" to the remaining jobs
that didn't have it added in preceding commits.

The Makefile parameters are luckily all accepted via the environment,
so we can export these instead of adding them to MAKEFLAGS.

Let's also use the documented GIT_CONIFG_* mechanism for setting
config instead of the internal GIT_CONFIG_PARAMETERS variable. This
adjusts code added in 889cacb6897 (ci: configure GitHub Actions for
CI/PR, 2020-04-11), we could probably use the GIT_AUTHOR_NAME etc, but
let's keep setting this via config, just with the documented
mechanism.

By setting "ARTIFACTS_DIRECTORY=artifacts" once we don't need to
repeat it in various places, let's get it from the environment
instead.

Finally, the "DEVELOPER: 1" was only needed by the "sparse" job, which
wasn't using "ci/lib.sh" (and which sets DEVELOPER=1). Let's instead
have the "sparse" job use the "ci/lib.sh" and remove DEVELOPER=1 from
".github/workflows/main.yml".

This substantially speeds up the "sparse" job, since it'll now pick up
the "--jobs" setting in MAKEFLAGS that we use everywhere else. Before
it ran in around 4m30s, now in around 2m30s.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 29 +++++++++++------------------
 ci/lib.sh                  | 21 +++++++++++++++++++++
 2 files changed, 32 insertions(+), 18 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 3fd3e4f873c..fe37e0f1b36 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -2,9 +2,6 @@ name: CI
 
 on: [push, pull_request]
 
-env:
-  DEVELOPER: 1
-
 jobs:
   ci-config:
     name: config
@@ -93,8 +90,7 @@ jobs:
       shell: bash
       env:
         HOME: ${{runner.workspace}}
-        NO_PERL: 1
-      run: . /etc/profile && make artifacts-tar ARTIFACTS_DIRECTORY=artifacts
+      run: . /etc/profile && make artifacts-tar
       if: success()
     - run: ci/check-unignored-build-artifacts.sh
       if: success()
@@ -105,7 +101,7 @@ jobs:
       uses: actions/upload-artifact@v2
       with:
         name: windows-artifacts
-        path: artifacts
+        path: ${{env.ARTIFACTS_DIRECTORY}}
   windows-test:
     name: win test
     env:
@@ -126,6 +122,8 @@ jobs:
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
+    - run: ci/lib.sh
+      shell: bash
     - name: test
       shell: bash
       run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
@@ -145,9 +143,6 @@ jobs:
       jobname: vs-build
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
-    env:
-      NO_PERL: 1
-      GIT_CONFIG_PARAMETERS: "'user.name=CI' 'user.email=ci@git'"
     runs-on: windows-latest
     steps:
     - uses: actions/checkout@v2
@@ -171,6 +166,8 @@ jobs:
     - name: copy dlls to root
       shell: cmd
       run: compat\vcbuild\vcpkg_copy_dlls.bat release
+    - run: ci/lib.sh
+      shell: bash
     - name: generate Visual Studio solution
       shell: bash
       run: |
@@ -181,21 +178,16 @@ jobs:
     - name: bundle artifact tar
       shell: bash
       env:
-        MSVC: 1
         VCPKG_ROOT: ${{github.workspace}}\compat\vcbuild\vcpkg
       run: |
-        mkdir -p artifacts &&
-        eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts NO_GETTEXT=YesPlease 2>&1 | grep ^tar)"
-    - run: ci/check-unignored-build-artifacts.sh
-      if: success()
-      shell: bash
+        eval "$(make -n artifacts-tar 2>&1 | grep -e ^mkdir -e ^tar)"
     - name: zip up tracked files
       run: git archive -o artifacts/tracked.tar.gz HEAD
     - name: upload tracked files and build artifacts
       uses: actions/upload-artifact@v2
       with:
         name: vs-artifacts
-        path: artifacts
+        path: ${{env.ARTIFACTS_DIRECTORY}}
   vs-test:
     name: win+VS test
     env:
@@ -216,10 +208,10 @@ jobs:
     - name: extract tracked files and build artifacts
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
+    - run: ci/lib.sh
+      shell: bash
     - name: test
       shell: bash
-      env:
-        NO_SVN_TESTS: 1
       run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
     - name: ci/print-test-failures.sh
       if: failure()
@@ -354,6 +346,7 @@ jobs:
     - uses: actions/checkout@v2
     - name: Install other dependencies
       run: ci/install-dependencies.sh
+    - run: ci/lib.sh
     - run: make sparse
   documentation:
     name: documentation
diff --git a/ci/lib.sh b/ci/lib.sh
index fc6ce4d3e04..b882849ed5d 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -95,6 +95,27 @@ macos-latest)
 esac
 
 case "$jobname" in
+windows-build)
+	setenv --build NO_PERL NoThanks
+	setenv --build ARTIFACTS_DIRECTORY artifacts
+	;;
+vs-build)
+	setenv --build NO_PERL NoThanks
+	setenv --build NO_GETTEXT NoThanks
+	setenv --build ARTIFACTS_DIRECTORY artifacts
+	setenv --build INCLUDE_DLLS_IN_ARTIFACTS YesPlease
+	setenv --build MSVC YesPlease
+
+	setenv --build GIT_CONFIG_COUNT 2
+	setenv --build GIT_CONFIG_KEY_0 user.name
+	setenv --build GIT_CONFIG_VALUE_0 CI
+	setenv --build GIT_CONFIG_KEY_1 user.emailname
+	setenv --build GIT_CONFIG_VALUE_1 ci@git
+	setenv --build GIT_CONFIG_VALUE_1 ci@git
+	;;
+vs-test)
+	setenv --test NO_SVN_TESTS YesPlease
+	;;
 linux-gcc)
 	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
 	;;
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 18/25] ci/run-test-slice.sh: replace shelling out with "echo"

[Ævar Arnfjörð Bjarmason]

Amend this code added in b819f1d2cec (ci: parallelize testing on
Windows, 2019-01-29) to stop shelling out to "tr", all we're trying to
do here is to turn '\n'-delimited output into a ' '-delimited list. To
do that we can just use "echo".

None of these files will contain the odd characters using "echo" might
choke on in the general case (i.e. '\'-escapes and the like).

This has the added benefit of not adding a redundant ' ' to the end of
the T-parameter, which didn't matter in practice, but as we'll see in
a subsequent commit makes for slightly nicer output, as we'll show
this variable using a GitHub CI-native feature.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/run-test-slice.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index 70326961454..1de6a18ca47 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -5,6 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-make --quiet -C t T="$(cd t &&
-	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
-	tr '\n' ' ')"
+tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \
+	t[0-9]*.sh))
+make --quiet -C t T="$tests"
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 19/25] CI: pre-select test slice in Windows & VS tests

[Ævar Arnfjörð Bjarmason]

In preceding commits the tests have been changed to do their setup via
$GITHUB_ENV in one step, and to then have subsequent steps that re-use
that environment.

Let's change the "test slice" tests added in b819f1d2cec (ci:
parallelize testing on Windows, 2019-01-29) to do the same. These
tests select 10% of the tests to run in 10 "test slices". Now we'll
select those in a step that immediately precedes the testing step, and
then simply invoke "make -C t -e".

This has the advantage that the tests to be run are now listed in the
standard "Run" drop-down at the start of the "test" step.

Since the "T" variable in "t/Makefile" doesn't normally accept
overrides from the environment we need to invoke "make" with the "-e"
option (a.k.a. "--environment-overrides"). We could also make $(T) in
t/Makefile be a "?=" assigned variable, but this way works, and is
arguably clearer as it's more obvious that we're injecting a special
list of tests that override the normal behavior of that Makefile.

Note that we cannot run the top-level "make test" here, because of how
the Windows CI builds git, i.e. either via CMake or some option that
would cause "make test" to recompile git itself. Instead we run "make
-C t [...]".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml                     | 10 ++++++++--
 ci/{run-test-slice.sh => select-test-slice.sh} |  4 ++--
 2 files changed, 10 insertions(+), 4 deletions(-)
 rename ci/{run-test-slice.sh => select-test-slice.sh} (57%)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index fe37e0f1b36..1e13718b989 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -124,9 +124,12 @@ jobs:
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
     - run: ci/lib.sh
       shell: bash
+    - name: select tests
+      run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
+      shell: bash
     - name: test
       shell: bash
-      run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
+      run: . /etc/profile && make -C t -e
     - name: ci/print-test-failures.sh
       if: failure()
       shell: bash
@@ -210,9 +213,12 @@ jobs:
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
     - run: ci/lib.sh
       shell: bash
+    - name: select tests
+      run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
+      shell: bash
     - name: test
       shell: bash
-      run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
+      run: . /etc/profile && make -C t -e
     - name: ci/print-test-failures.sh
       if: failure()
       shell: bash
diff --git a/ci/run-test-slice.sh b/ci/select-test-slice.sh
similarity index 57%
rename from ci/run-test-slice.sh
rename to ci/select-test-slice.sh
index 1de6a18ca47..ec602f8a053 100755
--- a/ci/run-test-slice.sh
+++ b/ci/select-test-slice.sh
@@ -1,10 +1,10 @@
 #!/bin/sh
 #
-# Test Git in parallel
+# Select a portion of the tests for testing Git in parallel
 #
 
 . ${0%/*}/lib.sh
 
 tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \
 	t[0-9]*.sh))
-make --quiet -C t T="$tests"
+echo T="$tests" >>$GITHUB_ENV
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 20/25] CI: only invoke ci/lib.sh as "steps" in main.yml

[Ævar Arnfjörð Bjarmason]

Change the scripts in ci/ to stop using ci/lib.sh as a library, now
that the only thing it did for them was to "set -ex" and possibly set
TERM=dumb.

Let's create a ci/lib-tput.sh for those that need to use "tput"
instead, and have these scripts invoke "set -ex" themselves.

This makes their invocation a lot less verbose, since they'll be
relying on an earlier step in the CI job to have set the variables in
$GITHUB_ENV, and won't be spewing their own trace output to set those
variables again.

Let's also create a ci/lib-ci-type.sh, and have ci/lib.sh and
ci/print-test-failures.sh share the logic to discover the CI type. We
could have set the CI_TYPE in the environment with "setenv", but let's
avoid that verbosity for this purely internal variable.

The "ci/lib.sh" is now no longer a "Library of functions shared by all
CI scripts", so let's remove that commentary, and the misleading
comment about "set -ex" being for "installing dependencies", we're now
no longer using it in "ci/install-dependencies.sh" (but it does its
own "set -ex").

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/check-unignored-build-artifacts.sh |  4 +++-
 ci/install-dependencies.sh            |  2 +-
 ci/lib-ci-type.sh                     |  8 ++++++++
 ci/lib-tput.sh                        |  2 ++
 ci/lib.sh                             | 27 ++++++++++-----------------
 ci/print-test-failures.sh             |  6 +++---
 ci/select-test-slice.sh               |  2 +-
 ci/test-documentation.sh              |  2 +-
 8 files changed, 29 insertions(+), 24 deletions(-)
 create mode 100644 ci/lib-ci-type.sh
 create mode 100644 ci/lib-tput.sh

diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh
index 0bc04f32804..c27d6a97f45 100755
--- a/ci/check-unignored-build-artifacts.sh
+++ b/ci/check-unignored-build-artifacts.sh
@@ -3,7 +3,9 @@
 # Check whether the build created anything not in our .gitignore
 #
 
-. ${0%/*}/lib.sh
+set -ex
+
+. ${0%/*}/lib-tput.sh
 
 check_unignored_build_artifacts ()
 {
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 9388289d0ed..ee9af62fc92 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -3,7 +3,7 @@
 # Install dependencies required to build and test Git on Linux and macOS
 #
 
-. ${0%/*}/lib.sh
+set -ex
 
 UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
  tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
diff --git a/ci/lib-ci-type.sh b/ci/lib-ci-type.sh
new file mode 100644
index 00000000000..bd6e093c8f4
--- /dev/null
+++ b/ci/lib-ci-type.sh
@@ -0,0 +1,8 @@
+if test "$GITHUB_ACTIONS" = "true"
+then
+	CI_TYPE=github-actions
+else
+	echo "Could not identify CI type" >&2
+	env >&2
+	exit 1
+fi
diff --git a/ci/lib-tput.sh b/ci/lib-tput.sh
new file mode 100644
index 00000000000..baed1892f69
--- /dev/null
+++ b/ci/lib-tput.sh
@@ -0,0 +1,2 @@
+# GitHub Action doesn't set TERM, which is required by tput
+export TERM=${TERM:-dumb}
diff --git a/ci/lib.sh b/ci/lib.sh
index b882849ed5d..5eadc96de29 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,13 +1,9 @@
 #!/bin/sh
-
-# Library of functions shared by all CI scripts
-
-# Set 'exit on error' for all CI scripts to let the caller know that
-# something went wrong.
-# Set tracing executed commands, primarily setting environment variables
-# and installing dependencies.
 set -ex
 
+# Helper libraries
+. ${0%/*}/lib-ci-type.sh
+
 # Starting assertions
 if test -z "$jobname"
 then
@@ -39,15 +35,11 @@ setenv () {
 	fi
 }
 
-# GitHub Action doesn't set TERM, which is required by tput
-setenv TERM ${TERM:-dumb}
-
 # Clear MAKEFLAGS that may come from the outside world.
 MAKEFLAGS=
 
-if test "$GITHUB_ACTIONS" = "true"
-then
-	CI_TYPE=github-actions
+case "$CI_TYPE" in
+github-actions)
 	CC="${CC:-gcc}"
 
 	setenv --test GIT_PROVE_OPTS "--timer --jobs 10"
@@ -57,11 +49,12 @@ then
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 
 	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
-else
-	echo "Could not identify CI type" >&2
-	env >&2
+	;;
+*)
+	echo "Unhandled CI type: $CI_TYPE" >&2
 	exit 1
-fi
+	;;
+esac
 
 setenv --build DEVELOPER 1
 setenv --test DEFAULT_TEST_TARGET prove
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 0c63b6f7962..452aff35d74 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -3,10 +3,10 @@
 # Print output of failing tests
 #
 
-. ${0%/*}/lib.sh
+set -e
 
-# Tracing executed commands would produce too much noise in the loop below.
-set +x
+. ${0%/*}/lib-ci-type.sh
+. ${0%/*}/lib-tput.sh
 
 cd t/
 
diff --git a/ci/select-test-slice.sh b/ci/select-test-slice.sh
index ec602f8a053..a0332095bb3 100755
--- a/ci/select-test-slice.sh
+++ b/ci/select-test-slice.sh
@@ -3,7 +3,7 @@
 # Select a portion of the tests for testing Git in parallel
 #
 
-. ${0%/*}/lib.sh
+set -ex
 
 tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \
 	t[0-9]*.sh))
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index b8a6a6f664e..64ff212cdaa 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -3,7 +3,7 @@
 # Perform sanity checks on "make doc" output and built documentation
 #
 
-. ${0%/*}/lib.sh
+set -ex
 
 generator=$1
 
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 21/25] CI: narrow down variable definitions in --build and --test

[Ævar Arnfjörð Bjarmason]

In a preceding step the "setvar" function was made to take a
"--build", "--test" or "--all" argument to indicate where the
variables it sets were used.

Let's make use of that by having the relevant parts of
".github/workflows/main.yml" invoke "ci/lib.sh" with those options.

By doing this the set of variables shown in build-only steps will be
fewer, which makes diagnosing anything going on there easier, as we
won't have to look at a deluge of e.g. GIT_TEST_* variables.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 20 +++++++++++---------
 ci/lib.sh                  | 17 +++++++++++++++++
 2 files changed, 28 insertions(+), 9 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 1e13718b989..0787cadc76b 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -84,7 +84,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
       shell: bash
     - name: build
       shell: bash
@@ -122,7 +122,7 @@ jobs:
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
-    - run: ci/lib.sh
+    - run: ci/lib.sh --test
       shell: bash
     - name: select tests
       run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
@@ -169,7 +169,7 @@ jobs:
     - name: copy dlls to root
       shell: cmd
       run: compat\vcbuild\vcpkg_copy_dlls.bat release
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
       shell: bash
     - name: generate Visual Studio solution
       shell: bash
@@ -211,7 +211,7 @@ jobs:
     - name: extract tracked files and build artifacts
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
-    - run: ci/lib.sh
+    - run: ci/lib.sh --test
       shell: bash
     - name: select tests
       run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
@@ -275,8 +275,9 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make
+    - run: ci/lib.sh --test
     - run: make test
       if: success()
     - run: ci/print-test-failures.sh
@@ -310,8 +311,9 @@ jobs:
     steps:
     - uses: actions/checkout@v1
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make
+    - run: ci/lib.sh --test
     - run: make test
       if: success() && matrix.vector.skip-tests != 'no'
     - run: ci/print-test-failures.sh
@@ -331,7 +333,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make ci-static-analysis
   sparse:
     needs: ci-config
@@ -352,7 +354,7 @@ jobs:
     - uses: actions/checkout@v2
     - name: Install other dependencies
       run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make sparse
   documentation:
     name: documentation
@@ -364,7 +366,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make check-docs
     - run: "make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)"
       shell: bash
diff --git a/ci/lib.sh b/ci/lib.sh
index 5eadc96de29..367d1ee05d5 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -5,6 +5,13 @@ set -ex
 . ${0%/*}/lib-ci-type.sh
 
 # Starting assertions
+mode=$1
+if test -z "$mode"
+then
+	echo "need a $0 mode, e.g. --build or --test"
+	exit 1
+fi
+
 if test -z "$jobname"
 then
 	echo "must set a CI jobname" >&2
@@ -13,9 +20,14 @@ fi
 
 # Helper functions
 setenv () {
+	skip=
 	varmode=
 	case "$1" in
 	--*)
+		if test "$1" != "$mode" && test "$1" != "--all"
+		then
+			skip=t
+		fi
 		varmode=$1
 		shift
 		;;
@@ -25,6 +37,11 @@ setenv () {
 	val=$2
 	shift 2
 
+	if test -n "$skip"
+	then
+		return 0
+	fi
+
 	if test -n "$GITHUB_ENV"
 	then
 		echo "$key=$val" >>"$GITHUB_ENV"
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 22/25] CI: add more variables to MAKEFLAGS, except under vs-build

[Ævar Arnfjörð Bjarmason]

It's clearer that "DEVELOPER" is a flag that affects the Makefile
itself in particular if it's put into "MAKEFLAGS than" if it
generically sits in the environment. Let's move both it and
"SKIP_DASHED_BUILT_INS" to "MAKEFLAGS".

We can't do this under vs-build, since that invokes cmake. Let's have
only that job set these in the environment.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 367d1ee05d5..475e9f63a74 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -55,6 +55,13 @@ setenv () {
 # Clear MAKEFLAGS that may come from the outside world.
 MAKEFLAGS=
 
+# Common make and cmake build options
+DEVELOPER=1
+SKIP_DASHED_BUILT_INS=YesPlease
+
+# Use common options for "make" (cmake in "vs-build" below)
+MAKEFLAGS="DEVELOPER=$DEVELOPER SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
+
 case "$CI_TYPE" in
 github-actions)
 	CC="${CC:-gcc}"
@@ -73,10 +80,8 @@ github-actions)
 	;;
 esac
 
-setenv --build DEVELOPER 1
 setenv --test DEFAULT_TEST_TARGET prove
 setenv --test GIT_TEST_CLONE_2GB true
-setenv --build SKIP_DASHED_BUILT_INS YesPlease
 
 case "$runs_on_pool" in
 ubuntu-latest)
@@ -110,6 +115,9 @@ windows-build)
 	setenv --build ARTIFACTS_DIRECTORY artifacts
 	;;
 vs-build)
+	setenv --build DEVELOPER $DEVELOPER
+	setenv --build SKIP_DASHED_BUILT_INS $SKIP_DASHED_BUILT_INS
+
 	setenv --build NO_PERL NoThanks
 	setenv --build NO_GETTEXT NoThanks
 	setenv --build ARTIFACTS_DIRECTORY artifacts
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 23/25] CI: set CC in MAKEFLAGS directly, don't add it to the environment

[Ævar Arnfjörð Bjarmason]

Rather than pass a "$CC" and "$CC_PACKAGE" in the environment to be
picked up in ci/lib.sh let's instead have ci/lib.sh itself add it
directly to MAKEFLAGS.

Setting CC=gcc by default made for confusing trace output, and since a
preceding change to carry it and others over across "steps" in the
GitHub CI it's been even more misleading.  E.g. the "win+VS build" job
confusingly has CC=gcc set, even though it builds with MSVC.

Let's instead reply on the Makefile default of CC=cc, and only
override it for those jobs where it's needed. This does mean that
we'll need to set it for the "pedantic" job, which previously relied
on the default CC=gcc in case "clang" become the default on that
platform.

This partially reverts my 707d2f2fe86 (CI: use "$runs_on_pool", not
"$jobname" to select packages & config, 2021-11-23), i.e. we're now
aiming to only set those variables specific jobs need.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 13 -------------
 ci/lib.sh                  | 27 +++++++++++++++++++++++----
 2 files changed, 23 insertions(+), 17 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 0787cadc76b..6d25ec4ae3b 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -238,37 +238,24 @@ jobs:
       matrix:
         vector:
           - jobname: linux-clang
-            cc: clang
             pool: ubuntu-latest
           - jobname: linux-sha256
-            cc: clang
             os: ubuntu
             pool: ubuntu-latest
           - jobname: linux-gcc
-            cc: gcc
-            cc_package: gcc-8
             pool: ubuntu-latest
           - jobname: linux-TEST-vars
-            cc: gcc
             os: ubuntu
-            cc_package: gcc-8
             pool: ubuntu-latest
           - jobname: osx-clang
-            cc: clang
             pool: macos-latest
           - jobname: osx-gcc
-            cc: gcc
-            cc_package: gcc-9
             pool: macos-latest
           - jobname: linux-gcc-default
-            cc: gcc
             pool: ubuntu-latest
           - jobname: linux-leaks
-            cc: gcc
             pool: ubuntu-latest
     env:
-      CC: ${{matrix.vector.cc}}
-      CC_PACKAGE: ${{matrix.vector.cc_package}}
       jobname: ${{matrix.vector.jobname}}
       runs_on_pool: ${{matrix.vector.pool}}
     runs-on: ${{matrix.vector.pool}}
diff --git a/ci/lib.sh b/ci/lib.sh
index 475e9f63a74..57c9117a35a 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -52,7 +52,9 @@ setenv () {
 	fi
 }
 
-# Clear MAKEFLAGS that may come from the outside world.
+# Clear variables that may come from the outside world.
+CC=
+CC_PACKAGE=
 MAKEFLAGS=
 
 # Common make and cmake build options
@@ -64,8 +66,6 @@ MAKEFLAGS="DEVELOPER=$DEVELOPER SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
 
 case "$CI_TYPE" in
 github-actions)
-	CC="${CC:-gcc}"
-
 	setenv --test GIT_PROVE_OPTS "--timer --jobs 10"
 	GIT_TEST_OPTS="--verbose-log -x"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
@@ -135,9 +135,16 @@ vs-test)
 	setenv --test NO_SVN_TESTS YesPlease
 	;;
 linux-gcc)
+	CC=gcc
+	CC_PACKAGE=gcc-8
 	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
 	;;
+linux-gcc-default)
+	CC=gcc
+	;;
 linux-TEST-vars)
+	CC=gcc
+	CC_PACKAGE=gcc-8
 	setenv --test GIT_TEST_SPLIT_INDEX yes
 	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
 	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
@@ -152,13 +159,23 @@ linux-TEST-vars)
 	setenv --test GIT_TEST_WRITE_REV_INDEX 1
 	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
 	;;
+osx-gcc)
+	CC=gcc
+	CC_PACKAGE=gcc-9
+	;;
+osx-clang)
+	CC=clang
+	;;
 linux-clang)
+	CC=clang
 	setenv --test GIT_TEST_DEFAULT_HASH sha1
 	;;
 linux-sha256)
+	CC=clang
 	setenv --test GIT_TEST_DEFAULT_HASH sha256
 	;;
 pedantic)
+	CC=gcc
 	# Don't run the tests; we only care about whether Git can be
 	# built.
 	setenv --build DEVOPTS pedantic
@@ -173,9 +190,11 @@ linux-musl)
 	MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8"
 	;;
 linux-leaks)
+	CC=gcc
 	setenv --build SANITIZE leak
 	setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true
 	;;
 esac
 
-setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
+MAKEFLAGS="$MAKEFLAGS${CC:+ CC=$CC}"
+setenv --all MAKEFLAGS "$MAKEFLAGS"
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 24/25] CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case

[Ævar Arnfjörð Bjarmason]

Move the setting of "PYTHON_PATH=[...]" to the "$jobname" case
statement. This partially backs out of my 707d2f2fe86 (CI: use
"$runs_on_pool", not "$jobname" to select packages & config,
2021-11-23), now that we have a "osx-{clang,gcc}" anyway for setting
"$CC" we might as well do away with this part of the "$runs_on_pool"
case.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 57c9117a35a..663cbdef9d1 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -99,14 +99,6 @@ ubuntu-latest)
 
 	setenv --test GIT_TEST_HTTPD true
 	;;
-macos-latest)
-	if [ "$jobname" = osx-gcc ]
-	then
-		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
-	else
-		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
-	fi
-	;;
 esac
 
 case "$jobname" in
@@ -160,10 +152,12 @@ linux-TEST-vars)
 	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
 	;;
 osx-gcc)
+	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
 	CC=gcc
 	CC_PACKAGE=gcc-9
 	;;
 osx-clang)
+	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
 	CC=clang
 	;;
 linux-clang)
-- 
2.35.1.1517.g20a06c426a7

[PATCH v2 25/25] CI: don't use "set -x" in "ci/lib.sh" output

[Ævar Arnfjörð Bjarmason]

Remove the "set -x" from the "ci/lib.sh" output. Before preceding
commits the logic in that file was much more complex, and likely to
fail in some scenarios.

Now we only task "ci/lib.sh" with setting various variables for
subsequent steps in our jobs, so we can start emitting more tailored
debugging output, which makes what it's doing easier to read.

This change also changes the output of the "ci/print-test-failures.sh"
script, since it's the only other user of "ci/lib-ci-type.sh". In that
case it's also helpful to know what "$CI_TYPE" we're using, as that
script doesn't "set -x" and will act differently depending on the
$CI_TYPE.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib-ci-type.sh | 2 ++
 ci/lib.sh         | 9 ++++++++-
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/ci/lib-ci-type.sh b/ci/lib-ci-type.sh
index bd6e093c8f4..6cfe58596d2 100644
--- a/ci/lib-ci-type.sh
+++ b/ci/lib-ci-type.sh
@@ -6,3 +6,5 @@ else
 	env >&2
 	exit 1
 fi
+
+echo "CONFIG: CI_TYPE=$CI_TYPE" >&2
diff --git a/ci/lib.sh b/ci/lib.sh
index 663cbdef9d1..304380958bd 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,5 +1,5 @@
 #!/bin/sh
-set -ex
+set -e
 
 # Helper libraries
 . ${0%/*}/lib-ci-type.sh
@@ -11,12 +11,16 @@ then
 	echo "need a $0 mode, e.g. --build or --test"
 	exit 1
 fi
+echo "CONFIG: mode=$mode" >&2
 
 if test -z "$jobname"
 then
 	echo "must set a CI jobname" >&2
 	exit 1
 fi
+echo "CONFIG: jobname=$jobname" >&2
+echo "CONFIG: runs_on_pool=$runs_on_pool" >&2
+echo "CONFIG: GITHUB_ENV=$GITHUB_ENV" >&2
 
 # Helper functions
 setenv () {
@@ -39,6 +43,7 @@ setenv () {
 
 	if test -n "$skip"
 	then
+		echo "SKIP '$key=$val'" >&2
 		return 0
 	fi
 
@@ -50,6 +55,8 @@ setenv () {
 		# itself.
 		eval "export $key=\"$val\""
 	fi
+
+	echo "SET: '$key=$val'" >&2
 }
 
 # Clear variables that may come from the outside world.
-- 
2.35.1.1517.g20a06c426a7

[RFC PATCH v3 0/6] CI: js/ci-github-workflow-markup rebased on "use $GITHUB_ENV"

[Ævar Arnfjörð Bjarmason]

This is a rebasing of v2 of js/ci-github-workflow-markup[1] on top of
my just-submitted v2 of my own CI changes[2] which make most of that
series's changes to ci/ unnecessary, as it'll get the split of "build"
and "test" output for free without the need for any special GitHub
workflow syntax.

That's because that series eliminates the monolithic:

    - ci/run-build-and-tests.sh

In favor of:

    - make
    - make test

So what it did with markup to delineate those two is now done
automatically by focusing on the relevant "step" within a job that
failed.

A demo of a failing run for this series (with an intentionally broken
patch on top) can be seen at
https://github.com/avar/git/actions/runs/2041259003

As it shows the GitHub workflow syntax for test failures is retained,
e.g.:
https://github.com/avar/git/runs/5695803104?check_suite_focus=true

But for build failures the GitHub workflow syntax isn't needed
anymore, e.g. this job where we focus on the failing "make":
https://github.com/avar/git/runs/5695557199?check_suite_focus=true

Note that the UI of the latter is also better because we can see in
the "skipped" portion that we didn't run the "make test", since we're
now piggy-backing on the CI flow to show that, as opposed to in
js/ci-github-workflow-markup:
https://github.com/git-for-windows/git/runs/4822802185?check_suite_focus=true

The 5/6 change here (as seen in the range diff) the gets rid of a
limitation that js/ci-github-workflow-markup commented on being
unfortunate, i.e. we can now do more with the GitHub workflow syntax
as a result of not usign it for the "top-level" of "make" and "make
test".

1. https://lore.kernel.org/git/pull.1117.v2.git.1646130289.gitgitgadget@gmail.com/
2. https://lore.kernel.org/git/cover-v2-00.25-00000000000-20220325T182534Z-avarab@gmail.com

Johannes Schindelin (6):
  ci: make it easier to find failed tests' logs in the GitHub workflow
  tests: refactor --write-junit-xml code
  test(junit): avoid line feeds in XML attributes
  ci: optionally mark up output in the GitHub workflow
  ci: use `--github-workflow-markup` in the GitHub workflow
  ci: call `finalize_test_case_output` a little later

 .github/workflows/main.yml           |  20 +---
 ci/lib.sh                            |   2 +-
 ci/print-test-failures-github.sh     |  35 +++++++
 t/test-lib-functions.sh              |   4 +-
 t/test-lib-github-workflow-markup.sh |  50 ++++++++++
 t/test-lib-junit.sh                  | 132 +++++++++++++++++++++++++++
 t/test-lib.sh                        | 128 ++++----------------------
 7 files changed, 244 insertions(+), 127 deletions(-)
 create mode 100755 ci/print-test-failures-github.sh
 create mode 100644 t/test-lib-github-workflow-markup.sh
 create mode 100644 t/test-lib-junit.sh

Range-diff against v2:
 1:  db08b07c37a <  -:  ----------- ci: fix code style
 2:  42ff3e170bf <  -:  ----------- ci/run-build-and-tests: take a more high-level view
 3:  bbbe1623257 !  1:  d88749c60c9 ci: make it easier to find failed tests' logs in the GitHub workflow
    @@ Commit message
         nested groupings in GitHub workflows' output).
     
         Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## .github/workflows/main.yml ##
     @@ .github/workflows/main.yml: jobs:
    +       shell: bash
          - name: test
            shell: bash
    -       run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
    +-      run: . /etc/profile && make -C t -e
     -    - name: ci/print-test-failures.sh
     -      if: failure()
     -      shell: bash
     -      run: ci/print-test-failures.sh
    ++      run: . /etc/profile && make -C t -e || ci/print-test-failures-github.sh
          - name: Upload failed tests' directories
            if: failure() && env.FAILED_TEST_ARTIFACTS != ''
            uses: actions/upload-artifact@v2
     @@ .github/workflows/main.yml: jobs:
    -       env:
    -         NO_SVN_TESTS: 1
    -       run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
    +       shell: bash
    +     - name: test
    +       shell: bash
    +-      run: . /etc/profile && make -C t -e
     -    - name: ci/print-test-failures.sh
     -      if: failure()
     -      shell: bash
     -      run: ci/print-test-failures.sh
    ++      run: . /etc/profile && make -C t -e || ci/print-test-failures-github.sh
          - name: Upload failed tests' directories
            if: failure() && env.FAILED_TEST_ARTIFACTS != ''
            uses: actions/upload-artifact@v2
     @@ .github/workflows/main.yml: jobs:
    -     - uses: actions/checkout@v2
    -     - run: ci/install-dependencies.sh
    -     - run: ci/run-build-and-tests.sh
    +     - run: ci/lib.sh --build
    +     - run: make
    +     - run: ci/lib.sh --test
    +-    - run: make test
    ++    - run: make test || ci/print-test-failures-github.sh
    +       if: success()
     -    - run: ci/print-test-failures.sh
     -      if: failure()
          - name: Upload failed tests' directories
            if: failure() && env.FAILED_TEST_ARTIFACTS != ''
            uses: actions/upload-artifact@v2
     @@ .github/workflows/main.yml: jobs:
    -     - uses: actions/checkout@v1
    -     - run: ci/install-docker-dependencies.sh
    -     - run: ci/run-build-and-tests.sh
    +     - run: ci/lib.sh --build
    +     - run: make
    +     - run: ci/lib.sh --test
    +-    - run: make test
    ++    - run: make test || ci/print-test-failures-github.sh
    +       if: success() && matrix.vector.skip-tests != 'no'
     -    - run: ci/print-test-failures.sh
    --      if: failure()
    +-      if: failure() && matrix.vector.skip-tests != 'no'
          - name: Upload failed tests' directories
            if: failure() && env.FAILED_TEST_ARTIFACTS != ''
            uses: actions/upload-artifact@v1
     
    - ## ci/lib.sh ##
    -@@ ci/lib.sh: check_unignored_build_artifacts () {
    - 	}
    - }
    - 
    -+handle_failed_tests () {
    -+	return 1
    -+}
    + ## ci/print-test-failures-github.sh (new) ##
    +@@
    ++#!/bin/sh
     +
    - # GitHub Action doesn't set TERM, which is required by tput
    - export TERM=${TERM:-dumb}
    - 
    -@@ ci/lib.sh: then
    - 	CI_JOB_ID="$GITHUB_RUN_ID"
    - 	CC="${CC:-gcc}"
    - 	DONT_SKIP_TAGS=t
    ++. ${0%/*}/lib-ci-type.sh
    ++
    ++set -e
    ++
    ++case "$CI_TYPE" in
    ++github-actions)
     +	handle_failed_tests () {
     +		mkdir -p t/failed-test-artifacts
     +		echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
    @@ ci/lib.sh: then
     +		done
     +		return 1
     +	}
    - 
    - 	cache_dir="$HOME/none"
    - 
    -
    - ## ci/run-build-and-tests.sh ##
    -@@ ci/run-build-and-tests.sh: esac
    - make
    - if test -n "$run_tests"
    - then
    --	make test
    -+	make test ||
    -+	handle_failed_tests
    - fi
    - check_unignored_build_artifacts
    - 
    -
    - ## ci/run-test-slice.sh ##
    -@@ ci/run-test-slice.sh: esac
    - 
    - make --quiet -C t T="$(cd t &&
    - 	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
    --	tr '\n' ' ')"
    -+	tr '\n' ' ')" ||
    ++	;;
    ++*)
    ++	echo "Unhandled CI type: $CI_TYPE" >&2
    ++	exit 1
    ++	;;
    ++esac
    ++
     +handle_failed_tests
    - 
    - check_unignored_build_artifacts
 4:  f72254a9ac6 <  -:  ----------- ci/run-build-and-tests: add some structure to the GitHub workflow output
 5:  9eda6574313 !  2:  ad1e1465a81 tests: refactor --write-junit-xml code
    @@ Commit message
         --color-moved-ws=allow-indentation-change <commit>`.
     
         Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## t/test-lib-junit.sh (new) ##
     @@
 6:  c8b240af749 !  3:  fc96e5b7296 test(junit): avoid line feeds in XML attributes
    @@ Commit message
         incorrect, so let's fix it.
     
         Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## t/test-lib-junit.sh ##
     @@ t/test-lib-junit.sh: finalize_test_case_output () {
 7:  15f199e810e !  4:  429c256ac62 ci: optionally mark up output in the GitHub workflow
    @@ Commit message
         to test cases that were expected to fail but didn't.
     
         Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## t/test-lib-functions.sh ##
     @@ t/test-lib-functions.sh: test_verify_prereq () {
 8:  91ea54f36c5 !  5:  72058db67b0 ci: use `--github-workflow-markup` in the GitHub workflow
    @@ Metadata
      ## Commit message ##
         ci: use `--github-workflow-markup` in the GitHub workflow
     
    -    This makes the output easier to digest.
    +    [Ævar: due to re-structuring on top of my series the {begin,end}_group
    +    in CI isn't needed at all to get "group" output for the test
    +    suite. This commit includes the now-squashed "ci/run-build-and-tests:
    +    add some structure to the GitHub workflow output":]
     
    -    Note: since workflow output currently cannot contain any nested groups
    -    (see https://github.com/actions/runner/issues/802 for details), we need
    -    to remove the explicit grouping that would span the entirety of each
    -    failed test script.
    +    The current output of Git's GitHub workflow can be quite confusing,
    +    especially for contributors new to the project.
    +
    +    To make it more helpful, let's introduce some collapsible grouping.
    +    Initially, readers will see the high-level view of what actually
    +    happened (did the build fail, or the test suite?). To drill down, the
    +    respective group can be expanded.
    +
    +    Note: sadly, workflow output currently cannot contain any nested groups
    +    (see https://github.com/actions/runner/issues/802 for details),
    +    therefore we take pains to ensure to end any previous group before
    +    starting a new one.
    +
    +    [Ævar: The above comment isn't true anymore, as that limitation has
    +    been removed by basing this on my patches to run "make" and "make
    +    test" directly from the top-level of main.yml.
    +
    +    Those are now effectively their own "group", effectively giving this
    +    stage another group "level" to use. This means that the equivalent of
    +    "make test" won't be on the same level as an individual test failure.
    +
    +    We no longer take any pains to ensure balanced group output as a
    +    result (which was a caveat the previous ci/lib.sh implementation had
    +    to deal with., We just need to "cat" the generated *.markup]
     
         Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## ci/lib.sh ##
    -@@ ci/lib.sh: then
    +@@ ci/lib.sh: MAKEFLAGS="DEVELOPER=$DEVELOPER SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
    + case "$CI_TYPE" in
    + github-actions)
    + 	setenv --test GIT_PROVE_OPTS "--timer --jobs 10"
    +-	GIT_TEST_OPTS="--verbose-log -x"
    ++	GIT_TEST_OPTS="--verbose-log -x --github-workflow-markup"
    + 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
    + 	test Windows != "$RUNNER_OS" ||
    + 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
    +
    + ## ci/print-test-failures-github.sh ##
    +@@ ci/print-test-failures-github.sh: github-actions)
      			test_name="${test_exit%.exit}"
      			test_name="${test_name##*/}"
      			printf "\\e[33m\\e[1m=== Failed test: ${test_name} ===\\e[m\\n"
    --			group "Failed test: $test_name" cat "t/test-results/$test_name.out"
    +-			cat "t/test-results/$test_name.out"
     +			cat "t/test-results/$test_name.markup"
      
      			trash_dir="t/trash directory.$test_name"
      			cp "t/test-results/$test_name.out" t/failed-test-artifacts/
    -@@ ci/lib.sh: then
    - 	cache_dir="$HOME/none"
    - 
    - 	export GIT_PROVE_OPTS="--timer --jobs 10"
    --	export GIT_TEST_OPTS="--verbose-log -x"
    -+	export GIT_TEST_OPTS="--verbose-log -x --github-workflow-markup"
    - 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
    - 	test windows != "$CI_OS_NAME" ||
    - 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 9:  be2a83f5da3 !  6:  1d2b94436fc ci: call `finalize_test_case_output` a little later
    @@ Commit message
         test case.
     
         Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## t/test-lib.sh ##
     @@ t/test-lib.sh: trap '{ code=$?; set +x; } 2>/dev/null; exit $code' INT TERM HUP
-- 
2.35.1.1517.g20a06c426a7

[RFC PATCH v3 1/6] ci: make it easier to find failed tests' logs in the GitHub workflow

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

When investigating a test failure, the time that matters most is the
time it takes from getting aware of the failure to displaying the output
of the failing test case.

You currently have to know a lot of implementation details when
investigating test failures in the CI runs. The first step is easy: the
failed job is marked quite clearly, but when opening it, the failed step
is expanded, which in our case is the one running
`ci/run-build-and-tests.sh`. This step, most notably, only offers a
high-level view of what went wrong: it prints the output of `prove`
which merely tells the reader which test script failed.

The actually interesting part is in the detailed log of said failed
test script. But that log is shown in the CI run's step that runs
`ci/print-test-failures.sh`. And that step is _not_ expanded in the web
UI by default. It is even marked as "successful", which makes it very
easy to miss that there is useful information hidden in there.

Let's help the reader by showing the failed tests' detailed logs in the
step that is expanded automatically, i.e. directly after the test suite
failed.

This also helps the situation where the _build_ failed and the
`print-test-failures` step was executed under the assumption that the
_test suite_ failed, and consequently failed to find any failed tests.

An alternative way to implement this patch would be to source
`ci/print-test-failures.sh` in the `handle_test_failures` function to
show these logs. However, over the course of the next few commits, we
want to introduce some grouping which would be harder to achieve that
way (for example, we do want a leaner, and colored, preamble for each
failed test script, and it would be trickier to accommodate the lack of
nested groupings in GitHub workflows' output).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml       | 20 ++++--------------
 ci/print-test-failures-github.sh | 35 ++++++++++++++++++++++++++++++++
 2 files changed, 39 insertions(+), 16 deletions(-)
 create mode 100755 ci/print-test-failures-github.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 6d25ec4ae3b..e55dda807f0 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -129,11 +129,7 @@ jobs:
       shell: bash
     - name: test
       shell: bash
-      run: . /etc/profile && make -C t -e
-    - name: ci/print-test-failures.sh
-      if: failure()
-      shell: bash
-      run: ci/print-test-failures.sh
+      run: . /etc/profile && make -C t -e || ci/print-test-failures-github.sh
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v2
@@ -218,11 +214,7 @@ jobs:
       shell: bash
     - name: test
       shell: bash
-      run: . /etc/profile && make -C t -e
-    - name: ci/print-test-failures.sh
-      if: failure()
-      shell: bash
-      run: ci/print-test-failures.sh
+      run: . /etc/profile && make -C t -e || ci/print-test-failures-github.sh
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v2
@@ -265,10 +257,8 @@ jobs:
     - run: ci/lib.sh --build
     - run: make
     - run: ci/lib.sh --test
-    - run: make test
+    - run: make test || ci/print-test-failures-github.sh
       if: success()
-    - run: ci/print-test-failures.sh
-      if: failure()
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v2
@@ -301,10 +291,8 @@ jobs:
     - run: ci/lib.sh --build
     - run: make
     - run: ci/lib.sh --test
-    - run: make test
+    - run: make test || ci/print-test-failures-github.sh
       if: success() && matrix.vector.skip-tests != 'no'
-    - run: ci/print-test-failures.sh
-      if: failure() && matrix.vector.skip-tests != 'no'
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v1
diff --git a/ci/print-test-failures-github.sh b/ci/print-test-failures-github.sh
new file mode 100755
index 00000000000..218aaf19763
--- /dev/null
+++ b/ci/print-test-failures-github.sh
@@ -0,0 +1,35 @@
+#!/bin/sh
+
+. ${0%/*}/lib-ci-type.sh
+
+set -e
+
+case "$CI_TYPE" in
+github-actions)
+	handle_failed_tests () {
+		mkdir -p t/failed-test-artifacts
+		echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
+
+		for test_exit in t/test-results/*.exit
+		do
+			test 0 != "$(cat "$test_exit")" || continue
+
+			test_name="${test_exit%.exit}"
+			test_name="${test_name##*/}"
+			printf "\\e[33m\\e[1m=== Failed test: ${test_name} ===\\e[m\\n"
+			cat "t/test-results/$test_name.out"
+
+			trash_dir="t/trash directory.$test_name"
+			cp "t/test-results/$test_name.out" t/failed-test-artifacts/
+			tar czf t/failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
+		done
+		return 1
+	}
+	;;
+*)
+	echo "Unhandled CI type: $CI_TYPE" >&2
+	exit 1
+	;;
+esac
+
+handle_failed_tests
-- 
2.35.1.1517.g20a06c426a7

[RFC PATCH v3 2/6] tests: refactor --write-junit-xml code

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The code writing JUnit XML is interspersed directly with all the code in
`t/test-lib.sh`, and it is therefore not only ill-separated, but
introducing yet another output format would make the situation even
worse.

Let's introduce an abstraction layer by hiding the JUnit XML code behind
four new functions that are supposed to be called before and after each
test and test case.

This is not just an academic exercise, refactoring for refactoring's
sake. We _actually_ want to introduce such a new output format, to
make it substantially easier to diagnose test failures in our GitHub
workflow, therefore we do need this refactoring.

This commit is best viewed with `git show --color-moved
--color-moved-ws=allow-indentation-change <commit>`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib-junit.sh | 126 ++++++++++++++++++++++++++++++++++++++++++++
 t/test-lib.sh       | 124 ++++++-------------------------------------
 2 files changed, 142 insertions(+), 108 deletions(-)
 create mode 100644 t/test-lib-junit.sh

diff --git a/t/test-lib-junit.sh b/t/test-lib-junit.sh
new file mode 100644
index 00000000000..9d55d74d764
--- /dev/null
+++ b/t/test-lib-junit.sh
@@ -0,0 +1,126 @@
+# Library of functions to format test scripts' output in JUnit XML
+# format, to support Git's test suite result to be presented in an
+# easily digestible way on Azure Pipelines.
+#
+# Copyright (c) 2022 Johannes Schindelin
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see http://www.gnu.org/licenses/ .
+#
+# The idea is for `test-lib.sh` to source this file when the user asks
+# for JUnit XML; these functions will then override (empty) functions
+# that are are called at the appropriate times during the test runs.
+
+start_test_output () {
+	junit_xml_dir="$TEST_OUTPUT_DIRECTORY/out"
+	mkdir -p "$junit_xml_dir"
+	junit_xml_base=${1##*/}
+	junit_xml_path="$junit_xml_dir/TEST-${junit_xml_base%.sh}.xml"
+	junit_attrs="name=\"${junit_xml_base%.sh}\""
+	junit_attrs="$junit_attrs timestamp=\"$(TZ=UTC \
+		date +%Y-%m-%dT%H:%M:%S)\""
+	write_junit_xml --truncate "<testsuites>" "  <testsuite $junit_attrs>"
+	junit_suite_start=$(test-tool date getnanos)
+	if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
+	then
+		GIT_TEST_TEE_OFFSET=0
+	fi
+}
+
+start_test_case_output () {
+	junit_start=$(test-tool date getnanos)
+}
+
+finalize_test_case_output () {
+	test_case_result=$1
+	shift
+	case "$test_case_result" in
+	ok)
+		set "$*"
+		;;
+	failure)
+		junit_insert="<failure message=\"not ok $test_count -"
+		junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
+		junit_insert="$junit_insert $(xml_attr_encode \
+			"$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
+			   then
+				test-tool path-utils skip-n-bytes \
+					"$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET
+			   else
+				printf '%s\n' "$@" | sed 1d
+			   fi)")"
+		junit_insert="$junit_insert</failure>"
+		if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
+		then
+			junit_insert="$junit_insert<system-err>$(xml_attr_encode \
+				"$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")</system-err>"
+		fi
+		set "$1" "      $junit_insert"
+		;;
+	fixed)
+		set "$* (breakage fixed)"
+		;;
+	broken)
+		set "$* (known breakage)"
+		;;
+	skip)
+		message="$(xml_attr_encode "$skipped_reason")"
+		set "$1" "      <skipped message=\"$message\" />"
+		;;
+	esac
+
+	junit_attrs="name=\"$(xml_attr_encode "$this_test.$test_count $1")\""
+	shift
+	junit_attrs="$junit_attrs classname=\"$this_test\""
+	junit_attrs="$junit_attrs time=\"$(test-tool \
+		date getnanos $junit_start)\""
+	write_junit_xml "$(printf '%s\n' \
+		"    <testcase $junit_attrs>" "$@" "    </testcase>")"
+	junit_have_testcase=t
+}
+
+finalize_test_output () {
+	if test -n "$junit_xml_path"
+	then
+		test -n "$junit_have_testcase" || {
+			junit_start=$(test-tool date getnanos)
+			write_junit_xml_testcase "all tests skipped"
+		}
+
+		# adjust the overall time
+		junit_time=$(test-tool date getnanos $junit_suite_start)
+		sed -e "s/\(<testsuite.*\) time=\"[^\"]*\"/\1/" \
+			-e "s/<testsuite [^>]*/& time=\"$junit_time\"/" \
+			-e '/^ *<\/testsuite/d' \
+			<"$junit_xml_path" >"$junit_xml_path.new"
+		mv "$junit_xml_path.new" "$junit_xml_path"
+
+		write_junit_xml "  </testsuite>" "</testsuites>"
+		write_junit_xml=
+	fi
+}
+
+write_junit_xml () {
+	case "$1" in
+	--truncate)
+		>"$junit_xml_path"
+		junit_have_testcase=
+		shift
+		;;
+	esac
+	printf '%s\n' "$@" >>"$junit_xml_path"
+}
+
+xml_attr_encode () {
+	printf '%s\n' "$@" | test-tool xml-encode
+}
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 515b1af7ed4..65b63359ddb 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -137,6 +137,12 @@ mark_option_requires_arg () {
 	store_arg_to=$2
 }
 
+# These functions can be overridden e.g. to output JUnit XML
+start_test_output () { :; }
+start_test_case_output () { :; }
+finalize_test_case_output () { :; }
+finalize_test_output () { :; }
+
 parse_option () {
 	local opt="$1"
 
@@ -196,7 +202,7 @@ parse_option () {
 		tee=t
 		;;
 	--write-junit-xml)
-		write_junit_xml=t
+		. "$TEST_DIRECTORY/test-lib-junit.sh"
 		;;
 	--stress)
 		stress=t ;;
@@ -663,7 +669,7 @@ exec 6<&0
 exec 7>&2
 
 _error_exit () {
-	finalize_junit_xml
+	finalize_test_output
 	GIT_EXIT_OK=t
 	exit 1
 }
@@ -773,35 +779,13 @@ trap '{ code=$?; set +x; } 2>/dev/null; exit $code' INT TERM HUP
 # the test_expect_* functions instead.
 
 test_ok_ () {
-	if test -n "$write_junit_xml"
-	then
-		write_junit_xml_testcase "$*"
-	fi
+	finalize_test_case_output ok "$@"
 	test_success=$(($test_success + 1))
 	say_color "" "ok $test_count - $@"
 }
 
 test_failure_ () {
-	if test -n "$write_junit_xml"
-	then
-		junit_insert="<failure message=\"not ok $test_count -"
-		junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
-		junit_insert="$junit_insert $(xml_attr_encode \
-			"$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
-			   then
-				test-tool path-utils skip-n-bytes \
-					"$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET
-			   else
-				printf '%s\n' "$@" | sed 1d
-			   fi)")"
-		junit_insert="$junit_insert</failure>"
-		if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
-		then
-			junit_insert="$junit_insert<system-err>$(xml_attr_encode \
-				"$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")</system-err>"
-		fi
-		write_junit_xml_testcase "$1" "      $junit_insert"
-	fi
+	finalize_test_case_output failure "$@"
 	test_failure=$(($test_failure + 1))
 	say_color error "not ok $test_count - $1"
 	shift
@@ -810,19 +794,13 @@ test_failure_ () {
 }
 
 test_known_broken_ok_ () {
-	if test -n "$write_junit_xml"
-	then
-		write_junit_xml_testcase "$* (breakage fixed)"
-	fi
+	finalize_test_case_output fixed "$@"
 	test_fixed=$(($test_fixed+1))
 	say_color error "ok $test_count - $@ # TODO known breakage vanished"
 }
 
 test_known_broken_failure_ () {
-	if test -n "$write_junit_xml"
-	then
-		write_junit_xml_testcase "$* (known breakage)"
-	fi
+	finalize_test_case_output broken "$@"
 	test_broken=$(($test_broken+1))
 	say_color warn "not ok $test_count - $@ # TODO known breakage"
 }
@@ -1099,10 +1077,7 @@ test_start_ () {
 	test_count=$(($test_count+1))
 	maybe_setup_verbose
 	maybe_setup_valgrind
-	if test -n "$write_junit_xml"
-	then
-		junit_start=$(test-tool date getnanos)
-	fi
+	start_test_case_output
 }
 
 test_finish_ () {
@@ -1153,12 +1128,7 @@ test_skip () {
 
 	case "$to_skip" in
 	t)
-		if test -n "$write_junit_xml"
-		then
-			message="$(xml_attr_encode "$skipped_reason")"
-			write_junit_xml_testcase "$1" \
-				"      <skipped message=\"$message\" />"
-		fi
+		finalize_test_case_output skip "$@"
 
 		say_color skip "ok $test_count # skip $1 ($skipped_reason)"
 		: true
@@ -1174,53 +1144,6 @@ test_at_end_hook_ () {
 	:
 }
 
-write_junit_xml () {
-	case "$1" in
-	--truncate)
-		>"$junit_xml_path"
-		junit_have_testcase=
-		shift
-		;;
-	esac
-	printf '%s\n' "$@" >>"$junit_xml_path"
-}
-
-xml_attr_encode () {
-	printf '%s\n' "$@" | test-tool xml-encode
-}
-
-write_junit_xml_testcase () {
-	junit_attrs="name=\"$(xml_attr_encode "$this_test.$test_count $1")\""
-	shift
-	junit_attrs="$junit_attrs classname=\"$this_test\""
-	junit_attrs="$junit_attrs time=\"$(test-tool \
-		date getnanos $junit_start)\""
-	write_junit_xml "$(printf '%s\n' \
-		"    <testcase $junit_attrs>" "$@" "    </testcase>")"
-	junit_have_testcase=t
-}
-
-finalize_junit_xml () {
-	if test -n "$write_junit_xml" && test -n "$junit_xml_path"
-	then
-		test -n "$junit_have_testcase" || {
-			junit_start=$(test-tool date getnanos)
-			write_junit_xml_testcase "all tests skipped"
-		}
-
-		# adjust the overall time
-		junit_time=$(test-tool date getnanos $junit_suite_start)
-		sed -e "s/\(<testsuite.*\) time=\"[^\"]*\"/\1/" \
-			-e "s/<testsuite [^>]*/& time=\"$junit_time\"/" \
-			-e '/^ *<\/testsuite/d' \
-			<"$junit_xml_path" >"$junit_xml_path.new"
-		mv "$junit_xml_path.new" "$junit_xml_path"
-
-		write_junit_xml "  </testsuite>" "</testsuites>"
-		write_junit_xml=
-	fi
-}
-
 test_atexit_cleanup=:
 test_atexit_handler () {
 	# In a succeeding test script 'test_atexit_handler' is invoked
@@ -1243,7 +1166,7 @@ test_done () {
 	# removed, so the commands can access pidfiles and socket files.
 	test_atexit_handler
 
-	finalize_junit_xml
+	finalize_test_output
 
 	if test -z "$HARNESS_ACTIVE"
 	then
@@ -1534,22 +1457,7 @@ fi
 # in subprocesses like git equals our $PWD (for pathname comparisons).
 cd -P "$TRASH_DIRECTORY" || exit 1
 
-if test -n "$write_junit_xml"
-then
-	junit_xml_dir="$TEST_OUTPUT_DIRECTORY/out"
-	mkdir -p "$junit_xml_dir"
-	junit_xml_base=${0##*/}
-	junit_xml_path="$junit_xml_dir/TEST-${junit_xml_base%.sh}.xml"
-	junit_attrs="name=\"${junit_xml_base%.sh}\""
-	junit_attrs="$junit_attrs timestamp=\"$(TZ=UTC \
-		date +%Y-%m-%dT%H:%M:%S)\""
-	write_junit_xml --truncate "<testsuites>" "  <testsuite $junit_attrs>"
-	junit_suite_start=$(test-tool date getnanos)
-	if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
-	then
-		GIT_TEST_TEE_OFFSET=0
-	fi
-fi
+start_test_output "$0"
 
 # Convenience
 # A regexp to match 5 and 35 hexdigits
-- 
2.35.1.1517.g20a06c426a7

[RFC PATCH v3 3/6] test(junit): avoid line feeds in XML attributes

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

In the test case's output, we do want newline characters, but in the XML
attributes we do not want them.

However, the `xml_attr_encode` function always adds a Line Feed at the
end (which are then encoded as `&#x0a;`, even for XML attributes.

This seems not to faze Azure Pipelines' XML parser, but it still is
incorrect, so let's fix it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib-junit.sh | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/t/test-lib-junit.sh b/t/test-lib-junit.sh
index 9d55d74d764..c959183c7e2 100644
--- a/t/test-lib-junit.sh
+++ b/t/test-lib-junit.sh
@@ -50,7 +50,7 @@ finalize_test_case_output () {
 		;;
 	failure)
 		junit_insert="<failure message=\"not ok $test_count -"
-		junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
+		junit_insert="$junit_insert $(xml_attr_encode --no-lf "$1")\">"
 		junit_insert="$junit_insert $(xml_attr_encode \
 			"$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
 			   then
@@ -74,12 +74,12 @@ finalize_test_case_output () {
 		set "$* (known breakage)"
 		;;
 	skip)
-		message="$(xml_attr_encode "$skipped_reason")"
+		message="$(xml_attr_encode --no-lf "$skipped_reason")"
 		set "$1" "      <skipped message=\"$message\" />"
 		;;
 	esac
 
-	junit_attrs="name=\"$(xml_attr_encode "$this_test.$test_count $1")\""
+	junit_attrs="name=\"$(xml_attr_encode --no-lf "$this_test.$test_count $1")\""
 	shift
 	junit_attrs="$junit_attrs classname=\"$this_test\""
 	junit_attrs="$junit_attrs time=\"$(test-tool \
@@ -122,5 +122,11 @@ write_junit_xml () {
 }
 
 xml_attr_encode () {
-	printf '%s\n' "$@" | test-tool xml-encode
+	if test "x$1" = "x--no-lf"
+	then
+		shift
+		printf '%s' "$*" | test-tool xml-encode
+	else
+		printf '%s\n' "$@" | test-tool xml-encode
+	fi
 }
-- 
2.35.1.1517.g20a06c426a7

[RFC PATCH v3 4/6] ci: optionally mark up output in the GitHub workflow

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

A couple of commands exist to spruce up the output in GitHub workflows:
https://docs.github.com/en/actions/learn-github-actions/workflow-commands-for-github-actions

In addition to the `::group::<label>`/`::endgroup::` commands (which we
already use to structure the output of the build step better), we also
use `::error::`/`::notice::` to draw the attention to test failures and
to test cases that were expected to fail but didn't.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib-functions.sh              |  4 +--
 t/test-lib-github-workflow-markup.sh | 50 ++++++++++++++++++++++++++++
 t/test-lib.sh                        |  5 ++-
 3 files changed, 56 insertions(+), 3 deletions(-)
 create mode 100644 t/test-lib-github-workflow-markup.sh

diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index 0f439c99d61..4e86f7d9d0e 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -719,7 +719,7 @@ test_verify_prereq () {
 }
 
 test_expect_failure () {
-	test_start_
+	test_start_ "$@"
 	test "$#" = 3 && { test_prereq=$1; shift; } || test_prereq=
 	test "$#" = 2 ||
 	BUG "not 2 or 3 parameters to test-expect-failure"
@@ -739,7 +739,7 @@ test_expect_failure () {
 }
 
 test_expect_success () {
-	test_start_
+	test_start_ "$@"
 	test "$#" = 3 && { test_prereq=$1; shift; } || test_prereq=
 	test "$#" = 2 ||
 	BUG "not 2 or 3 parameters to test-expect-success"
diff --git a/t/test-lib-github-workflow-markup.sh b/t/test-lib-github-workflow-markup.sh
new file mode 100644
index 00000000000..d8dc969df4a
--- /dev/null
+++ b/t/test-lib-github-workflow-markup.sh
@@ -0,0 +1,50 @@
+# Library of functions to mark up test scripts' output suitable for
+# pretty-printing it in GitHub workflows.
+#
+# Copyright (c) 2022 Johannes Schindelin
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see http://www.gnu.org/licenses/ .
+#
+# The idea is for `test-lib.sh` to source this file when run in GitHub
+# workflows; these functions will then override (empty) functions
+# that are are called at the appropriate times during the test runs.
+
+start_test_output () {
+	test -n "$GIT_TEST_TEE_OUTPUT_FILE" ||
+	die "--github-workflow-markup requires --verbose-log"
+	github_markup_output="${GIT_TEST_TEE_OUTPUT_FILE%.out}.markup"
+	>$github_markup_output
+	GIT_TEST_TEE_OFFSET=0
+}
+
+# No need to override start_test_case_output
+
+finalize_test_case_output () {
+	test_case_result=$1
+	shift
+	case "$test_case_result" in
+	failure)
+		echo >>$github_markup_output "::error::failed: $this_test.$test_count $1"
+		;;
+	fixed)
+		echo >>$github_markup_output "::notice::fixed: $this_test.$test_count $1"
+		;;
+	esac
+	echo >>$github_markup_output "::group::$test_case_result: $this_test.$test_count $*"
+	test-tool >>$github_markup_output path-utils skip-n-bytes \
+		"$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET
+	echo >>$github_markup_output "::endgroup::"
+}
+
+# No need to override finalize_test_output
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 65b63359ddb..5ba4051d588 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -204,6 +204,9 @@ parse_option () {
 	--write-junit-xml)
 		. "$TEST_DIRECTORY/test-lib-junit.sh"
 		;;
+	--github-workflow-markup)
+		. "$TEST_DIRECTORY/test-lib-github-workflow-markup.sh"
+		;;
 	--stress)
 		stress=t ;;
 	--stress=*)
@@ -1077,7 +1080,7 @@ test_start_ () {
 	test_count=$(($test_count+1))
 	maybe_setup_verbose
 	maybe_setup_valgrind
-	start_test_case_output
+	start_test_case_output "$@"
 }
 
 test_finish_ () {
-- 
2.35.1.1517.g20a06c426a7

[RFC PATCH v3 5/6] ci: use `--github-workflow-markup` in the GitHub workflow

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

[Ævar: due to re-structuring on top of my series the {begin,end}_group
in CI isn't needed at all to get "group" output for the test
suite. This commit includes the now-squashed "ci/run-build-and-tests:
add some structure to the GitHub workflow output":]

The current output of Git's GitHub workflow can be quite confusing,
especially for contributors new to the project.

To make it more helpful, let's introduce some collapsible grouping.
Initially, readers will see the high-level view of what actually
happened (did the build fail, or the test suite?). To drill down, the
respective group can be expanded.

Note: sadly, workflow output currently cannot contain any nested groups
(see https://github.com/actions/runner/issues/802 for details),
therefore we take pains to ensure to end any previous group before
starting a new one.

[Ævar: The above comment isn't true anymore, as that limitation has
been removed by basing this on my patches to run "make" and "make
test" directly from the top-level of main.yml.

Those are now effectively their own "group", effectively giving this
stage another group "level" to use. This means that the equivalent of
"make test" won't be on the same level as an individual test failure.

We no longer take any pains to ensure balanced group output as a
result (which was a caveat the previous ci/lib.sh implementation had
to deal with., We just need to "cat" the generated *.markup]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                        | 2 +-
 ci/print-test-failures-github.sh | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 304380958bd..f84bac0fd36 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -74,7 +74,7 @@ MAKEFLAGS="DEVELOPER=$DEVELOPER SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
 case "$CI_TYPE" in
 github-actions)
 	setenv --test GIT_PROVE_OPTS "--timer --jobs 10"
-	GIT_TEST_OPTS="--verbose-log -x"
+	GIT_TEST_OPTS="--verbose-log -x --github-workflow-markup"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
diff --git a/ci/print-test-failures-github.sh b/ci/print-test-failures-github.sh
index 218aaf19763..e432a601621 100755
--- a/ci/print-test-failures-github.sh
+++ b/ci/print-test-failures-github.sh
@@ -17,7 +17,7 @@ github-actions)
 			test_name="${test_exit%.exit}"
 			test_name="${test_name##*/}"
 			printf "\\e[33m\\e[1m=== Failed test: ${test_name} ===\\e[m\\n"
-			cat "t/test-results/$test_name.out"
+			cat "t/test-results/$test_name.markup"
 
 			trash_dir="t/trash directory.$test_name"
 			cp "t/test-results/$test_name.out" t/failed-test-artifacts/
-- 
2.35.1.1517.g20a06c426a7

[RFC PATCH v3 6/6] ci: call `finalize_test_case_output` a little later

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

We used to call that function already before printing the final verdict.
However, now that we added grouping to the GitHub workflow output, we
will want to include even that part in the collapsible group for that
test case.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib.sh | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/t/test-lib.sh b/t/test-lib.sh
index 5ba4051d588..b3ee37988e2 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -782,30 +782,31 @@ trap '{ code=$?; set +x; } 2>/dev/null; exit $code' INT TERM HUP
 # the test_expect_* functions instead.
 
 test_ok_ () {
-	finalize_test_case_output ok "$@"
 	test_success=$(($test_success + 1))
 	say_color "" "ok $test_count - $@"
+	finalize_test_case_output ok "$@"
 }
 
 test_failure_ () {
-	finalize_test_case_output failure "$@"
+	failure_label=$1
 	test_failure=$(($test_failure + 1))
 	say_color error "not ok $test_count - $1"
 	shift
 	printf '%s\n' "$*" | sed -e 's/^/#	/'
 	test "$immediate" = "" || _error_exit
+	finalize_test_case_output failure "$failure_label" "$@"
 }
 
 test_known_broken_ok_ () {
-	finalize_test_case_output fixed "$@"
 	test_fixed=$(($test_fixed+1))
 	say_color error "ok $test_count - $@ # TODO known breakage vanished"
+	finalize_test_case_output fixed "$@"
 }
 
 test_known_broken_failure_ () {
-	finalize_test_case_output broken "$@"
 	test_broken=$(($test_broken+1))
 	say_color warn "not ok $test_count - $@ # TODO known breakage"
+	finalize_test_case_output broken "$@"
 }
 
 test_debug () {
@@ -1131,10 +1132,10 @@ test_skip () {
 
 	case "$to_skip" in
 	t)
-		finalize_test_case_output skip "$@"
 
 		say_color skip "ok $test_count # skip $1 ($skipped_reason)"
 		: true
+		finalize_test_case_output skip "$@"
 		;;
 	*)
 		false
-- 
2.35.1.1517.g20a06c426a7

[RFC PATCH v4 0/6] CI: js/ci-github-workflow-markup rebased on "use $GITHUB_ENV"

[Ævar Arnfjörð Bjarmason]

See the "v3" for the "real" cover-letter:
https://lore.kernel.org/git/RFC-cover-v3-0.6-00000000000-20220325T183946Z-avarab@gmail.com/

The only change in this v4 is to rebase this series on the
just-submitted v3 of the parent series at:
https://lore.kernel.org/git/cover-v3-00.29-00000000000-20220413T194847Z-avarab@gmail.com/

Also seen in the range-diff is a rebasing on the --immediate behavior
for for t/test-lib.sh, which landed on master since tho "v3" was
submitted.

Johannes Schindelin (6):
  ci: make it easier to find failed tests' logs in the GitHub workflow
  tests: refactor --write-junit-xml code
  test(junit): avoid line feeds in XML attributes
  ci: optionally mark up output in the GitHub workflow
  ci: use `--github-workflow-markup` in the GitHub workflow
  ci: call `finalize_test_case_output` a little later

 .github/workflows/main.yml           |  20 +---
 ci/lib.sh                            |   3 +-
 ci/print-test-failures-github.sh     |  35 +++++++
 t/test-lib-functions.sh              |   4 +-
 t/test-lib-github-workflow-markup.sh |  50 ++++++++++
 t/test-lib-junit.sh                  | 132 +++++++++++++++++++++++++++
 t/test-lib.sh                        | 128 ++++----------------------
 7 files changed, 245 insertions(+), 127 deletions(-)
 create mode 100755 ci/print-test-failures-github.sh
 create mode 100644 t/test-lib-github-workflow-markup.sh
 create mode 100644 t/test-lib-junit.sh

Range-diff against v3:
1:  d88749c60c9 = 1:  cc137c69ee1 ci: make it easier to find failed tests' logs in the GitHub workflow
2:  ad1e1465a81 = 2:  91f96c4f210 tests: refactor --write-junit-xml code
3:  fc96e5b7296 = 3:  84c722969d5 test(junit): avoid line feeds in XML attributes
4:  429c256ac62 = 4:  8acaa800d3a ci: optionally mark up output in the GitHub workflow
5:  72058db67b0 ! 5:  4499f743dd1 ci: use `--github-workflow-markup` in the GitHub workflow
    @@ Commit message
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## ci/lib.sh ##
    -@@ ci/lib.sh: MAKEFLAGS="DEVELOPER=$DEVELOPER SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
    +@@ ci/lib.sh: MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
      case "$CI_TYPE" in
      github-actions)
    - 	setenv --test GIT_PROVE_OPTS "--timer --jobs 10"
    + 	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
     -	GIT_TEST_OPTS="--verbose-log -x"
     +	GIT_TEST_OPTS="--verbose-log -x --github-workflow-markup"
    - 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
    ++	MAKEFLAGS="$MAKEFLAGS --jobs=10"
      	test Windows != "$RUNNER_OS" ||
      	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
    + 	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
     
      ## ci/print-test-failures-github.sh ##
     @@ ci/print-test-failures-github.sh: github-actions)
6:  1d2b94436fc ! 6:  b291f64821c ci: call `finalize_test_case_output` a little later
    @@ t/test-lib.sh: trap '{ code=$?; set +x; } 2>/dev/null; exit $code' INT TERM HUP
      	test_failure=$(($test_failure + 1))
      	say_color error "not ok $test_count - $1"
      	shift
    - 	printf '%s\n' "$*" | sed -e 's/^/#	/'
    - 	test "$immediate" = "" || _error_exit
    +@@ t/test-lib.sh: test_failure_ () {
    + 		say_color error "1..$test_count"
    + 		_error_exit
    + 	fi
     +	finalize_test_case_output failure "$failure_label" "$@"
      }
      
-- 
2.36.0.rc2.843.g193535c2aa7

[RFC PATCH v4 1/6] ci: make it easier to find failed tests' logs in the GitHub workflow

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

When investigating a test failure, the time that matters most is the
time it takes from getting aware of the failure to displaying the output
of the failing test case.

You currently have to know a lot of implementation details when
investigating test failures in the CI runs. The first step is easy: the
failed job is marked quite clearly, but when opening it, the failed step
is expanded, which in our case is the one running
`ci/run-build-and-tests.sh`. This step, most notably, only offers a
high-level view of what went wrong: it prints the output of `prove`
which merely tells the reader which test script failed.

The actually interesting part is in the detailed log of said failed
test script. But that log is shown in the CI run's step that runs
`ci/print-test-failures.sh`. And that step is _not_ expanded in the web
UI by default. It is even marked as "successful", which makes it very
easy to miss that there is useful information hidden in there.

Let's help the reader by showing the failed tests' detailed logs in the
step that is expanded automatically, i.e. directly after the test suite
failed.

This also helps the situation where the _build_ failed and the
`print-test-failures` step was executed under the assumption that the
_test suite_ failed, and consequently failed to find any failed tests.

An alternative way to implement this patch would be to source
`ci/print-test-failures.sh` in the `handle_test_failures` function to
show these logs. However, over the course of the next few commits, we
want to introduce some grouping which would be harder to achieve that
way (for example, we do want a leaner, and colored, preamble for each
failed test script, and it would be trickier to accommodate the lack of
nested groupings in GitHub workflows' output).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml       | 20 ++++--------------
 ci/print-test-failures-github.sh | 35 ++++++++++++++++++++++++++++++++
 2 files changed, 39 insertions(+), 16 deletions(-)
 create mode 100755 ci/print-test-failures-github.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 6d25ec4ae3b..e55dda807f0 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -129,11 +129,7 @@ jobs:
       shell: bash
     - name: test
       shell: bash
-      run: . /etc/profile && make -C t -e
-    - name: ci/print-test-failures.sh
-      if: failure()
-      shell: bash
-      run: ci/print-test-failures.sh
+      run: . /etc/profile && make -C t -e || ci/print-test-failures-github.sh
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v2
@@ -218,11 +214,7 @@ jobs:
       shell: bash
     - name: test
       shell: bash
-      run: . /etc/profile && make -C t -e
-    - name: ci/print-test-failures.sh
-      if: failure()
-      shell: bash
-      run: ci/print-test-failures.sh
+      run: . /etc/profile && make -C t -e || ci/print-test-failures-github.sh
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v2
@@ -265,10 +257,8 @@ jobs:
     - run: ci/lib.sh --build
     - run: make
     - run: ci/lib.sh --test
-    - run: make test
+    - run: make test || ci/print-test-failures-github.sh
       if: success()
-    - run: ci/print-test-failures.sh
-      if: failure()
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v2
@@ -301,10 +291,8 @@ jobs:
     - run: ci/lib.sh --build
     - run: make
     - run: ci/lib.sh --test
-    - run: make test
+    - run: make test || ci/print-test-failures-github.sh
       if: success() && matrix.vector.skip-tests != 'no'
-    - run: ci/print-test-failures.sh
-      if: failure() && matrix.vector.skip-tests != 'no'
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v1
diff --git a/ci/print-test-failures-github.sh b/ci/print-test-failures-github.sh
new file mode 100755
index 00000000000..218aaf19763
--- /dev/null
+++ b/ci/print-test-failures-github.sh
@@ -0,0 +1,35 @@
+#!/bin/sh
+
+. ${0%/*}/lib-ci-type.sh
+
+set -e
+
+case "$CI_TYPE" in
+github-actions)
+	handle_failed_tests () {
+		mkdir -p t/failed-test-artifacts
+		echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
+
+		for test_exit in t/test-results/*.exit
+		do
+			test 0 != "$(cat "$test_exit")" || continue
+
+			test_name="${test_exit%.exit}"
+			test_name="${test_name##*/}"
+			printf "\\e[33m\\e[1m=== Failed test: ${test_name} ===\\e[m\\n"
+			cat "t/test-results/$test_name.out"
+
+			trash_dir="t/trash directory.$test_name"
+			cp "t/test-results/$test_name.out" t/failed-test-artifacts/
+			tar czf t/failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
+		done
+		return 1
+	}
+	;;
+*)
+	echo "Unhandled CI type: $CI_TYPE" >&2
+	exit 1
+	;;
+esac
+
+handle_failed_tests
-- 
2.36.0.rc2.843.g193535c2aa7

[RFC PATCH v4 2/6] tests: refactor --write-junit-xml code

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The code writing JUnit XML is interspersed directly with all the code in
`t/test-lib.sh`, and it is therefore not only ill-separated, but
introducing yet another output format would make the situation even
worse.

Let's introduce an abstraction layer by hiding the JUnit XML code behind
four new functions that are supposed to be called before and after each
test and test case.

This is not just an academic exercise, refactoring for refactoring's
sake. We _actually_ want to introduce such a new output format, to
make it substantially easier to diagnose test failures in our GitHub
workflow, therefore we do need this refactoring.

This commit is best viewed with `git show --color-moved
--color-moved-ws=allow-indentation-change <commit>`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib-junit.sh | 126 ++++++++++++++++++++++++++++++++++++++++++++
 t/test-lib.sh       | 124 ++++++-------------------------------------
 2 files changed, 142 insertions(+), 108 deletions(-)
 create mode 100644 t/test-lib-junit.sh

diff --git a/t/test-lib-junit.sh b/t/test-lib-junit.sh
new file mode 100644
index 00000000000..9d55d74d764
--- /dev/null
+++ b/t/test-lib-junit.sh
@@ -0,0 +1,126 @@
+# Library of functions to format test scripts' output in JUnit XML
+# format, to support Git's test suite result to be presented in an
+# easily digestible way on Azure Pipelines.
+#
+# Copyright (c) 2022 Johannes Schindelin
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see http://www.gnu.org/licenses/ .
+#
+# The idea is for `test-lib.sh` to source this file when the user asks
+# for JUnit XML; these functions will then override (empty) functions
+# that are are called at the appropriate times during the test runs.
+
+start_test_output () {
+	junit_xml_dir="$TEST_OUTPUT_DIRECTORY/out"
+	mkdir -p "$junit_xml_dir"
+	junit_xml_base=${1##*/}
+	junit_xml_path="$junit_xml_dir/TEST-${junit_xml_base%.sh}.xml"
+	junit_attrs="name=\"${junit_xml_base%.sh}\""
+	junit_attrs="$junit_attrs timestamp=\"$(TZ=UTC \
+		date +%Y-%m-%dT%H:%M:%S)\""
+	write_junit_xml --truncate "<testsuites>" "  <testsuite $junit_attrs>"
+	junit_suite_start=$(test-tool date getnanos)
+	if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
+	then
+		GIT_TEST_TEE_OFFSET=0
+	fi
+}
+
+start_test_case_output () {
+	junit_start=$(test-tool date getnanos)
+}
+
+finalize_test_case_output () {
+	test_case_result=$1
+	shift
+	case "$test_case_result" in
+	ok)
+		set "$*"
+		;;
+	failure)
+		junit_insert="<failure message=\"not ok $test_count -"
+		junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
+		junit_insert="$junit_insert $(xml_attr_encode \
+			"$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
+			   then
+				test-tool path-utils skip-n-bytes \
+					"$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET
+			   else
+				printf '%s\n' "$@" | sed 1d
+			   fi)")"
+		junit_insert="$junit_insert</failure>"
+		if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
+		then
+			junit_insert="$junit_insert<system-err>$(xml_attr_encode \
+				"$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")</system-err>"
+		fi
+		set "$1" "      $junit_insert"
+		;;
+	fixed)
+		set "$* (breakage fixed)"
+		;;
+	broken)
+		set "$* (known breakage)"
+		;;
+	skip)
+		message="$(xml_attr_encode "$skipped_reason")"
+		set "$1" "      <skipped message=\"$message\" />"
+		;;
+	esac
+
+	junit_attrs="name=\"$(xml_attr_encode "$this_test.$test_count $1")\""
+	shift
+	junit_attrs="$junit_attrs classname=\"$this_test\""
+	junit_attrs="$junit_attrs time=\"$(test-tool \
+		date getnanos $junit_start)\""
+	write_junit_xml "$(printf '%s\n' \
+		"    <testcase $junit_attrs>" "$@" "    </testcase>")"
+	junit_have_testcase=t
+}
+
+finalize_test_output () {
+	if test -n "$junit_xml_path"
+	then
+		test -n "$junit_have_testcase" || {
+			junit_start=$(test-tool date getnanos)
+			write_junit_xml_testcase "all tests skipped"
+		}
+
+		# adjust the overall time
+		junit_time=$(test-tool date getnanos $junit_suite_start)
+		sed -e "s/\(<testsuite.*\) time=\"[^\"]*\"/\1/" \
+			-e "s/<testsuite [^>]*/& time=\"$junit_time\"/" \
+			-e '/^ *<\/testsuite/d' \
+			<"$junit_xml_path" >"$junit_xml_path.new"
+		mv "$junit_xml_path.new" "$junit_xml_path"
+
+		write_junit_xml "  </testsuite>" "</testsuites>"
+		write_junit_xml=
+	fi
+}
+
+write_junit_xml () {
+	case "$1" in
+	--truncate)
+		>"$junit_xml_path"
+		junit_have_testcase=
+		shift
+		;;
+	esac
+	printf '%s\n' "$@" >>"$junit_xml_path"
+}
+
+xml_attr_encode () {
+	printf '%s\n' "$@" | test-tool xml-encode
+}
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 531cef097db..62a57b00730 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -137,6 +137,12 @@ mark_option_requires_arg () {
 	store_arg_to=$2
 }
 
+# These functions can be overridden e.g. to output JUnit XML
+start_test_output () { :; }
+start_test_case_output () { :; }
+finalize_test_case_output () { :; }
+finalize_test_output () { :; }
+
 parse_option () {
 	local opt="$1"
 
@@ -196,7 +202,7 @@ parse_option () {
 		tee=t
 		;;
 	--write-junit-xml)
-		write_junit_xml=t
+		. "$TEST_DIRECTORY/test-lib-junit.sh"
 		;;
 	--stress)
 		stress=t ;;
@@ -663,7 +669,7 @@ exec 6<&0
 exec 7>&2
 
 _error_exit () {
-	finalize_junit_xml
+	finalize_test_output
 	GIT_EXIT_OK=t
 	exit 1
 }
@@ -773,35 +779,13 @@ trap '{ code=$?; set +x; } 2>/dev/null; exit $code' INT TERM HUP
 # the test_expect_* functions instead.
 
 test_ok_ () {
-	if test -n "$write_junit_xml"
-	then
-		write_junit_xml_testcase "$*"
-	fi
+	finalize_test_case_output ok "$@"
 	test_success=$(($test_success + 1))
 	say_color "" "ok $test_count - $@"
 }
 
 test_failure_ () {
-	if test -n "$write_junit_xml"
-	then
-		junit_insert="<failure message=\"not ok $test_count -"
-		junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
-		junit_insert="$junit_insert $(xml_attr_encode \
-			"$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
-			   then
-				test-tool path-utils skip-n-bytes \
-					"$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET
-			   else
-				printf '%s\n' "$@" | sed 1d
-			   fi)")"
-		junit_insert="$junit_insert</failure>"
-		if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
-		then
-			junit_insert="$junit_insert<system-err>$(xml_attr_encode \
-				"$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")</system-err>"
-		fi
-		write_junit_xml_testcase "$1" "      $junit_insert"
-	fi
+	finalize_test_case_output failure "$@"
 	test_failure=$(($test_failure + 1))
 	say_color error "not ok $test_count - $1"
 	shift
@@ -814,19 +798,13 @@ test_failure_ () {
 }
 
 test_known_broken_ok_ () {
-	if test -n "$write_junit_xml"
-	then
-		write_junit_xml_testcase "$* (breakage fixed)"
-	fi
+	finalize_test_case_output fixed "$@"
 	test_fixed=$(($test_fixed+1))
 	say_color error "ok $test_count - $@ # TODO known breakage vanished"
 }
 
 test_known_broken_failure_ () {
-	if test -n "$write_junit_xml"
-	then
-		write_junit_xml_testcase "$* (known breakage)"
-	fi
+	finalize_test_case_output broken "$@"
 	test_broken=$(($test_broken+1))
 	say_color warn "not ok $test_count - $@ # TODO known breakage"
 }
@@ -1103,10 +1081,7 @@ test_start_ () {
 	test_count=$(($test_count+1))
 	maybe_setup_verbose
 	maybe_setup_valgrind
-	if test -n "$write_junit_xml"
-	then
-		junit_start=$(test-tool date getnanos)
-	fi
+	start_test_case_output
 }
 
 test_finish_ () {
@@ -1157,12 +1132,7 @@ test_skip () {
 
 	case "$to_skip" in
 	t)
-		if test -n "$write_junit_xml"
-		then
-			message="$(xml_attr_encode "$skipped_reason")"
-			write_junit_xml_testcase "$1" \
-				"      <skipped message=\"$message\" />"
-		fi
+		finalize_test_case_output skip "$@"
 
 		say_color skip "ok $test_count # skip $1 ($skipped_reason)"
 		: true
@@ -1178,53 +1148,6 @@ test_at_end_hook_ () {
 	:
 }
 
-write_junit_xml () {
-	case "$1" in
-	--truncate)
-		>"$junit_xml_path"
-		junit_have_testcase=
-		shift
-		;;
-	esac
-	printf '%s\n' "$@" >>"$junit_xml_path"
-}
-
-xml_attr_encode () {
-	printf '%s\n' "$@" | test-tool xml-encode
-}
-
-write_junit_xml_testcase () {
-	junit_attrs="name=\"$(xml_attr_encode "$this_test.$test_count $1")\""
-	shift
-	junit_attrs="$junit_attrs classname=\"$this_test\""
-	junit_attrs="$junit_attrs time=\"$(test-tool \
-		date getnanos $junit_start)\""
-	write_junit_xml "$(printf '%s\n' \
-		"    <testcase $junit_attrs>" "$@" "    </testcase>")"
-	junit_have_testcase=t
-}
-
-finalize_junit_xml () {
-	if test -n "$write_junit_xml" && test -n "$junit_xml_path"
-	then
-		test -n "$junit_have_testcase" || {
-			junit_start=$(test-tool date getnanos)
-			write_junit_xml_testcase "all tests skipped"
-		}
-
-		# adjust the overall time
-		junit_time=$(test-tool date getnanos $junit_suite_start)
-		sed -e "s/\(<testsuite.*\) time=\"[^\"]*\"/\1/" \
-			-e "s/<testsuite [^>]*/& time=\"$junit_time\"/" \
-			-e '/^ *<\/testsuite/d' \
-			<"$junit_xml_path" >"$junit_xml_path.new"
-		mv "$junit_xml_path.new" "$junit_xml_path"
-
-		write_junit_xml "  </testsuite>" "</testsuites>"
-		write_junit_xml=
-	fi
-}
-
 test_atexit_cleanup=:
 test_atexit_handler () {
 	# In a succeeding test script 'test_atexit_handler' is invoked
@@ -1247,7 +1170,7 @@ test_done () {
 	# removed, so the commands can access pidfiles and socket files.
 	test_atexit_handler
 
-	finalize_junit_xml
+	finalize_test_output
 
 	if test -z "$HARNESS_ACTIVE"
 	then
@@ -1538,22 +1461,7 @@ fi
 # in subprocesses like git equals our $PWD (for pathname comparisons).
 cd -P "$TRASH_DIRECTORY" || exit 1
 
-if test -n "$write_junit_xml"
-then
-	junit_xml_dir="$TEST_OUTPUT_DIRECTORY/out"
-	mkdir -p "$junit_xml_dir"
-	junit_xml_base=${0##*/}
-	junit_xml_path="$junit_xml_dir/TEST-${junit_xml_base%.sh}.xml"
-	junit_attrs="name=\"${junit_xml_base%.sh}\""
-	junit_attrs="$junit_attrs timestamp=\"$(TZ=UTC \
-		date +%Y-%m-%dT%H:%M:%S)\""
-	write_junit_xml --truncate "<testsuites>" "  <testsuite $junit_attrs>"
-	junit_suite_start=$(test-tool date getnanos)
-	if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
-	then
-		GIT_TEST_TEE_OFFSET=0
-	fi
-fi
+start_test_output "$0"
 
 # Convenience
 # A regexp to match 5 and 35 hexdigits
-- 
2.36.0.rc2.843.g193535c2aa7

[RFC PATCH v4 3/6] test(junit): avoid line feeds in XML attributes

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

In the test case's output, we do want newline characters, but in the XML
attributes we do not want them.

However, the `xml_attr_encode` function always adds a Line Feed at the
end (which are then encoded as `&#x0a;`, even for XML attributes.

This seems not to faze Azure Pipelines' XML parser, but it still is
incorrect, so let's fix it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib-junit.sh | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/t/test-lib-junit.sh b/t/test-lib-junit.sh
index 9d55d74d764..c959183c7e2 100644
--- a/t/test-lib-junit.sh
+++ b/t/test-lib-junit.sh
@@ -50,7 +50,7 @@ finalize_test_case_output () {
 		;;
 	failure)
 		junit_insert="<failure message=\"not ok $test_count -"
-		junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
+		junit_insert="$junit_insert $(xml_attr_encode --no-lf "$1")\">"
 		junit_insert="$junit_insert $(xml_attr_encode \
 			"$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
 			   then
@@ -74,12 +74,12 @@ finalize_test_case_output () {
 		set "$* (known breakage)"
 		;;
 	skip)
-		message="$(xml_attr_encode "$skipped_reason")"
+		message="$(xml_attr_encode --no-lf "$skipped_reason")"
 		set "$1" "      <skipped message=\"$message\" />"
 		;;
 	esac
 
-	junit_attrs="name=\"$(xml_attr_encode "$this_test.$test_count $1")\""
+	junit_attrs="name=\"$(xml_attr_encode --no-lf "$this_test.$test_count $1")\""
 	shift
 	junit_attrs="$junit_attrs classname=\"$this_test\""
 	junit_attrs="$junit_attrs time=\"$(test-tool \
@@ -122,5 +122,11 @@ write_junit_xml () {
 }
 
 xml_attr_encode () {
-	printf '%s\n' "$@" | test-tool xml-encode
+	if test "x$1" = "x--no-lf"
+	then
+		shift
+		printf '%s' "$*" | test-tool xml-encode
+	else
+		printf '%s\n' "$@" | test-tool xml-encode
+	fi
 }
-- 
2.36.0.rc2.843.g193535c2aa7

[RFC PATCH v4 4/6] ci: optionally mark up output in the GitHub workflow

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

A couple of commands exist to spruce up the output in GitHub workflows:
https://docs.github.com/en/actions/learn-github-actions/workflow-commands-for-github-actions

In addition to the `::group::<label>`/`::endgroup::` commands (which we
already use to structure the output of the build step better), we also
use `::error::`/`::notice::` to draw the attention to test failures and
to test cases that were expected to fail but didn't.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib-functions.sh              |  4 +--
 t/test-lib-github-workflow-markup.sh | 50 ++++++++++++++++++++++++++++
 t/test-lib.sh                        |  5 ++-
 3 files changed, 56 insertions(+), 3 deletions(-)
 create mode 100644 t/test-lib-github-workflow-markup.sh

diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index 93c03380d44..af4831a54c6 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -795,7 +795,7 @@ test_verify_prereq () {
 }
 
 test_expect_failure () {
-	test_start_
+	test_start_ "$@"
 	test "$#" = 3 && { test_prereq=$1; shift; } || test_prereq=
 	test "$#" = 2 ||
 	BUG "not 2 or 3 parameters to test-expect-failure"
@@ -815,7 +815,7 @@ test_expect_failure () {
 }
 
 test_expect_success () {
-	test_start_
+	test_start_ "$@"
 	test "$#" = 3 && { test_prereq=$1; shift; } || test_prereq=
 	test "$#" = 2 ||
 	BUG "not 2 or 3 parameters to test-expect-success"
diff --git a/t/test-lib-github-workflow-markup.sh b/t/test-lib-github-workflow-markup.sh
new file mode 100644
index 00000000000..d8dc969df4a
--- /dev/null
+++ b/t/test-lib-github-workflow-markup.sh
@@ -0,0 +1,50 @@
+# Library of functions to mark up test scripts' output suitable for
+# pretty-printing it in GitHub workflows.
+#
+# Copyright (c) 2022 Johannes Schindelin
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see http://www.gnu.org/licenses/ .
+#
+# The idea is for `test-lib.sh` to source this file when run in GitHub
+# workflows; these functions will then override (empty) functions
+# that are are called at the appropriate times during the test runs.
+
+start_test_output () {
+	test -n "$GIT_TEST_TEE_OUTPUT_FILE" ||
+	die "--github-workflow-markup requires --verbose-log"
+	github_markup_output="${GIT_TEST_TEE_OUTPUT_FILE%.out}.markup"
+	>$github_markup_output
+	GIT_TEST_TEE_OFFSET=0
+}
+
+# No need to override start_test_case_output
+
+finalize_test_case_output () {
+	test_case_result=$1
+	shift
+	case "$test_case_result" in
+	failure)
+		echo >>$github_markup_output "::error::failed: $this_test.$test_count $1"
+		;;
+	fixed)
+		echo >>$github_markup_output "::notice::fixed: $this_test.$test_count $1"
+		;;
+	esac
+	echo >>$github_markup_output "::group::$test_case_result: $this_test.$test_count $*"
+	test-tool >>$github_markup_output path-utils skip-n-bytes \
+		"$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET
+	echo >>$github_markup_output "::endgroup::"
+}
+
+# No need to override finalize_test_output
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 62a57b00730..8a63b4b64c9 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -204,6 +204,9 @@ parse_option () {
 	--write-junit-xml)
 		. "$TEST_DIRECTORY/test-lib-junit.sh"
 		;;
+	--github-workflow-markup)
+		. "$TEST_DIRECTORY/test-lib-github-workflow-markup.sh"
+		;;
 	--stress)
 		stress=t ;;
 	--stress=*)
@@ -1081,7 +1084,7 @@ test_start_ () {
 	test_count=$(($test_count+1))
 	maybe_setup_verbose
 	maybe_setup_valgrind
-	start_test_case_output
+	start_test_case_output "$@"
 }
 
 test_finish_ () {
-- 
2.36.0.rc2.843.g193535c2aa7

[RFC PATCH v4 5/6] ci: use `--github-workflow-markup` in the GitHub workflow

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

[Ævar: due to re-structuring on top of my series the {begin,end}_group
in CI isn't needed at all to get "group" output for the test
suite. This commit includes the now-squashed "ci/run-build-and-tests:
add some structure to the GitHub workflow output":]

The current output of Git's GitHub workflow can be quite confusing,
especially for contributors new to the project.

To make it more helpful, let's introduce some collapsible grouping.
Initially, readers will see the high-level view of what actually
happened (did the build fail, or the test suite?). To drill down, the
respective group can be expanded.

Note: sadly, workflow output currently cannot contain any nested groups
(see https://github.com/actions/runner/issues/802 for details),
therefore we take pains to ensure to end any previous group before
starting a new one.

[Ævar: The above comment isn't true anymore, as that limitation has
been removed by basing this on my patches to run "make" and "make
test" directly from the top-level of main.yml.

Those are now effectively their own "group", effectively giving this
stage another group "level" to use. This means that the equivalent of
"make test" won't be on the same level as an individual test failure.

We no longer take any pains to ensure balanced group output as a
result (which was a caveat the previous ci/lib.sh implementation had
to deal with., We just need to "cat" the generated *.markup]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                        | 3 ++-
 ci/print-test-failures-github.sh | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 3fd5291bc80..718c3252789 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -145,7 +145,8 @@ MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
 case "$CI_TYPE" in
 github-actions)
 	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
-	GIT_TEST_OPTS="--verbose-log -x"
+	GIT_TEST_OPTS="--verbose-log -x --github-workflow-markup"
+	MAKEFLAGS="$MAKEFLAGS --jobs=10"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
diff --git a/ci/print-test-failures-github.sh b/ci/print-test-failures-github.sh
index 218aaf19763..e432a601621 100755
--- a/ci/print-test-failures-github.sh
+++ b/ci/print-test-failures-github.sh
@@ -17,7 +17,7 @@ github-actions)
 			test_name="${test_exit%.exit}"
 			test_name="${test_name##*/}"
 			printf "\\e[33m\\e[1m=== Failed test: ${test_name} ===\\e[m\\n"
-			cat "t/test-results/$test_name.out"
+			cat "t/test-results/$test_name.markup"
 
 			trash_dir="t/trash directory.$test_name"
 			cp "t/test-results/$test_name.out" t/failed-test-artifacts/
-- 
2.36.0.rc2.843.g193535c2aa7

[RFC PATCH v4 6/6] ci: call `finalize_test_case_output` a little later

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

We used to call that function already before printing the final verdict.
However, now that we added grouping to the GitHub workflow output, we
will want to include even that part in the collapsible group for that
test case.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib.sh | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/t/test-lib.sh b/t/test-lib.sh
index 8a63b4b64c9..1ceba46215f 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -782,13 +782,13 @@ trap '{ code=$?; set +x; } 2>/dev/null; exit $code' INT TERM HUP
 # the test_expect_* functions instead.
 
 test_ok_ () {
-	finalize_test_case_output ok "$@"
 	test_success=$(($test_success + 1))
 	say_color "" "ok $test_count - $@"
+	finalize_test_case_output ok "$@"
 }
 
 test_failure_ () {
-	finalize_test_case_output failure "$@"
+	failure_label=$1
 	test_failure=$(($test_failure + 1))
 	say_color error "not ok $test_count - $1"
 	shift
@@ -798,18 +798,19 @@ test_failure_ () {
 		say_color error "1..$test_count"
 		_error_exit
 	fi
+	finalize_test_case_output failure "$failure_label" "$@"
 }
 
 test_known_broken_ok_ () {
-	finalize_test_case_output fixed "$@"
 	test_fixed=$(($test_fixed+1))
 	say_color error "ok $test_count - $@ # TODO known breakage vanished"
+	finalize_test_case_output fixed "$@"
 }
 
 test_known_broken_failure_ () {
-	finalize_test_case_output broken "$@"
 	test_broken=$(($test_broken+1))
 	say_color warn "not ok $test_count - $@ # TODO known breakage"
+	finalize_test_case_output broken "$@"
 }
 
 test_debug () {
@@ -1135,10 +1136,10 @@ test_skip () {
 
 	case "$to_skip" in
 	t)
-		finalize_test_case_output skip "$@"
 
 		say_color skip "ok $test_count # skip $1 ($skipped_reason)"
 		: true
+		finalize_test_case_output skip "$@"
 		;;
 	*)
 		false
-- 
2.36.0.rc2.843.g193535c2aa7

Re: [RFC PATCH v4 0/6] CI: js/ci-github-workflow-markup rebased on "use $GITHUB_ENV"

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> 6:  1d2b94436fc ! 6:  b291f64821c ci: call `finalize_test_case_output` a little later
>     @@ t/test-lib.sh: trap '{ code=$?; set +x; } 2>/dev/null; exit $code' INT TERM HUP
>       	test_failure=$(($test_failure + 1))
>       	say_color error "not ok $test_count - $1"
>       	shift
>     - 	printf '%s\n' "$*" | sed -e 's/^/#	/'
>     - 	test "$immediate" = "" || _error_exit
>     +@@ t/test-lib.sh: test_failure_ () {
>     + 		say_color error "1..$test_count"
>     + 		_error_exit
>     + 	fi
>      +	finalize_test_case_output failure "$failure_label" "$@"
>       }

This part, specifically the following from [6/6], looks iffy.  

@@ -782,13 +782,13 @@ trap '{ code=$?; set +x; } 2>/dev/null; exit $code' INT TERM HUP
 # the test_expect_* functions instead.
 
 test_ok_ () {
-	finalize_test_case_output ok "$@"
 	test_success=$(($test_success + 1))
 	say_color "" "ok $test_count - $@"
+	finalize_test_case_output ok "$@"
 }
 
 test_failure_ () {
-	finalize_test_case_output failure "$@"
+	failure_label=$1
 	test_failure=$(($test_failure + 1))
 	say_color error "not ok $test_count - $1"
 	shift
@@ -798,18 +798,19 @@ test_failure_ () {
 		say_color error "1..$test_count"
 		_error_exit
 	fi
+	finalize_test_case_output failure "$failure_label" "$@"
 }
 


With the other 29-patch series applied on the same base as before,
test_failure_ does not have such "fi" inside.  Misapplication of
rebase or something?

In any case, I've wiggled both series in and rebuilt 'seen'.
Looking good as before.

Thanks.

Re: [RFC PATCH v4 0/6] CI: js/ci-github-workflow-markup rebased on "use $GITHUB_ENV"

[Ævar Arnfjörð Bjarmason]

On Thu, Apr 14 2022, Junio C Hamano wrote:

> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>
>> 6:  1d2b94436fc ! 6:  b291f64821c ci: call `finalize_test_case_output` a little later
>>     @@ t/test-lib.sh: trap '{ code=$?; set +x; } 2>/dev/null; exit $code' INT TERM HUP
>>       	test_failure=$(($test_failure + 1))
>>       	say_color error "not ok $test_count - $1"
>>       	shift
>>     - 	printf '%s\n' "$*" | sed -e 's/^/#	/'
>>     - 	test "$immediate" = "" || _error_exit
>>     +@@ t/test-lib.sh: test_failure_ () {
>>     + 		say_color error "1..$test_count"
>>     + 		_error_exit
>>     + 	fi
>>      +	finalize_test_case_output failure "$failure_label" "$@"
>>       }
>
> This part, specifically the following from [6/6], looks iffy.  
>
> @@ -782,13 +782,13 @@ trap '{ code=$?; set +x; } 2>/dev/null; exit $code' INT TERM HUP
>  # the test_expect_* functions instead.
>  
>  test_ok_ () {
> -	finalize_test_case_output ok "$@"
>  	test_success=$(($test_success + 1))
>  	say_color "" "ok $test_count - $@"
> +	finalize_test_case_output ok "$@"
>  }
>  
>  test_failure_ () {
> -	finalize_test_case_output failure "$@"
> +	failure_label=$1
>  	test_failure=$(($test_failure + 1))
>  	say_color error "not ok $test_count - $1"
>  	shift
> @@ -798,18 +798,19 @@ test_failure_ () {
>  		say_color error "1..$test_count"
>  		_error_exit
>  	fi
> +	finalize_test_case_output failure "$failure_label" "$@"
>  }
>  
>
>
> With the other 29-patch series applied on the same base as before,
> test_failure_ does not have such "fi" inside.  Misapplication of
> rebase or something?

This re-submission was rebased on "master", so that "fi" in the context
is from the now-landed ab/test-tap-fix-for-immediate.

I saw you'd fixed that conflict already, but figured rebasing before
submission (as usual) would be helpful anyway, sorry about the
confusion.

> In any case, I've wiggled both series in and rebuilt 'seen'.
> Looking good as before.

Thanks, the end-state of the resolution looks good, and matches what I
have locally.

Re: [RFC PATCH v4 0/6] CI: js/ci-github-workflow-markup rebased on "use $GITHUB_ENV"

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

>> With the other 29-patch series applied on the same base as before,
>> test_failure_ does not have such "fi" inside.  Misapplication of
>> rebase or something?
>
> This re-submission was rebased on "master", so that "fi" in the context
> is from the now-landed ab/test-tap-fix-for-immediate.
>
> I saw you'd fixed that conflict already, but figured rebasing before
> submission (as usual) would be helpful anyway, sorry about the
> confusion.
>
>> In any case, I've wiggled both series in and rebuilt 'seen'.
>> Looking good as before.
>
> Thanks, the end-state of the resolution looks good, and matches what I
> have locally.

OK, I guessed that much.

It may be OK for a topic to be rebased to an updated 'master' when
it is not close to be merged to 'next', but I prefer to see a reroll
to keep the same base, unless the new round starts depending on the
new base in a way more than just textually (i.e. e.g. wants to use a
new API function), as that makes it easier to read the comparison
with the previous round because a reasonably looking range-diff may
not necessarily mean that a patch that hasn't changed much from the
previous round would fit well inside the updated context.  This is
especially true for a series on the large-ish side, as I cannot
trust range-diff for an rebased series as much as I can for an
update on the same base.

This time, I tried to queue the new ones on updated 'master' and
then compared it with the result of merging the ones that I wriggled
to apply to the same base to the same updated 'master' to make sure
the end result is the same while I was working on making them apply
before I sent the message you are responding to, so I am reasonably
happy with the result.

It may not be a bad idea, as we are in pre-release freeze anyway,
for me to discard the two topics and replace them with a fresh
application of the same version on top of 'master', as we have
already verified that the updates from the previous iteration look
reasonable, to prepare for post-release (either merging to 'next' or
taking another reroll).

Thanks.

[RFC PATCH v5 00/10] CI: js/ci-github-workflow-markup rebased on "use $GITHUB_ENV"

[Ævar Arnfjörð Bjarmason]

See the "v3" for the "real" cover-letter:
https://lore.kernel.org/git/RFC-cover-v3-0.6-00000000000-20220325T183946Z-avarab@gmail.com/

I submitted a re-roll of the parent series at
https://lore.kernel.org/git/cover-v5-00.29-00000000000-20220421T181526Z-avarab@gmail.com
which required a conflict resolution for this one.

This version also has outstanding changes I had for using
ci/print-test-failures.sh with a parameter instead of the previous
copy/pasted and adjusted ci/print-test-failures-github.sh:
https://lore.kernel.org/git/RFC-patch-v4-1.6-cc137c69ee1-20220413T195514Z-avarab@gmail.com/

There's some discussion about these two serieses in general, and
Johannes intends to re-roll this:
https://lore.kernel.org/git/nycvar.QRO.7.76.6.2204211454500.355@tvgsbejvaqbjf.bet/

So whatever Junio ends up (re-)picking up (and I'll reply in some of
those threads) hopefully having this version for comparison helps.

Johannes Schindelin (6):
  ci: make it easier to find failed tests' logs in the GitHub workflow
  tests: refactor --write-junit-xml code
  test(junit): avoid line feeds in XML attributes
  ci: optionally mark up output in the GitHub workflow
  ci: use `--github-workflow-markup` in the GitHub workflow
  ci: call `finalize_test_case_output` a little later

Ævar Arnfjörð Bjarmason (4):
  CI: don't "cd" in ci/print-test-failures.sh
  CI: add --exit-code to ci/print-test-failures.sh
  CI: don't include "test-results/" in ci/print-test-failures.sh output
  CI: stop setting FAILED_TEST_ARTIFACTS N times

 .github/workflows/main.yml           |  20 +---
 ci/lib.sh                            |   2 +-
 ci/print-test-failures.sh            |  89 ++++++++++++++----
 t/test-lib-functions.sh              |   4 +-
 t/test-lib-github-workflow-markup.sh |  50 ++++++++++
 t/test-lib-junit.sh                  | 132 +++++++++++++++++++++++++++
 t/test-lib.sh                        | 128 ++++----------------------
 7 files changed, 282 insertions(+), 143 deletions(-)
 create mode 100644 t/test-lib-github-workflow-markup.sh
 create mode 100644 t/test-lib-junit.sh

Range-diff against v4:
 -:  ----------- >  1:  56910df26e7 CI: don't "cd" in ci/print-test-failures.sh
 -:  ----------- >  2:  caec0b1089a CI: add --exit-code to ci/print-test-failures.sh
 -:  ----------- >  3:  577a5be34f4 CI: don't include "test-results/" in ci/print-test-failures.sh output
 -:  ----------- >  4:  1482f840f64 CI: stop setting FAILED_TEST_ARTIFACTS N times
 1:  cc137c69ee1 !  5:  4291892fbd3 ci: make it easier to find failed tests' logs in the GitHub workflow
    @@ .github/workflows/main.yml: jobs:
     -      if: failure()
     -      shell: bash
     -      run: ci/print-test-failures.sh
    -+      run: . /etc/profile && make -C t -e || ci/print-test-failures-github.sh
    ++      run: . /etc/profile && make -C t -e || ci/print-test-failures.sh
          - name: Upload failed tests' directories
            if: failure() && env.FAILED_TEST_ARTIFACTS != ''
            uses: actions/upload-artifact@v2
    @@ .github/workflows/main.yml: jobs:
            if: failure() && env.FAILED_TEST_ARTIFACTS != ''
            uses: actions/upload-artifact@v2
     @@ .github/workflows/main.yml: jobs:
    -     - run: ci/lib.sh --build
          - run: make
          - run: ci/lib.sh --test
    +       if: success() && matrix.vector.skip-tests != 'yes'
     -    - run: make test
     +    - run: make test || ci/print-test-failures-github.sh
    -       if: success() && matrix.vector.skip-tests != 'no'
    +       if: success() && matrix.vector.skip-tests != 'yes'
     -    - run: ci/print-test-failures.sh
    --      if: failure() && matrix.vector.skip-tests != 'no'
    +-      if: failure() && matrix.vector.skip-tests != 'yes'
          - name: Upload failed tests' directories
            if: failure() && env.FAILED_TEST_ARTIFACTS != ''
            uses: actions/upload-artifact@v1
     
    - ## ci/print-test-failures-github.sh (new) ##
    -@@
    -+#!/bin/sh
    -+
    -+. ${0%/*}/lib-ci-type.sh
    -+
    -+set -e
    -+
    + ## ci/print-test-failures.sh ##
    +@@ ci/print-test-failures.sh: set -e
    + . ${0%/*}/lib-ci-type.sh
    + . ${0%/*}/lib-tput.sh
    + 
    +-exit_code=
     +case "$CI_TYPE" in
     +github-actions)
    -+	handle_failed_tests () {
    -+		mkdir -p t/failed-test-artifacts
    -+		echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
    -+
    -+		for test_exit in t/test-results/*.exit
    -+		do
    -+			test 0 != "$(cat "$test_exit")" || continue
    -+
    -+			test_name="${test_exit%.exit}"
    -+			test_name="${test_name##*/}"
    -+			printf "\\e[33m\\e[1m=== Failed test: ${test_name} ===\\e[m\\n"
    -+			cat "t/test-results/$test_name.out"
    -+
    -+			trash_dir="t/trash directory.$test_name"
    -+			cp "t/test-results/$test_name.out" t/failed-test-artifacts/
    -+			tar czf t/failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
    -+		done
    -+		return 1
    -+	}
    ++	exit_code=t
    ++	github_workflow_markup=t
     +	;;
     +*)
    -+	echo "Unhandled CI type: $CI_TYPE" >&2
    -+	exit 1
    ++	exit_code=
    ++	github_workflow_markup=
     +	;;
     +esac
     +
    -+handle_failed_tests
    + while test $# != 0
    + do
    + 	case "$1" in
    + 	--exit-code)
    + 		exit_code=t
    + 		;;
    ++	--no-exit-code)
    ++		exit_code=
    ++		;;
    ++	--github-workflow-markup)
    ++		github_workflow_markup=t
    ++		;;
    ++	--no-github-workflow-markup)
    ++		github_workflow_markup=
    ++		;;
    + 	*)
    + 		echo "BUG: invalid $0 argument: $1" >&2
    + 		exit 1
    +@@ ci/print-test-failures.sh: do
    + 		TEST_OUT="${TEST_NAME}.out"
    + 		TEST_MARKUP="${TEST_NAME}.markup"
    + 
    +-		echo "------------------------------------------------------------------------"
    +-		echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
    +-		echo "------------------------------------------------------------------------"
    ++		if test -n "$github_workflow_markup"
    ++		then
    ++			printf "\\e[33m\\e[1m=== Failed test: ${TEST_NAME} ===\\e[m\\n"
    ++		else
    ++			echo "------------------------------------------------------------------------"
    ++			echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
    ++			echo "------------------------------------------------------------------------"
    ++		fi
    + 		cat "t/test-results/${TEST_OUT}"
    + 
    + 		trash_dir="trash directory.$TEST_NAME"
 2:  91f96c4f210 =  6:  d9b7d5ddf5a tests: refactor --write-junit-xml code
 3:  84c722969d5 =  7:  2d233bbdc22 test(junit): avoid line feeds in XML attributes
 4:  8acaa800d3a =  8:  5dd43a1624b ci: optionally mark up output in the GitHub workflow
 5:  4499f743dd1 !  9:  877edff3b4d ci: use `--github-workflow-markup` in the GitHub workflow
    @@ ci/lib.sh: MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
      	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
     -	GIT_TEST_OPTS="--verbose-log -x"
     +	GIT_TEST_OPTS="--verbose-log -x --github-workflow-markup"
    -+	MAKEFLAGS="$MAKEFLAGS --jobs=10"
      	test Windows != "$RUNNER_OS" ||
      	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
      	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
     
    - ## ci/print-test-failures-github.sh ##
    -@@ ci/print-test-failures-github.sh: github-actions)
    - 			test_name="${test_exit%.exit}"
    - 			test_name="${test_name##*/}"
    - 			printf "\\e[33m\\e[1m=== Failed test: ${test_name} ===\\e[m\\n"
    --			cat "t/test-results/$test_name.out"
    -+			cat "t/test-results/$test_name.markup"
    + ## ci/print-test-failures.sh ##
    +@@ ci/print-test-failures.sh: do
    + 		if test -n "$github_workflow_markup"
    + 		then
    + 			printf "\\e[33m\\e[1m=== Failed test: ${TEST_NAME} ===\\e[m\\n"
    ++			cat "t/test-results/$TEST_MARKUP"
    + 		else
    + 			echo "------------------------------------------------------------------------"
    + 			echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
    + 			echo "------------------------------------------------------------------------"
    ++			cat "t/test-results/${TEST_OUT}"
    + 		fi
    +-		cat "t/test-results/${TEST_OUT}"
      
    - 			trash_dir="t/trash directory.$test_name"
    - 			cp "t/test-results/$test_name.out" t/failed-test-artifacts/
    + 		trash_dir="trash directory.$TEST_NAME"
    + 		case "$CI_TYPE" in
 6:  b291f64821c = 10:  6b278194f9a ci: call `finalize_test_case_output` a little later
-- 
2.36.0.879.g3659959fcca

[RFC PATCH v5 01/10] CI: don't "cd" in ci/print-test-failures.sh

[Ævar Arnfjörð Bjarmason]

Change the logic in ci/print-test-failures.sh so that we don't need to
"cd" at the top-level, we'll now only do so for the "tar" command.

This way we can remove some duplicate code added in
aea8879a6ac (travis-ci: include the trash directories of failed tests
in the trace log, 2018-08-01), i.e. not have both a $TEST_NAME and
$test_name. The output is exactly the same as before,
i.e. "test-results/" was included in the heading.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/print-test-failures.sh | 29 ++++++++++++++++-------------
 1 file changed, 16 insertions(+), 13 deletions(-)

diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index ec395c79ccd..f6736f424e0 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -8,33 +8,36 @@ set -e
 . ${0%/*}/lib-ci-type.sh
 . ${0%/*}/lib-tput.sh
 
-cd t/
-
-if ! ls test-results/*.exit >/dev/null 2>/dev/null
+if ! ls t/test-results/*.exit >/dev/null 2>/dev/null
 then
 	echo "Build job failed before the tests could have been run"
 	exit
 fi
 
-for TEST_EXIT in test-results/*.exit
+for TEST_EXIT in t/test-results/*.exit
 do
 	if [ "$(cat "$TEST_EXIT")" != "0" ]
 	then
-		TEST_OUT="${TEST_EXIT%exit}out"
+		TEST_NAME="${TEST_EXIT%.exit}"
+		TEST_NAME="${TEST_NAME##*/}"
+		TEST_OUT="${TEST_NAME}.out"
+		TEST_MARKUP="${TEST_NAME}.markup"
+
 		echo "------------------------------------------------------------------------"
-		echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
+		echo "$(tput setaf 1)test-results/${TEST_OUT}...$(tput sgr0)"
 		echo "------------------------------------------------------------------------"
-		cat "${TEST_OUT}"
+		cat "t/test-results/${TEST_OUT}"
 
-		test_name="${TEST_EXIT%.exit}"
-		test_name="${test_name##*/}"
-		trash_dir="trash directory.$test_name"
+		trash_dir="trash directory.$TEST_NAME"
 		case "$CI_TYPE" in
 		github-actions)
-			mkdir -p failed-test-artifacts
+			mkdir -p t/failed-test-artifacts
 			echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
-			cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
-			tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
+			cp "t/test-results/${TEST_OUT}" t/failed-test-artifacts/
+			(
+				cd t &&
+				tar czf failed-test-artifacts/"$TEST_NAME".trash.tar.gz "$trash_dir"
+			)
 			;;
 		esac
 	fi
-- 
2.36.0.879.g3659959fcca

[RFC PATCH v5 02/10] CI: add --exit-code to ci/print-test-failures.sh

[Ævar Arnfjörð Bjarmason]

The ci/print-test-failures.sh scripts will exit 0 when there's failed
tests, let's teach it --exit-code to have it exit non-zero in those
cases.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/print-test-failures.sh | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index f6736f424e0..d00cd0e4944 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -8,16 +8,33 @@ set -e
 . ${0%/*}/lib-ci-type.sh
 . ${0%/*}/lib-tput.sh
 
+exit_code=
+while test $# != 0
+do
+	case "$1" in
+	--exit-code)
+		exit_code=t
+		;;
+	*)
+		echo "BUG: invalid $0 argument: $1" >&2
+		exit 1
+		;;
+	esac
+	shift
+done
+
 if ! ls t/test-results/*.exit >/dev/null 2>/dev/null
 then
 	echo "Build job failed before the tests could have been run"
 	exit
 fi
 
+failed=
 for TEST_EXIT in t/test-results/*.exit
 do
 	if [ "$(cat "$TEST_EXIT")" != "0" ]
 	then
+		failed=t
 		TEST_NAME="${TEST_EXIT%.exit}"
 		TEST_NAME="${TEST_NAME##*/}"
 		TEST_OUT="${TEST_NAME}.out"
@@ -42,3 +59,11 @@ do
 		esac
 	fi
 done
+
+if test -n "$failed"
+then
+	if test -n "$exit_code"
+	then
+		exit 1
+	fi
+fi
-- 
2.36.0.879.g3659959fcca

[RFC PATCH v5 03/10] CI: don't include "test-results/" in ci/print-test-failures.sh output

[Ævar Arnfjörð Bjarmason]

Change the ci/print-test-failures.sh output introduced in
657343a602e (travis-ci: move Travis CI code into dedicated scripts,
2017-09-10) to stop including "test-results/" in the heading, it was
more of an artifact of how the iteration was being done in that
initial version than anything else.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/print-test-failures.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index d00cd0e4944..23324021f13 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -41,7 +41,7 @@ do
 		TEST_MARKUP="${TEST_NAME}.markup"
 
 		echo "------------------------------------------------------------------------"
-		echo "$(tput setaf 1)test-results/${TEST_OUT}...$(tput sgr0)"
+		echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
 		echo "------------------------------------------------------------------------"
 		cat "t/test-results/${TEST_OUT}"
 
-- 
2.36.0.879.g3659959fcca

[RFC PATCH v5 04/10] CI: stop setting FAILED_TEST_ARTIFACTS N times

[Ævar Arnfjörð Bjarmason]

The FAILED_TEST_ARTIFACTS variable introduced in f72f328bc57 (ci: let
GitHub Actions upload failed tests' directories, 2020-04-11) is only
used once in the main.yml, there is no need to set it N times if we
have N failing tests.

See also 92bf1b60673 (ci: avoid `set-env` construct in
print-test-failures.sh, 2020-11-17) which moved it to using
$GITHUB_ENV, we'll now check if it's set like ci/lib.sh, this makes it
easier to test this script locally.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/print-test-failures.sh | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 23324021f13..27df5081f8b 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -49,7 +49,6 @@ do
 		case "$CI_TYPE" in
 		github-actions)
 			mkdir -p t/failed-test-artifacts
-			echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
 			cp "t/test-results/${TEST_OUT}" t/failed-test-artifacts/
 			(
 				cd t &&
@@ -62,6 +61,11 @@ done
 
 if test -n "$failed"
 then
+	if test -n "$GITHUB_ENV"
+	then
+		echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
+	fi
+
 	if test -n "$exit_code"
 	then
 		exit 1
-- 
2.36.0.879.g3659959fcca

[RFC PATCH v5 05/10] ci: make it easier to find failed tests' logs in the GitHub workflow

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

When investigating a test failure, the time that matters most is the
time it takes from getting aware of the failure to displaying the output
of the failing test case.

You currently have to know a lot of implementation details when
investigating test failures in the CI runs. The first step is easy: the
failed job is marked quite clearly, but when opening it, the failed step
is expanded, which in our case is the one running
`ci/run-build-and-tests.sh`. This step, most notably, only offers a
high-level view of what went wrong: it prints the output of `prove`
which merely tells the reader which test script failed.

The actually interesting part is in the detailed log of said failed
test script. But that log is shown in the CI run's step that runs
`ci/print-test-failures.sh`. And that step is _not_ expanded in the web
UI by default. It is even marked as "successful", which makes it very
easy to miss that there is useful information hidden in there.

Let's help the reader by showing the failed tests' detailed logs in the
step that is expanded automatically, i.e. directly after the test suite
failed.

This also helps the situation where the _build_ failed and the
`print-test-failures` step was executed under the assumption that the
_test suite_ failed, and consequently failed to find any failed tests.

An alternative way to implement this patch would be to source
`ci/print-test-failures.sh` in the `handle_test_failures` function to
show these logs. However, over the course of the next few commits, we
want to introduce some grouping which would be harder to achieve that
way (for example, we do want a leaner, and colored, preamble for each
failed test script, and it would be trickier to accommodate the lack of
nested groupings in GitHub workflows' output).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 20 ++++----------------
 ci/print-test-failures.sh  | 32 ++++++++++++++++++++++++++++----
 2 files changed, 32 insertions(+), 20 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 6835e942280..aa31c78d4c2 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -129,11 +129,7 @@ jobs:
       shell: bash
     - name: test
       shell: bash
-      run: . /etc/profile && make -C t -e
-    - name: ci/print-test-failures.sh
-      if: failure()
-      shell: bash
-      run: ci/print-test-failures.sh
+      run: . /etc/profile && make -C t -e || ci/print-test-failures.sh
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v2
@@ -218,11 +214,7 @@ jobs:
       shell: bash
     - name: test
       shell: bash
-      run: . /etc/profile && make -C t -e
-    - name: ci/print-test-failures.sh
-      if: failure()
-      shell: bash
-      run: ci/print-test-failures.sh
+      run: . /etc/profile && make -C t -e || ci/print-test-failures-github.sh
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v2
@@ -265,10 +257,8 @@ jobs:
     - run: ci/lib.sh --build
     - run: make
     - run: ci/lib.sh --test
-    - run: make test
+    - run: make test || ci/print-test-failures-github.sh
       if: success()
-    - run: ci/print-test-failures.sh
-      if: failure()
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v2
@@ -302,10 +292,8 @@ jobs:
     - run: make
     - run: ci/lib.sh --test
       if: success() && matrix.vector.skip-tests != 'yes'
-    - run: make test
+    - run: make test || ci/print-test-failures-github.sh
       if: success() && matrix.vector.skip-tests != 'yes'
-    - run: ci/print-test-failures.sh
-      if: failure() && matrix.vector.skip-tests != 'yes'
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v1
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 27df5081f8b..814c7799b68 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -8,13 +8,32 @@ set -e
 . ${0%/*}/lib-ci-type.sh
 . ${0%/*}/lib-tput.sh
 
-exit_code=
+case "$CI_TYPE" in
+github-actions)
+	exit_code=t
+	github_workflow_markup=t
+	;;
+*)
+	exit_code=
+	github_workflow_markup=
+	;;
+esac
+
 while test $# != 0
 do
 	case "$1" in
 	--exit-code)
 		exit_code=t
 		;;
+	--no-exit-code)
+		exit_code=
+		;;
+	--github-workflow-markup)
+		github_workflow_markup=t
+		;;
+	--no-github-workflow-markup)
+		github_workflow_markup=
+		;;
 	*)
 		echo "BUG: invalid $0 argument: $1" >&2
 		exit 1
@@ -40,9 +59,14 @@ do
 		TEST_OUT="${TEST_NAME}.out"
 		TEST_MARKUP="${TEST_NAME}.markup"
 
-		echo "------------------------------------------------------------------------"
-		echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
-		echo "------------------------------------------------------------------------"
+		if test -n "$github_workflow_markup"
+		then
+			printf "\\e[33m\\e[1m=== Failed test: ${TEST_NAME} ===\\e[m\\n"
+		else
+			echo "------------------------------------------------------------------------"
+			echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
+			echo "------------------------------------------------------------------------"
+		fi
 		cat "t/test-results/${TEST_OUT}"
 
 		trash_dir="trash directory.$TEST_NAME"
-- 
2.36.0.879.g3659959fcca

[RFC PATCH v5 06/10] tests: refactor --write-junit-xml code

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The code writing JUnit XML is interspersed directly with all the code in
`t/test-lib.sh`, and it is therefore not only ill-separated, but
introducing yet another output format would make the situation even
worse.

Let's introduce an abstraction layer by hiding the JUnit XML code behind
four new functions that are supposed to be called before and after each
test and test case.

This is not just an academic exercise, refactoring for refactoring's
sake. We _actually_ want to introduce such a new output format, to
make it substantially easier to diagnose test failures in our GitHub
workflow, therefore we do need this refactoring.

This commit is best viewed with `git show --color-moved
--color-moved-ws=allow-indentation-change <commit>`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib-junit.sh | 126 ++++++++++++++++++++++++++++++++++++++++++++
 t/test-lib.sh       | 124 ++++++-------------------------------------
 2 files changed, 142 insertions(+), 108 deletions(-)
 create mode 100644 t/test-lib-junit.sh

diff --git a/t/test-lib-junit.sh b/t/test-lib-junit.sh
new file mode 100644
index 00000000000..9d55d74d764
--- /dev/null
+++ b/t/test-lib-junit.sh
@@ -0,0 +1,126 @@
+# Library of functions to format test scripts' output in JUnit XML
+# format, to support Git's test suite result to be presented in an
+# easily digestible way on Azure Pipelines.
+#
+# Copyright (c) 2022 Johannes Schindelin
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see http://www.gnu.org/licenses/ .
+#
+# The idea is for `test-lib.sh` to source this file when the user asks
+# for JUnit XML; these functions will then override (empty) functions
+# that are are called at the appropriate times during the test runs.
+
+start_test_output () {
+	junit_xml_dir="$TEST_OUTPUT_DIRECTORY/out"
+	mkdir -p "$junit_xml_dir"
+	junit_xml_base=${1##*/}
+	junit_xml_path="$junit_xml_dir/TEST-${junit_xml_base%.sh}.xml"
+	junit_attrs="name=\"${junit_xml_base%.sh}\""
+	junit_attrs="$junit_attrs timestamp=\"$(TZ=UTC \
+		date +%Y-%m-%dT%H:%M:%S)\""
+	write_junit_xml --truncate "<testsuites>" "  <testsuite $junit_attrs>"
+	junit_suite_start=$(test-tool date getnanos)
+	if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
+	then
+		GIT_TEST_TEE_OFFSET=0
+	fi
+}
+
+start_test_case_output () {
+	junit_start=$(test-tool date getnanos)
+}
+
+finalize_test_case_output () {
+	test_case_result=$1
+	shift
+	case "$test_case_result" in
+	ok)
+		set "$*"
+		;;
+	failure)
+		junit_insert="<failure message=\"not ok $test_count -"
+		junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
+		junit_insert="$junit_insert $(xml_attr_encode \
+			"$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
+			   then
+				test-tool path-utils skip-n-bytes \
+					"$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET
+			   else
+				printf '%s\n' "$@" | sed 1d
+			   fi)")"
+		junit_insert="$junit_insert</failure>"
+		if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
+		then
+			junit_insert="$junit_insert<system-err>$(xml_attr_encode \
+				"$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")</system-err>"
+		fi
+		set "$1" "      $junit_insert"
+		;;
+	fixed)
+		set "$* (breakage fixed)"
+		;;
+	broken)
+		set "$* (known breakage)"
+		;;
+	skip)
+		message="$(xml_attr_encode "$skipped_reason")"
+		set "$1" "      <skipped message=\"$message\" />"
+		;;
+	esac
+
+	junit_attrs="name=\"$(xml_attr_encode "$this_test.$test_count $1")\""
+	shift
+	junit_attrs="$junit_attrs classname=\"$this_test\""
+	junit_attrs="$junit_attrs time=\"$(test-tool \
+		date getnanos $junit_start)\""
+	write_junit_xml "$(printf '%s\n' \
+		"    <testcase $junit_attrs>" "$@" "    </testcase>")"
+	junit_have_testcase=t
+}
+
+finalize_test_output () {
+	if test -n "$junit_xml_path"
+	then
+		test -n "$junit_have_testcase" || {
+			junit_start=$(test-tool date getnanos)
+			write_junit_xml_testcase "all tests skipped"
+		}
+
+		# adjust the overall time
+		junit_time=$(test-tool date getnanos $junit_suite_start)
+		sed -e "s/\(<testsuite.*\) time=\"[^\"]*\"/\1/" \
+			-e "s/<testsuite [^>]*/& time=\"$junit_time\"/" \
+			-e '/^ *<\/testsuite/d' \
+			<"$junit_xml_path" >"$junit_xml_path.new"
+		mv "$junit_xml_path.new" "$junit_xml_path"
+
+		write_junit_xml "  </testsuite>" "</testsuites>"
+		write_junit_xml=
+	fi
+}
+
+write_junit_xml () {
+	case "$1" in
+	--truncate)
+		>"$junit_xml_path"
+		junit_have_testcase=
+		shift
+		;;
+	esac
+	printf '%s\n' "$@" >>"$junit_xml_path"
+}
+
+xml_attr_encode () {
+	printf '%s\n' "$@" | test-tool xml-encode
+}
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 531cef097db..62a57b00730 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -137,6 +137,12 @@ mark_option_requires_arg () {
 	store_arg_to=$2
 }
 
+# These functions can be overridden e.g. to output JUnit XML
+start_test_output () { :; }
+start_test_case_output () { :; }
+finalize_test_case_output () { :; }
+finalize_test_output () { :; }
+
 parse_option () {
 	local opt="$1"
 
@@ -196,7 +202,7 @@ parse_option () {
 		tee=t
 		;;
 	--write-junit-xml)
-		write_junit_xml=t
+		. "$TEST_DIRECTORY/test-lib-junit.sh"
 		;;
 	--stress)
 		stress=t ;;
@@ -663,7 +669,7 @@ exec 6<&0
 exec 7>&2
 
 _error_exit () {
-	finalize_junit_xml
+	finalize_test_output
 	GIT_EXIT_OK=t
 	exit 1
 }
@@ -773,35 +779,13 @@ trap '{ code=$?; set +x; } 2>/dev/null; exit $code' INT TERM HUP
 # the test_expect_* functions instead.
 
 test_ok_ () {
-	if test -n "$write_junit_xml"
-	then
-		write_junit_xml_testcase "$*"
-	fi
+	finalize_test_case_output ok "$@"
 	test_success=$(($test_success + 1))
 	say_color "" "ok $test_count - $@"
 }
 
 test_failure_ () {
-	if test -n "$write_junit_xml"
-	then
-		junit_insert="<failure message=\"not ok $test_count -"
-		junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
-		junit_insert="$junit_insert $(xml_attr_encode \
-			"$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
-			   then
-				test-tool path-utils skip-n-bytes \
-					"$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET
-			   else
-				printf '%s\n' "$@" | sed 1d
-			   fi)")"
-		junit_insert="$junit_insert</failure>"
-		if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
-		then
-			junit_insert="$junit_insert<system-err>$(xml_attr_encode \
-				"$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")</system-err>"
-		fi
-		write_junit_xml_testcase "$1" "      $junit_insert"
-	fi
+	finalize_test_case_output failure "$@"
 	test_failure=$(($test_failure + 1))
 	say_color error "not ok $test_count - $1"
 	shift
@@ -814,19 +798,13 @@ test_failure_ () {
 }
 
 test_known_broken_ok_ () {
-	if test -n "$write_junit_xml"
-	then
-		write_junit_xml_testcase "$* (breakage fixed)"
-	fi
+	finalize_test_case_output fixed "$@"
 	test_fixed=$(($test_fixed+1))
 	say_color error "ok $test_count - $@ # TODO known breakage vanished"
 }
 
 test_known_broken_failure_ () {
-	if test -n "$write_junit_xml"
-	then
-		write_junit_xml_testcase "$* (known breakage)"
-	fi
+	finalize_test_case_output broken "$@"
 	test_broken=$(($test_broken+1))
 	say_color warn "not ok $test_count - $@ # TODO known breakage"
 }
@@ -1103,10 +1081,7 @@ test_start_ () {
 	test_count=$(($test_count+1))
 	maybe_setup_verbose
 	maybe_setup_valgrind
-	if test -n "$write_junit_xml"
-	then
-		junit_start=$(test-tool date getnanos)
-	fi
+	start_test_case_output
 }
 
 test_finish_ () {
@@ -1157,12 +1132,7 @@ test_skip () {
 
 	case "$to_skip" in
 	t)
-		if test -n "$write_junit_xml"
-		then
-			message="$(xml_attr_encode "$skipped_reason")"
-			write_junit_xml_testcase "$1" \
-				"      <skipped message=\"$message\" />"
-		fi
+		finalize_test_case_output skip "$@"
 
 		say_color skip "ok $test_count # skip $1 ($skipped_reason)"
 		: true
@@ -1178,53 +1148,6 @@ test_at_end_hook_ () {
 	:
 }
 
-write_junit_xml () {
-	case "$1" in
-	--truncate)
-		>"$junit_xml_path"
-		junit_have_testcase=
-		shift
-		;;
-	esac
-	printf '%s\n' "$@" >>"$junit_xml_path"
-}
-
-xml_attr_encode () {
-	printf '%s\n' "$@" | test-tool xml-encode
-}
-
-write_junit_xml_testcase () {
-	junit_attrs="name=\"$(xml_attr_encode "$this_test.$test_count $1")\""
-	shift
-	junit_attrs="$junit_attrs classname=\"$this_test\""
-	junit_attrs="$junit_attrs time=\"$(test-tool \
-		date getnanos $junit_start)\""
-	write_junit_xml "$(printf '%s\n' \
-		"    <testcase $junit_attrs>" "$@" "    </testcase>")"
-	junit_have_testcase=t
-}
-
-finalize_junit_xml () {
-	if test -n "$write_junit_xml" && test -n "$junit_xml_path"
-	then
-		test -n "$junit_have_testcase" || {
-			junit_start=$(test-tool date getnanos)
-			write_junit_xml_testcase "all tests skipped"
-		}
-
-		# adjust the overall time
-		junit_time=$(test-tool date getnanos $junit_suite_start)
-		sed -e "s/\(<testsuite.*\) time=\"[^\"]*\"/\1/" \
-			-e "s/<testsuite [^>]*/& time=\"$junit_time\"/" \
-			-e '/^ *<\/testsuite/d' \
-			<"$junit_xml_path" >"$junit_xml_path.new"
-		mv "$junit_xml_path.new" "$junit_xml_path"
-
-		write_junit_xml "  </testsuite>" "</testsuites>"
-		write_junit_xml=
-	fi
-}
-
 test_atexit_cleanup=:
 test_atexit_handler () {
 	# In a succeeding test script 'test_atexit_handler' is invoked
@@ -1247,7 +1170,7 @@ test_done () {
 	# removed, so the commands can access pidfiles and socket files.
 	test_atexit_handler
 
-	finalize_junit_xml
+	finalize_test_output
 
 	if test -z "$HARNESS_ACTIVE"
 	then
@@ -1538,22 +1461,7 @@ fi
 # in subprocesses like git equals our $PWD (for pathname comparisons).
 cd -P "$TRASH_DIRECTORY" || exit 1
 
-if test -n "$write_junit_xml"
-then
-	junit_xml_dir="$TEST_OUTPUT_DIRECTORY/out"
-	mkdir -p "$junit_xml_dir"
-	junit_xml_base=${0##*/}
-	junit_xml_path="$junit_xml_dir/TEST-${junit_xml_base%.sh}.xml"
-	junit_attrs="name=\"${junit_xml_base%.sh}\""
-	junit_attrs="$junit_attrs timestamp=\"$(TZ=UTC \
-		date +%Y-%m-%dT%H:%M:%S)\""
-	write_junit_xml --truncate "<testsuites>" "  <testsuite $junit_attrs>"
-	junit_suite_start=$(test-tool date getnanos)
-	if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
-	then
-		GIT_TEST_TEE_OFFSET=0
-	fi
-fi
+start_test_output "$0"
 
 # Convenience
 # A regexp to match 5 and 35 hexdigits
-- 
2.36.0.879.g3659959fcca

[RFC PATCH v5 07/10] test(junit): avoid line feeds in XML attributes

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

In the test case's output, we do want newline characters, but in the XML
attributes we do not want them.

However, the `xml_attr_encode` function always adds a Line Feed at the
end (which are then encoded as `&#x0a;`, even for XML attributes.

This seems not to faze Azure Pipelines' XML parser, but it still is
incorrect, so let's fix it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib-junit.sh | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/t/test-lib-junit.sh b/t/test-lib-junit.sh
index 9d55d74d764..c959183c7e2 100644
--- a/t/test-lib-junit.sh
+++ b/t/test-lib-junit.sh
@@ -50,7 +50,7 @@ finalize_test_case_output () {
 		;;
 	failure)
 		junit_insert="<failure message=\"not ok $test_count -"
-		junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
+		junit_insert="$junit_insert $(xml_attr_encode --no-lf "$1")\">"
 		junit_insert="$junit_insert $(xml_attr_encode \
 			"$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
 			   then
@@ -74,12 +74,12 @@ finalize_test_case_output () {
 		set "$* (known breakage)"
 		;;
 	skip)
-		message="$(xml_attr_encode "$skipped_reason")"
+		message="$(xml_attr_encode --no-lf "$skipped_reason")"
 		set "$1" "      <skipped message=\"$message\" />"
 		;;
 	esac
 
-	junit_attrs="name=\"$(xml_attr_encode "$this_test.$test_count $1")\""
+	junit_attrs="name=\"$(xml_attr_encode --no-lf "$this_test.$test_count $1")\""
 	shift
 	junit_attrs="$junit_attrs classname=\"$this_test\""
 	junit_attrs="$junit_attrs time=\"$(test-tool \
@@ -122,5 +122,11 @@ write_junit_xml () {
 }
 
 xml_attr_encode () {
-	printf '%s\n' "$@" | test-tool xml-encode
+	if test "x$1" = "x--no-lf"
+	then
+		shift
+		printf '%s' "$*" | test-tool xml-encode
+	else
+		printf '%s\n' "$@" | test-tool xml-encode
+	fi
 }
-- 
2.36.0.879.g3659959fcca

[RFC PATCH v5 08/10] ci: optionally mark up output in the GitHub workflow

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

A couple of commands exist to spruce up the output in GitHub workflows:
https://docs.github.com/en/actions/learn-github-actions/workflow-commands-for-github-actions

In addition to the `::group::<label>`/`::endgroup::` commands (which we
already use to structure the output of the build step better), we also
use `::error::`/`::notice::` to draw the attention to test failures and
to test cases that were expected to fail but didn't.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib-functions.sh              |  4 +--
 t/test-lib-github-workflow-markup.sh | 50 ++++++++++++++++++++++++++++
 t/test-lib.sh                        |  5 ++-
 3 files changed, 56 insertions(+), 3 deletions(-)
 create mode 100644 t/test-lib-github-workflow-markup.sh

diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index 93c03380d44..af4831a54c6 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -795,7 +795,7 @@ test_verify_prereq () {
 }
 
 test_expect_failure () {
-	test_start_
+	test_start_ "$@"
 	test "$#" = 3 && { test_prereq=$1; shift; } || test_prereq=
 	test "$#" = 2 ||
 	BUG "not 2 or 3 parameters to test-expect-failure"
@@ -815,7 +815,7 @@ test_expect_failure () {
 }
 
 test_expect_success () {
-	test_start_
+	test_start_ "$@"
 	test "$#" = 3 && { test_prereq=$1; shift; } || test_prereq=
 	test "$#" = 2 ||
 	BUG "not 2 or 3 parameters to test-expect-success"
diff --git a/t/test-lib-github-workflow-markup.sh b/t/test-lib-github-workflow-markup.sh
new file mode 100644
index 00000000000..d8dc969df4a
--- /dev/null
+++ b/t/test-lib-github-workflow-markup.sh
@@ -0,0 +1,50 @@
+# Library of functions to mark up test scripts' output suitable for
+# pretty-printing it in GitHub workflows.
+#
+# Copyright (c) 2022 Johannes Schindelin
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see http://www.gnu.org/licenses/ .
+#
+# The idea is for `test-lib.sh` to source this file when run in GitHub
+# workflows; these functions will then override (empty) functions
+# that are are called at the appropriate times during the test runs.
+
+start_test_output () {
+	test -n "$GIT_TEST_TEE_OUTPUT_FILE" ||
+	die "--github-workflow-markup requires --verbose-log"
+	github_markup_output="${GIT_TEST_TEE_OUTPUT_FILE%.out}.markup"
+	>$github_markup_output
+	GIT_TEST_TEE_OFFSET=0
+}
+
+# No need to override start_test_case_output
+
+finalize_test_case_output () {
+	test_case_result=$1
+	shift
+	case "$test_case_result" in
+	failure)
+		echo >>$github_markup_output "::error::failed: $this_test.$test_count $1"
+		;;
+	fixed)
+		echo >>$github_markup_output "::notice::fixed: $this_test.$test_count $1"
+		;;
+	esac
+	echo >>$github_markup_output "::group::$test_case_result: $this_test.$test_count $*"
+	test-tool >>$github_markup_output path-utils skip-n-bytes \
+		"$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET
+	echo >>$github_markup_output "::endgroup::"
+}
+
+# No need to override finalize_test_output
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 62a57b00730..8a63b4b64c9 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -204,6 +204,9 @@ parse_option () {
 	--write-junit-xml)
 		. "$TEST_DIRECTORY/test-lib-junit.sh"
 		;;
+	--github-workflow-markup)
+		. "$TEST_DIRECTORY/test-lib-github-workflow-markup.sh"
+		;;
 	--stress)
 		stress=t ;;
 	--stress=*)
@@ -1081,7 +1084,7 @@ test_start_ () {
 	test_count=$(($test_count+1))
 	maybe_setup_verbose
 	maybe_setup_valgrind
-	start_test_case_output
+	start_test_case_output "$@"
 }
 
 test_finish_ () {
-- 
2.36.0.879.g3659959fcca

[RFC PATCH v5 09/10] ci: use `--github-workflow-markup` in the GitHub workflow

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

[Ævar: due to re-structuring on top of my series the {begin,end}_group
in CI isn't needed at all to get "group" output for the test
suite. This commit includes the now-squashed "ci/run-build-and-tests:
add some structure to the GitHub workflow output":]

The current output of Git's GitHub workflow can be quite confusing,
especially for contributors new to the project.

To make it more helpful, let's introduce some collapsible grouping.
Initially, readers will see the high-level view of what actually
happened (did the build fail, or the test suite?). To drill down, the
respective group can be expanded.

Note: sadly, workflow output currently cannot contain any nested groups
(see https://github.com/actions/runner/issues/802 for details),
therefore we take pains to ensure to end any previous group before
starting a new one.

[Ævar: The above comment isn't true anymore, as that limitation has
been removed by basing this on my patches to run "make" and "make
test" directly from the top-level of main.yml.

Those are now effectively their own "group", effectively giving this
stage another group "level" to use. This means that the equivalent of
"make test" won't be on the same level as an individual test failure.

We no longer take any pains to ensure balanced group output as a
result (which was a caveat the previous ci/lib.sh implementation had
to deal with., We just need to "cat" the generated *.markup]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                 | 2 +-
 ci/print-test-failures.sh | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 80e89f89b7f..422399b1305 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -189,7 +189,7 @@ MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
 case "$CI_TYPE" in
 github-actions)
 	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
-	GIT_TEST_OPTS="--verbose-log -x"
+	GIT_TEST_OPTS="--verbose-log -x --github-workflow-markup"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 814c7799b68..2dda16a0d97 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -62,12 +62,13 @@ do
 		if test -n "$github_workflow_markup"
 		then
 			printf "\\e[33m\\e[1m=== Failed test: ${TEST_NAME} ===\\e[m\\n"
+			cat "t/test-results/$TEST_MARKUP"
 		else
 			echo "------------------------------------------------------------------------"
 			echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
 			echo "------------------------------------------------------------------------"
+			cat "t/test-results/${TEST_OUT}"
 		fi
-		cat "t/test-results/${TEST_OUT}"
 
 		trash_dir="trash directory.$TEST_NAME"
 		case "$CI_TYPE" in
-- 
2.36.0.879.g3659959fcca

[RFC PATCH v5 10/10] ci: call `finalize_test_case_output` a little later

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

We used to call that function already before printing the final verdict.
However, now that we added grouping to the GitHub workflow output, we
will want to include even that part in the collapsible group for that
test case.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib.sh | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/t/test-lib.sh b/t/test-lib.sh
index 8a63b4b64c9..1ceba46215f 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -782,13 +782,13 @@ trap '{ code=$?; set +x; } 2>/dev/null; exit $code' INT TERM HUP
 # the test_expect_* functions instead.
 
 test_ok_ () {
-	finalize_test_case_output ok "$@"
 	test_success=$(($test_success + 1))
 	say_color "" "ok $test_count - $@"
+	finalize_test_case_output ok "$@"
 }
 
 test_failure_ () {
-	finalize_test_case_output failure "$@"
+	failure_label=$1
 	test_failure=$(($test_failure + 1))
 	say_color error "not ok $test_count - $1"
 	shift
@@ -798,18 +798,19 @@ test_failure_ () {
 		say_color error "1..$test_count"
 		_error_exit
 	fi
+	finalize_test_case_output failure "$failure_label" "$@"
 }
 
 test_known_broken_ok_ () {
-	finalize_test_case_output fixed "$@"
 	test_fixed=$(($test_fixed+1))
 	say_color error "ok $test_count - $@ # TODO known breakage vanished"
+	finalize_test_case_output fixed "$@"
 }
 
 test_known_broken_failure_ () {
-	finalize_test_case_output broken "$@"
 	test_broken=$(($test_broken+1))
 	say_color warn "not ok $test_count - $@ # TODO known breakage"
+	finalize_test_case_output broken "$@"
 }
 
 test_debug () {
@@ -1135,10 +1136,10 @@ test_skip () {
 
 	case "$to_skip" in
 	t)
-		finalize_test_case_output skip "$@"
 
 		say_color skip "ok $test_count # skip $1 ($skipped_reason)"
 		: true
+		finalize_test_case_output skip "$@"
 		;;
 	*)
 		false
-- 
2.36.0.879.g3659959fcca

[PATCH v6 00/14] CI: js/ci-github-workflow-markup rebased on "use $GITHUB_ENV"

[Ævar Arnfjörð Bjarmason]

This non-RFC re-roll of [1] is based on the just-re-rolled [2]
(hereafter "base") and proposes to replace [3]. It incorporates the
relevant parts of [3] that were new in that re-roll.

Before getting into the details it's helpful to open the following
links:

  master: https://github.com/avar/git/actions/runs/2377697338
  js: https://github.com/avar/git/actions/runs/2377281297
  base: https://github.com/avar/git/actions/runs/2377257487
  this: https://github.com/avar/git/actions/runs/2383110429
  this with 'raw': https://github.com/avar/git/actions/runs/2383134850

These are all versions of the relevant branches in play with the same
commit on top to trigger CI failures. The "master" and "base" are the
same as those noted in [2], the "js" is [3] (but rebased on master),
and "this" is this topic. The "this with 'raw'" is this topic, but
configured to omit the new GitHub CI Markdown output (see 13/14 and
14/14).

Having these open allows us to compare the output and performance of
these different proposed CI topics. Here clicking on e.g. linux-sha256
(which has a verbose test failure) will on these topics on in the
Chrome debugger[4].

  master: https://github.com/avar/git/runs/6573058614?check_suite_focus=true
  262 ms  Loading
  3977 ms  Scripting
  1632 ms  Rendering
  641 ms  Painting
  934 ms  System
  920 ms  Idle
  8366 ms  Total

  js: https://github.com/avar/git/runs/6571972194?check_suite_focus=true
  170 ms  Loading
  12934 ms  Scripting
  7364 ms  Rendering
  6940 ms  Painting
  5489 ms  System
  5940 ms  Idle
  38836 ms  Total

  base: https://github.com/avar/git/runs/6571804381?check_suite_focus=true
  215 ms  Loading
  3270 ms  Scripting
  1171 ms  Rendering
  649 ms  Painting
  941 ms  System
  731 ms  Idle
  6976 ms  Total

  this: https://github.com/avar/git/runs/6588407676?check_suite_focus=true
  201 ms  Loading
  9192 ms  Scripting
  6104 ms  Rendering
  5553 ms  Painting
  4684 ms  System
  6184 ms  Idle
  31917 ms  Total

  this with 'raw': https://github.com/avar/git/runs/6588579493?check_suite_focus=true
  173 ms  Loading
  3036 ms  Scripting
  1110 ms  Rendering
  617 ms  Painting
  1028 ms  System
  714 ms  Idle
  6678 ms  Total

As noted in previous performance testing [5] what these results
indicate is that the more output you put under GitHub CI's "groups"
the slower it will get. With the "js" variant in particular it can be
seen to struggle to render as you're watching it.

It's faster with "this" because there's less rendering to be done. Due
to the changes in the "base" topic to make "make" and "make test"
top-level commands we can run there's less data in the step where we
need to print failures in detail.

So re: [6] this shows how the two serieses are complimentary and make
the overall CI UX better when combined than stand-alone.

Changes since the RFC v5:

 * Incorporated the v3[3] of js/ci-github-workflow-markup.

 * Applied fix-up to get rid of broken
   ci/print-test-failures-github.sh (which didn't exist anymore)

 * The v5 of this series had an issue where we'd "zoom in" on the
   "make test" output, instead of the new GitHub CI markdown
   output. We now combine the best of both worlds. We'll have a "make
   test" and "ci/print-test-failures.sh" steps, both of which fail,
   but we'll "zoom in" on the latter, which contains the new output.

 * As before this series applied some DRY principles to the code [7]
   proposed to add, i.e. the "handle_failed_tests" was a
   near-duplication of the code in
   ci/print-test-failures-github.sh. The 01-03/14 here allow us to
   unify the two & share the code instead.

   My RFC v5 had some options we didn't need (--exit-code and
   --no-exit-code) that I removed. It furthermore defaults to "auto"
   discovering if we should emit the GitHub CI *.markup output based
   on if we find those files in t/test-results, which as we'll see in
   13-14/14 makes configuring the output simpler.

 * Due to the above and other changes here being smaller the "ci: make
   it easier to find failed tests' logs in the GitHub workflow" here
   is now a squashed version of changes done in several steps in
   [3]. We now add the ability to emit GitHub CI markup in one commit,
   and enable it in another.

 * In the latest "js" topic the "ok" tests were skipped, but we'd
   still include "skip" tests, i.e. "not ok .. TODO" tests. These
   should be considered "ok" for the purposes of the output, so a new
   09/14 here excludes them too.

 * An updated commit message for 10/14 notes a subtle and presumably
   unintended side-effect/bug in "ci(github): avoid printing test case
   preamble twice" new in the v3 of [3]. In tweaking the *.markup
   files we also change the output in the *.out files (i.e. the raw
   logs).

   You thus won't get the same output in *.out (the raw logs) with the
   new output.

 * New 13/14 and 14/14 commits here make the CI output type
   configurable using the same "ci-config" method we use for the
   existing "check-ref". As explained in 14/14 this is what I'm using
   to produce "this with 'raw'" output the same commit as "this". I
   push to a branch whose name ends with "_CI_OUT_RAW".

   This allows users (such as me, or per 13/14 probably SZEDER too) to
   configure our output to avoid this GitHub markdown in our
   ci-config.

 * As 13/14 argues for I think a better thing to do would be to avoid
   14/14 for now and take this series without making the new "github"
   output the default.

   But in the interest of not conflating that with this proposed
   re-roll 14/14 changes the default from "raw" to "github" again,
   thus making the output [3] is proposing the default here too.

   We can always revert 14/14 later if we think that was premature,
   which as shown there is now a one-line change.

To continue from the last note above: This series also allows for
combining the new & old output in a way where we can have our cake &
eat it too. If we load the "this" output and "zoom in" on the failed
"make test" step instead of the subsequent "ci/print-test-failures.sh"
step we'll load the page in around 1/6 the time, or:

  this (zoomed in on 'make test'): https://github.com/avar/git/runs/6588407676?check_suite_focus=true#step:7:1437
  149 ms  Loading
  2559 ms  Scripting
  934 ms  Rendering
  591 ms  Painting
  845 ms  System
  605 ms  Idle
  5684 ms  Total

Clicking on the "ci/print-test-failures.sh" will then take just as
long. I haven't implemented this as a configurable feature is just a
matter of making ci/print-test-failures.sh exit with 0 where it now
exits with 1 (as we always "zoom in" on the last failed step, which
then will be "make test").

For Junio:

If you eject js/ci-github-workflow-markup you'll get a minor conflict
(as before) with js/use-builtin-add-i, and a new conflict with
jx/l10n-workflow-change. The correct resolution is to "git rm
ci/run-build-and-tests.sh ci/run-static-analysis.sh" and apply this
diff on top:

	diff --git a/Makefile b/Makefile
	index b602bdaed4a..984b1a1905d 100644
	--- a/Makefile
	+++ b/Makefile
	@@ -3625,3 +3625,4 @@ ci-static-analysis: ci-check-directional-formatting
	 ci-static-analysis: check-builtins
	 ci-static-analysis: check-coccicheck
	 ci-static-analysis: hdr-check
	+ci-static-analysis: check-pot
	diff --git a/ci/lib.sh b/ci/lib.sh
	index b148d3f209f..c6b17b4cb22 100755
	--- a/ci/lib.sh
	+++ b/ci/lib.sh
	@@ -282,7 +282,7 @@ linux-TEST-vars)
	 	setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
	 	setenv --test GIT_TEST_MULTI_PACK_INDEX 1
	 	setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
	-	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
	+	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 0
	 	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
	 	setenv --test GIT_TEST_WRITE_REV_INDEX 1
	 	setenv --test GIT_TEST_CHECKOUT_WORKERS 2


1. https://lore.kernel.org/git/RFC-cover-v5-00.10-00000000000-20220421T183001Z-avarab@gmail.com/
2. https://lore.kernel.org/git/cover-v6-00.29-00000000000-20220525T094123Z-avarab@gmail.com/
3. https://lore.kernel.org/git/pull.1117.v3.git.1653171536.gitgitgadget@gmail.com/
4. Version 101.0.4951.54 (Official Build) (64-bit). Opening the
   debugger in an empty tab (F12), going to "Performance" and
   recording a trace (Ctrl+E) (note: *not* Ctrl+Shift+E, as that would
   end it after network load, but before rendering is done & the page
   is ready to interact with).

   The "Total" times here are "real" and don't very with how fast I am
   to "stop" the trace (tested by leaving it lingering for a long
   time).
5. https://lore.kernel.org/git/220523.86ee0kzjix.gmgdl@evledraar.gmail.com/
6. https://lore.kernel.org/git/nycvar.QRO.7.76.6.2205232040130.352@tvgsbejvaqbjf.bet/
7. https://lore.kernel.org/git/417f702a2454316f0bdc5b66eb16751272ff2989.1653171536.git.gitgitgadget@gmail.com/

Johannes Schindelin (6):
  tests: refactor --write-junit-xml code
  test(junit): avoid line feeds in XML attributes
  ci: optionally mark up output in the GitHub workflow
  ci(github): skip the logs of the successful test cases
  ci(github): mention where the full logs can be found
  ci: call `finalize_test_case_output` a little later

Victoria Dye (1):
  ci(github): avoid printing test case preamble twice

Ævar Arnfjörð Bjarmason (7):
  CI: don't "cd" in ci/print-test-failures.sh
  CI: don't include "test-results/" in ci/print-test-failures.sh output
  CI: stop setting FAILED_TEST_ARTIFACTS N times
  ci: make it easier to find failed tests' logs in the GitHub workflow
  ci(github): skip "skip" tests in --github-workflow-markup
  CI: make --github-workflow-markup ci-config, off by default
  CI: make the --github-workflow-markup "github" output the default

 .github/workflows/main.yml           |  24 +++++
 ci/lib.sh                            |  12 +++
 ci/print-test-failures.sh            | 100 ++++++++++++++++----
 t/test-lib-functions.sh              |   6 +-
 t/test-lib-github-workflow-markup.sh |  56 ++++++++++++
 t/test-lib-junit.sh                  | 132 +++++++++++++++++++++++++++
 t/test-lib.sh                        | 128 ++++----------------------
 7 files changed, 332 insertions(+), 126 deletions(-)
 create mode 100644 t/test-lib-github-workflow-markup.sh
 create mode 100644 t/test-lib-junit.sh

Range-diff against v5:
 1:  56910df26e7 =  1:  c10a11fdae8 CI: don't "cd" in ci/print-test-failures.sh
 3:  577a5be34f4 =  2:  1d07ca58322 CI: don't include "test-results/" in ci/print-test-failures.sh output
 4:  1482f840f64 !  3:  fdc5b534644 CI: stop setting FAILED_TEST_ARTIFACTS N times
    @@ ci/print-test-failures.sh: do
      			cp "t/test-results/${TEST_OUT}" t/failed-test-artifacts/
      			(
      				cd t &&
    -@@ ci/print-test-failures.sh: done
    - 
    - if test -n "$failed"
    - then
    +@@ ci/print-test-failures.sh: do
    + 		esac
    + 	fi
    + done
    ++
    ++if test -n "$failed"
    ++then
     +	if test -n "$GITHUB_ENV"
     +	then
     +		echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
     +	fi
    -+
    - 	if test -n "$exit_code"
    - 	then
    - 		exit 1
    ++fi
 6:  d9b7d5ddf5a =  4:  a75846a12f2 tests: refactor --write-junit-xml code
 7:  2d233bbdc22 =  5:  5b709dde86a test(junit): avoid line feeds in XML attributes
 8:  5dd43a1624b =  6:  f5b39f1660d ci: optionally mark up output in the GitHub workflow
 2:  caec0b1089a !  7:  e713ca6041b CI: add --exit-code to ci/print-test-failures.sh
    @@ Metadata
     Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## Commit message ##
    -    CI: add --exit-code to ci/print-test-failures.sh
    +    ci: make it easier to find failed tests' logs in the GitHub workflow
     
    -    The ci/print-test-failures.sh scripts will exit 0 when there's failed
    -    tests, let's teach it --exit-code to have it exit non-zero in those
    -    cases.
    +    When investigating a test failure, the time that matters most is the
    +    time it takes from getting aware of the failure to displaying the output
    +    of the failing test case.
     
    +    You currently have to know a lot of implementation details when
    +    investigating test failures in the CI runs. The first step is easy: the
    +    failed job is marked quite clearly, but when opening it, the failed step
    +    is expanded, which in our case is the one running
    +    `make` or 'make test`. When failing in the latter we only show a
    +    high-level view of what went wrong: it prints the output of `prove`
    +    which merely tells the reader which test script failed, along with the failed
    +    test numbers.
    +
    +    To diagnose those failures in detail the user has then had to know
    +    that they should expand the subsequent "Run ci/print-test-failures.sh"
    +    step. Because that step always exits successfully it's not expanded by
    +    default (the GitHub CI only expands the last failing step), making it
    +    easy to miss that there is useful information hidden in there.
    +
    +    This change does a couple of things to improve the situation:
    +
    +     * We now make use of the "exit code" feature of
    +       ci/print-test-failures.sh added in a preceding commit, thus if "make
    +       test" fails we'll pick up that we have failing tests, and fail in
    +       ci/print-test-failures.sh too.
    +
    +       Note that we still exit successfully from ci/print-test-failures.sh
    +       if we don't have logs of any failed tests in t/test-results. Thus
    +       if we fail in "make" (i.e. when building), or even in "make test"
    +       in a way that doesn't yield a failed test (e.g. the t/Makefile
    +       itself failing) we'll correctly "zoom in" on the (hopefully
    +       failing) "make test" instead.
    +
    +       As before if there was no previously failed step the
    +       ci/print-test-failures.sh step is skipped entirely (see
    +       .github/workflows/main.yml)
    +
    +       To make the script DWYM the ci/print-test-failures.sh defaults to
    +       an "auto" mode. Thus since we turn on --github-workflow-markup in
    +       ci/lib.sh we'll write *.markup files, and will pick up on that in
    +       ci/print-test-failures.sh.
    +
    +       The "$github_workflow_markup" being "auto" (or "t") on failure is
    +       also what makes us exit non-zero. We could also add
    +       --[no-]exit-code arguments, but at this point we only need this for
    +       the "*.markup" mode, so let's leave that for now.
    +
    +       By having this be "auto" well pick up on whatever the GIT_TEST_OPTS
    +       setting is, which as we'll see in a subsequent commit will be
    +       useful if we'd like to configure the CI to emit it only some of the
    +       time.
    +
    +     * The ci/print-test-failures.sh now makes use of the GitHub markdown
    +       introduced for t/test-lib.sh in the preceding commit. I.e. in GitHub
    +       CI we'll now run with --github-workflow-markup in GIT_TEST_OPTS,
    +       which causes our "--verbose" output in
    +       "t/test-results/$TEST_OUT.markup" to be in the GitHub CI-specific
    +       Markdown format.
    +
    +    The new output is easier to read since it makes use of collapsible grouping.
    +    Initially, readers will see the high-level view of what actually happened.
    +    To drill down, the "group" for the specific failing test(s) can be expanded.
    +
    +    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
    + ## ci/lib.sh ##
    +@@ ci/lib.sh: MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
    + case "$CI_TYPE" in
    + github-actions)
    + 	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
    +-	GIT_TEST_OPTS="--verbose-log -x"
    ++	GIT_TEST_OPTS="--verbose-log -x --github-workflow-markup"
    + 	test Windows != "$RUNNER_OS" ||
    + 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
    + 	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
    +
      ## ci/print-test-failures.sh ##
     @@ ci/print-test-failures.sh: set -e
      . ${0%/*}/lib-ci-type.sh
      . ${0%/*}/lib-tput.sh
      
    -+exit_code=
    ++github_workflow_markup=auto
    ++case "$CI_TYPE" in
    ++github-actions)
    ++	github_workflow_markup=t
    ++	;;
    ++esac
    ++
     +while test $# != 0
     +do
     +	case "$1" in
    -+	--exit-code)
    -+		exit_code=t
    ++	--github-workflow-markup)
    ++		github_workflow_markup=t
    ++		;;
    ++	--no-github-workflow-markup)
    ++		github_workflow_markup=
     +		;;
     +	*)
     +		echo "BUG: invalid $0 argument: $1" >&2
    @@ ci/print-test-failures.sh: set -e
      		TEST_NAME="${TEST_EXIT%.exit}"
      		TEST_NAME="${TEST_NAME##*/}"
      		TEST_OUT="${TEST_NAME}.out"
    -@@ ci/print-test-failures.sh: do
    - 		esac
    + 		TEST_MARKUP="${TEST_NAME}.markup"
    + 
    +-		echo "------------------------------------------------------------------------"
    +-		echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
    +-		echo "------------------------------------------------------------------------"
    +-		cat "t/test-results/${TEST_OUT}"
    ++		do_markup=
    ++		case "$github_workflow_markup" in
    ++		t)
    ++			do_markup=t
    ++			;;
    ++		auto)
    ++			if test -f "t/test-results/$TEST_MARKUP"
    ++			then
    ++				do_markup=t
    ++			fi
    ++			;;
    ++		esac
    ++
    ++		if test -n "$do_markup"
    ++		then
    ++			printf "\\e[33m\\e[1m=== Failed test: ${TEST_NAME} ===\\e[m\\n"
    ++			cat "t/test-results/$TEST_MARKUP"
    ++		else
    ++			echo "------------------------------------------------------------------------"
    ++			echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
    ++			echo "------------------------------------------------------------------------"
    ++			cat "t/test-results/${TEST_OUT}"
    ++		fi
    + 
    + 		trash_dir="trash directory.$TEST_NAME"
    + 		case "$CI_TYPE" in
    +@@ ci/print-test-failures.sh: then
    + 	then
    + 		echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
      	fi
    - done
     +
    -+if test -n "$failed"
    -+then
    -+	if test -n "$exit_code"
    -+	then
    ++	case "$github_workflow_markup" in
    ++	t|auto)
     +		exit 1
    -+	fi
    -+fi
    ++		;;
    ++	'')
    ++		exit 0
    ++		;;
    ++	esac
    + fi
 -:  ----------- >  8:  9c9fd24fffa ci(github): skip the logs of the successful test cases
 -:  ----------- >  9:  03d7b54d68b ci(github): skip "skip" tests in --github-workflow-markup
 -:  ----------- > 10:  90a152d79f9 ci(github): avoid printing test case preamble twice
 9:  877edff3b4d ! 11:  68eed85162b ci: use `--github-workflow-markup` in the GitHub workflow
    @@ Metadata
     Author: Johannes Schindelin <Johannes.Schindelin@gmx.de>
     
      ## Commit message ##
    -    ci: use `--github-workflow-markup` in the GitHub workflow
    +    ci(github): mention where the full logs can be found
     
    -    [Ævar: due to re-structuring on top of my series the {begin,end}_group
    -    in CI isn't needed at all to get "group" output for the test
    -    suite. This commit includes the now-squashed "ci/run-build-and-tests:
    -    add some structure to the GitHub workflow output":]
    -
    -    The current output of Git's GitHub workflow can be quite confusing,
    -    especially for contributors new to the project.
    -
    -    To make it more helpful, let's introduce some collapsible grouping.
    -    Initially, readers will see the high-level view of what actually
    -    happened (did the build fail, or the test suite?). To drill down, the
    -    respective group can be expanded.
    -
    -    Note: sadly, workflow output currently cannot contain any nested groups
    -    (see https://github.com/actions/runner/issues/802 for details),
    -    therefore we take pains to ensure to end any previous group before
    -    starting a new one.
    -
    -    [Ævar: The above comment isn't true anymore, as that limitation has
    -    been removed by basing this on my patches to run "make" and "make
    -    test" directly from the top-level of main.yml.
    -
    -    Those are now effectively their own "group", effectively giving this
    -    stage another group "level" to use. This means that the equivalent of
    -    "make test" won't be on the same level as an individual test failure.
    -
    -    We no longer take any pains to ensure balanced group output as a
    -    result (which was a caveat the previous ci/lib.sh implementation had
    -    to deal with., We just need to "cat" the generated *.markup]
    +    The full logs are contained in the `failed-tests-*.zip` artifacts that
    +    are attached to the failed CI run. Since this is not immediately
    +    obvious to the well-disposed reader, let's mention it explicitly.
     
    +    Suggested-by: Victoria Dye <vdye@github.com>
         Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
    - ## ci/lib.sh ##
    -@@ ci/lib.sh: MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
    - case "$CI_TYPE" in
    - github-actions)
    - 	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
    --	GIT_TEST_OPTS="--verbose-log -x"
    -+	GIT_TEST_OPTS="--verbose-log -x --github-workflow-markup"
    - 	test Windows != "$RUNNER_OS" ||
    - 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
    - 	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
    -
      ## ci/print-test-failures.sh ##
     @@ ci/print-test-failures.sh: do
    - 		if test -n "$github_workflow_markup"
    + 		if test -n "$do_markup"
      		then
      			printf "\\e[33m\\e[1m=== Failed test: ${TEST_NAME} ===\\e[m\\n"
    -+			cat "t/test-results/$TEST_MARKUP"
    ++			echo "The full logs are in the artifacts attached to this run."
    + 			cat "t/test-results/$TEST_MARKUP"
      		else
      			echo "------------------------------------------------------------------------"
    - 			echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
    - 			echo "------------------------------------------------------------------------"
    -+			cat "t/test-results/${TEST_OUT}"
    - 		fi
    --		cat "t/test-results/${TEST_OUT}"
    - 
    - 		trash_dir="trash directory.$TEST_NAME"
    - 		case "$CI_TYPE" in
10:  6b278194f9a = 12:  11dab84b0dc ci: call `finalize_test_case_output` a little later
 5:  4291892fbd3 ! 13:  fbe0d99c6b3 ci: make it easier to find failed tests' logs in the GitHub workflow
    @@
      ## Metadata ##
    -Author: Johannes Schindelin <Johannes.Schindelin@gmx.de>
    +Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## Commit message ##
    -    ci: make it easier to find failed tests' logs in the GitHub workflow
    -
    -    When investigating a test failure, the time that matters most is the
    -    time it takes from getting aware of the failure to displaying the output
    -    of the failing test case.
    -
    -    You currently have to know a lot of implementation details when
    -    investigating test failures in the CI runs. The first step is easy: the
    -    failed job is marked quite clearly, but when opening it, the failed step
    -    is expanded, which in our case is the one running
    -    `ci/run-build-and-tests.sh`. This step, most notably, only offers a
    -    high-level view of what went wrong: it prints the output of `prove`
    -    which merely tells the reader which test script failed.
    -
    -    The actually interesting part is in the detailed log of said failed
    -    test script. But that log is shown in the CI run's step that runs
    -    `ci/print-test-failures.sh`. And that step is _not_ expanded in the web
    -    UI by default. It is even marked as "successful", which makes it very
    -    easy to miss that there is useful information hidden in there.
    -
    -    Let's help the reader by showing the failed tests' detailed logs in the
    -    step that is expanded automatically, i.e. directly after the test suite
    -    failed.
    -
    -    This also helps the situation where the _build_ failed and the
    -    `print-test-failures` step was executed under the assumption that the
    -    _test suite_ failed, and consequently failed to find any failed tests.
    -
    -    An alternative way to implement this patch would be to source
    -    `ci/print-test-failures.sh` in the `handle_test_failures` function to
    -    show these logs. However, over the course of the next few commits, we
    -    want to introduce some grouping which would be harder to achieve that
    -    way (for example, we do want a leaner, and colored, preamble for each
    -    failed test script, and it would be trickier to accommodate the lack of
    -    nested groupings in GitHub workflows' output).
    -
    -    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    +    CI: make --github-workflow-markup ci-config, off by default
    +
    +    In preceding commits the --github-workflow-markup output was made the
    +    default under GitHub CI.
    +
    +    There's a few outstanding issues with that new output target however:
    +
    +    A. As noted in [1] it's much slower in some common cases, i.e. the
    +       page rendering time (under Firefox & Chrome debugging) goes from ~20s
    +       to on the order of 60-80s (less pronounced when not debugging, but
    +       still in that ballpark). See also [2] for tests on an earlier iteration
    +       of the series.
    +
    +    B. Preceding commits in this series to omit non-failing test output
    +       may have mitigated that somewhat, i.e. skipping "ok" and "skip" tests
    +       in the *.markup files (although [1] is a benchmark with that "ok"
    +       change).
    +
    +       Per [3] there are some CI users that prefer to see only the full
    +       verbose output, i.e. we can't in the general case assume that we
    +       can emit only the trace output for the failing test, and that's
    +       going to give us the information we need to debug the test. It
    +       might be failing because of earlier setup, or from trace output
    +       outside of any test (i.e. in the main body of the file).
    +
    +       Users such as [3] could still just consult the *.out output, but
    +       then we'd make them download the archive instead of viewing the
    +       output in the web UX.
    +
    +    C. Per B above and [3] an earlier change in this series modified
    +       the *.out output to change the *.out output inadvertently while trying
    +       to modify the *.markup output. See the commit message of "ci(github):
    +       avoid printing test case preamble twice" earlier in this series.
    +
    +    It thus make sense for now to make the "ci/print-test-failures.sh"
    +    output be the "raw" one. I.e. the *.out output we'd get before
    +    --github-workflow-markup was implemented.
    +
    +    To make it use ci/print-test-failures.sh we can create a
    +    "ci/config/print-test-failures-output-type" script on the "ci-config"
    +    branch that does e.g.:
    +
    +            #!/bin/sh
    +            echo github
    +
    +    That script will (like the existing "check-ref") get the ref as an
    +    argument, so we can in the future turn this on experimentally for
    +    "seen", and flip the default in the future. The script can also emit
    +    nothing to get whatever our idea is of the default (currently "raw").
    +
    +    Note the corresponding change to ci/print-test-failures.sh, i.e. we'll
    +    do the right thing vis-a-vis what we "zoom in" to depending on the
    +    setting. If it's "raw" we'd like a failed "make test" to "zoom in" to
    +    that step, if it's "github" we'd like the to open the subsequent
    +    "ci/print-test-failures.sh" step instead.
    +
    +    1. https://lore.kernel.org/git/220523.86ee0kzjix.gmgdl@evledraar.gmail.com/
    +    2. https://lore.kernel.org/git/220222.86tucr6kz5.gmgdl@evledraar.gmail.com/
    +    3. https://lore.kernel.org/git/20210309175249.GE3590451@szeder.dev/
    +
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## .github/workflows/main.yml ##
     @@ .github/workflows/main.yml: jobs:
    +     runs-on: ubuntu-latest
    +     outputs:
    +       enabled: ${{ steps.check-ref.outputs.enabled }}${{ steps.skip-if-redundant.outputs.enabled }}
    ++      print-test-failures-output-type: ${{ steps.print-test-failures-output-type.outputs.type }}
    +     steps:
    +       - name: try to clone ci-config branch
    +         run: |
    +@@ .github/workflows/main.yml: jobs:
    +             config-repo &&
    +           cd config-repo &&
    +           git checkout HEAD -- ci/config || : ignore
    ++      - id: print-test-failures-output-type
    ++        name: check what output type ci/print-test-failures.sh uses
    ++        run: |
    ++          type_default=raw
    ++          type=$type_default
    ++
    ++          if test -x config-repo/ci/config/print-test-failures-output-type
    ++          then
    ++            type=$(config-repo/ci/config/print-test-failures-output-type '${{ github.ref }}')
    ++            if test -z "$type"
    ++            then
    ++              type=$type_default
    ++            fi
    ++          fi
    ++          echo "::set-output name=type::$type"
    +       - id: check-ref
    +         name: check whether CI is enabled for ref
    +         run: |
    +@@ .github/workflows/main.yml: jobs:
    +     - uses: git-for-windows/setup-git-for-windows-sdk@v1
    +     - run: ci/lib.sh --test
            shell: bash
    -     - name: test
    ++      env:
    ++        GIT_CI_PTF_OUTPUT_TYPE: ${{needs.ci-config.outputs.print-test-failures-output-type}}
    +     - name: select tests
    +       run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
            shell: bash
    --      run: . /etc/profile && make -C t -e
    --    - name: ci/print-test-failures.sh
    --      if: failure()
    --      shell: bash
    --      run: ci/print-test-failures.sh
    -+      run: . /etc/profile && make -C t -e || ci/print-test-failures.sh
    -     - name: Upload failed tests' directories
    -       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
    -       uses: actions/upload-artifact@v2
     @@ .github/workflows/main.yml: jobs:
    +       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
    +     - run: ci/lib.sh --test
            shell: bash
    -     - name: test
    ++      env:
    ++        GIT_CI_PTF_OUTPUT_TYPE: ${{needs.ci-config.outputs.print-test-failures-output-type}}
    +     - name: select tests
    +       run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
            shell: bash
    --      run: . /etc/profile && make -C t -e
    --    - name: ci/print-test-failures.sh
    --      if: failure()
    --      shell: bash
    --      run: ci/print-test-failures.sh
    -+      run: . /etc/profile && make -C t -e || ci/print-test-failures-github.sh
    -     - name: Upload failed tests' directories
    -       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
    -       uses: actions/upload-artifact@v2
     @@ .github/workflows/main.yml: jobs:
          - run: ci/lib.sh --build
          - run: make
          - run: ci/lib.sh --test
    --    - run: make test
    -+    - run: make test || ci/print-test-failures-github.sh
    ++      env:
    ++        GIT_CI_PTF_OUTPUT_TYPE: ${{needs.ci-config.outputs.print-test-failures-output-type}}
    +     - run: make test
            if: success()
    --    - run: ci/print-test-failures.sh
    --      if: failure()
    -     - name: Upload failed tests' directories
    -       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
    -       uses: actions/upload-artifact@v2
    +     - run: ci/print-test-failures.sh
     @@ .github/workflows/main.yml: jobs:
          - run: make
          - run: ci/lib.sh --test
            if: success() && matrix.vector.skip-tests != 'yes'
    --    - run: make test
    -+    - run: make test || ci/print-test-failures-github.sh
    ++      env:
    ++        GIT_CI_PTF_OUTPUT_TYPE: ${{needs.ci-config.outputs.print-test-failures-output-type}}
    +     - run: make test
            if: success() && matrix.vector.skip-tests != 'yes'
    --    - run: ci/print-test-failures.sh
    --      if: failure() && matrix.vector.skip-tests != 'yes'
    -     - name: Upload failed tests' directories
    -       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
    -       uses: actions/upload-artifact@v1
    +     - run: ci/print-test-failures.sh
    +
    + ## ci/lib.sh ##
    +@@ ci/lib.sh: if test -n "$GITHUB_ENV"
    + then
    + 	echo "CONFIG: GITHUB_ENV=$GITHUB_ENV" >&2
    + fi
    ++echo "CONFIG: GIT_CI_PTF_OUTPUT_TYPE=$GIT_CI_PTF_OUTPUT_TYPE" >&2
    + echo >&2
    + 
    + # Helper functions
    +@@ ci/lib.sh: MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
    + case "$CI_TYPE" in
    + github-actions)
    + 	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
    +-	GIT_TEST_OPTS="--verbose-log -x --github-workflow-markup"
    ++	GIT_TEST_OPTS="--verbose-log -x"
    ++	if test -n "$GIT_CI_PTF_OUTPUT_TYPE"
    ++	then
    ++		# For later use in ci/print-test-failures.sh
    ++		setenv --test GIT_CI_PTF_OUTPUT_TYPE "$GIT_CI_PTF_OUTPUT_TYPE"
    ++
    ++		case "$GIT_CI_PTF_OUTPUT_TYPE" in
    ++		github)
    ++			GIT_TEST_OPTS="$GIT_TEST_OPTS --github-workflow-markup"
    ++			;;
    ++		esac
    ++	fi
    + 	test Windows != "$RUNNER_OS" ||
    + 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
    + 	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
     
      ## ci/print-test-failures.sh ##
     @@ ci/print-test-failures.sh: set -e
    - . ${0%/*}/lib-ci-type.sh
      . ${0%/*}/lib-tput.sh
      
    --exit_code=
    -+case "$CI_TYPE" in
    -+github-actions)
    -+	exit_code=t
    -+	github_workflow_markup=t
    -+	;;
    -+*)
    -+	exit_code=
    + github_workflow_markup=auto
    +-case "$CI_TYPE" in
    +-github-actions)
    ++case "$GIT_CI_PTF_OUTPUT_TYPE" in
    ++github)
    + 	github_workflow_markup=t
    + 	;;
    ++raw)
     +	github_workflow_markup=
     +	;;
    -+esac
    -+
    - while test $# != 0
    - do
    - 	case "$1" in
    - 	--exit-code)
    - 		exit_code=t
    - 		;;
    -+	--no-exit-code)
    -+		exit_code=
    -+		;;
    -+	--github-workflow-markup)
    -+		github_workflow_markup=t
    -+		;;
    -+	--no-github-workflow-markup)
    -+		github_workflow_markup=
    -+		;;
    - 	*)
    - 		echo "BUG: invalid $0 argument: $1" >&2
    - 		exit 1
    -@@ ci/print-test-failures.sh: do
    - 		TEST_OUT="${TEST_NAME}.out"
    - 		TEST_MARKUP="${TEST_NAME}.markup"
    - 
    --		echo "------------------------------------------------------------------------"
    --		echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
    --		echo "------------------------------------------------------------------------"
    -+		if test -n "$github_workflow_markup"
    -+		then
    -+			printf "\\e[33m\\e[1m=== Failed test: ${TEST_NAME} ===\\e[m\\n"
    -+		else
    -+			echo "------------------------------------------------------------------------"
    -+			echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
    -+			echo "------------------------------------------------------------------------"
    -+		fi
    - 		cat "t/test-results/${TEST_OUT}"
    + esac
      
    - 		trash_dir="trash directory.$TEST_NAME"
    + while test $# != 0
 -:  ----------- > 14:  0b02b186c87 CI: make the --github-workflow-markup "github" output the default
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 01/14] CI: don't "cd" in ci/print-test-failures.sh

[Ævar Arnfjörð Bjarmason]

Change the logic in ci/print-test-failures.sh so that we don't need to
"cd" at the top-level, we'll now only do so for the "tar" command.

This way we can remove some duplicate code added in
aea8879a6ac (travis-ci: include the trash directories of failed tests
in the trace log, 2018-08-01), i.e. not have both a $TEST_NAME and
$test_name. The output is exactly the same as before,
i.e. "test-results/" was included in the heading.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/print-test-failures.sh | 29 ++++++++++++++++-------------
 1 file changed, 16 insertions(+), 13 deletions(-)

diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index ec395c79ccd..f6736f424e0 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -8,33 +8,36 @@ set -e
 . ${0%/*}/lib-ci-type.sh
 . ${0%/*}/lib-tput.sh
 
-cd t/
-
-if ! ls test-results/*.exit >/dev/null 2>/dev/null
+if ! ls t/test-results/*.exit >/dev/null 2>/dev/null
 then
 	echo "Build job failed before the tests could have been run"
 	exit
 fi
 
-for TEST_EXIT in test-results/*.exit
+for TEST_EXIT in t/test-results/*.exit
 do
 	if [ "$(cat "$TEST_EXIT")" != "0" ]
 	then
-		TEST_OUT="${TEST_EXIT%exit}out"
+		TEST_NAME="${TEST_EXIT%.exit}"
+		TEST_NAME="${TEST_NAME##*/}"
+		TEST_OUT="${TEST_NAME}.out"
+		TEST_MARKUP="${TEST_NAME}.markup"
+
 		echo "------------------------------------------------------------------------"
-		echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
+		echo "$(tput setaf 1)test-results/${TEST_OUT}...$(tput sgr0)"
 		echo "------------------------------------------------------------------------"
-		cat "${TEST_OUT}"
+		cat "t/test-results/${TEST_OUT}"
 
-		test_name="${TEST_EXIT%.exit}"
-		test_name="${test_name##*/}"
-		trash_dir="trash directory.$test_name"
+		trash_dir="trash directory.$TEST_NAME"
 		case "$CI_TYPE" in
 		github-actions)
-			mkdir -p failed-test-artifacts
+			mkdir -p t/failed-test-artifacts
 			echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
-			cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
-			tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
+			cp "t/test-results/${TEST_OUT}" t/failed-test-artifacts/
+			(
+				cd t &&
+				tar czf failed-test-artifacts/"$TEST_NAME".trash.tar.gz "$trash_dir"
+			)
 			;;
 		esac
 	fi
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 02/14] CI: don't include "test-results/" in ci/print-test-failures.sh output

[Ævar Arnfjörð Bjarmason]

Change the ci/print-test-failures.sh output introduced in
657343a602e (travis-ci: move Travis CI code into dedicated scripts,
2017-09-10) to stop including "test-results/" in the heading, it was
more of an artifact of how the iteration was being done in that
initial version than anything else.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/print-test-failures.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index f6736f424e0..1249bb0c0cf 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -24,7 +24,7 @@ do
 		TEST_MARKUP="${TEST_NAME}.markup"
 
 		echo "------------------------------------------------------------------------"
-		echo "$(tput setaf 1)test-results/${TEST_OUT}...$(tput sgr0)"
+		echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
 		echo "------------------------------------------------------------------------"
 		cat "t/test-results/${TEST_OUT}"
 
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 03/14] CI: stop setting FAILED_TEST_ARTIFACTS N times

[Ævar Arnfjörð Bjarmason]

The FAILED_TEST_ARTIFACTS variable introduced in f72f328bc57 (ci: let
GitHub Actions upload failed tests' directories, 2020-04-11) is only
used once in the main.yml, there is no need to set it N times if we
have N failing tests.

See also 92bf1b60673 (ci: avoid `set-env` construct in
print-test-failures.sh, 2020-11-17) which moved it to using
$GITHUB_ENV, we'll now check if it's set like ci/lib.sh, this makes it
easier to test this script locally.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/print-test-failures.sh | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 1249bb0c0cf..8d4d2225371 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -32,7 +32,6 @@ do
 		case "$CI_TYPE" in
 		github-actions)
 			mkdir -p t/failed-test-artifacts
-			echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
 			cp "t/test-results/${TEST_OUT}" t/failed-test-artifacts/
 			(
 				cd t &&
@@ -42,3 +41,11 @@ do
 		esac
 	fi
 done
+
+if test -n "$failed"
+then
+	if test -n "$GITHUB_ENV"
+	then
+		echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
+	fi
+fi
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 04/14] tests: refactor --write-junit-xml code

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The code writing JUnit XML is interspersed directly with all the code in
`t/test-lib.sh`, and it is therefore not only ill-separated, but
introducing yet another output format would make the situation even
worse.

Let's introduce an abstraction layer by hiding the JUnit XML code behind
four new functions that are supposed to be called before and after each
test and test case.

This is not just an academic exercise, refactoring for refactoring's
sake. We _actually_ want to introduce such a new output format, to
make it substantially easier to diagnose test failures in our GitHub
workflow, therefore we do need this refactoring.

This commit is best viewed with `git show --color-moved
--color-moved-ws=allow-indentation-change <commit>`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib-junit.sh | 126 ++++++++++++++++++++++++++++++++++++++++++++
 t/test-lib.sh       | 124 ++++++-------------------------------------
 2 files changed, 142 insertions(+), 108 deletions(-)
 create mode 100644 t/test-lib-junit.sh

diff --git a/t/test-lib-junit.sh b/t/test-lib-junit.sh
new file mode 100644
index 00000000000..9d55d74d764
--- /dev/null
+++ b/t/test-lib-junit.sh
@@ -0,0 +1,126 @@
+# Library of functions to format test scripts' output in JUnit XML
+# format, to support Git's test suite result to be presented in an
+# easily digestible way on Azure Pipelines.
+#
+# Copyright (c) 2022 Johannes Schindelin
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see http://www.gnu.org/licenses/ .
+#
+# The idea is for `test-lib.sh` to source this file when the user asks
+# for JUnit XML; these functions will then override (empty) functions
+# that are are called at the appropriate times during the test runs.
+
+start_test_output () {
+	junit_xml_dir="$TEST_OUTPUT_DIRECTORY/out"
+	mkdir -p "$junit_xml_dir"
+	junit_xml_base=${1##*/}
+	junit_xml_path="$junit_xml_dir/TEST-${junit_xml_base%.sh}.xml"
+	junit_attrs="name=\"${junit_xml_base%.sh}\""
+	junit_attrs="$junit_attrs timestamp=\"$(TZ=UTC \
+		date +%Y-%m-%dT%H:%M:%S)\""
+	write_junit_xml --truncate "<testsuites>" "  <testsuite $junit_attrs>"
+	junit_suite_start=$(test-tool date getnanos)
+	if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
+	then
+		GIT_TEST_TEE_OFFSET=0
+	fi
+}
+
+start_test_case_output () {
+	junit_start=$(test-tool date getnanos)
+}
+
+finalize_test_case_output () {
+	test_case_result=$1
+	shift
+	case "$test_case_result" in
+	ok)
+		set "$*"
+		;;
+	failure)
+		junit_insert="<failure message=\"not ok $test_count -"
+		junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
+		junit_insert="$junit_insert $(xml_attr_encode \
+			"$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
+			   then
+				test-tool path-utils skip-n-bytes \
+					"$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET
+			   else
+				printf '%s\n' "$@" | sed 1d
+			   fi)")"
+		junit_insert="$junit_insert</failure>"
+		if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
+		then
+			junit_insert="$junit_insert<system-err>$(xml_attr_encode \
+				"$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")</system-err>"
+		fi
+		set "$1" "      $junit_insert"
+		;;
+	fixed)
+		set "$* (breakage fixed)"
+		;;
+	broken)
+		set "$* (known breakage)"
+		;;
+	skip)
+		message="$(xml_attr_encode "$skipped_reason")"
+		set "$1" "      <skipped message=\"$message\" />"
+		;;
+	esac
+
+	junit_attrs="name=\"$(xml_attr_encode "$this_test.$test_count $1")\""
+	shift
+	junit_attrs="$junit_attrs classname=\"$this_test\""
+	junit_attrs="$junit_attrs time=\"$(test-tool \
+		date getnanos $junit_start)\""
+	write_junit_xml "$(printf '%s\n' \
+		"    <testcase $junit_attrs>" "$@" "    </testcase>")"
+	junit_have_testcase=t
+}
+
+finalize_test_output () {
+	if test -n "$junit_xml_path"
+	then
+		test -n "$junit_have_testcase" || {
+			junit_start=$(test-tool date getnanos)
+			write_junit_xml_testcase "all tests skipped"
+		}
+
+		# adjust the overall time
+		junit_time=$(test-tool date getnanos $junit_suite_start)
+		sed -e "s/\(<testsuite.*\) time=\"[^\"]*\"/\1/" \
+			-e "s/<testsuite [^>]*/& time=\"$junit_time\"/" \
+			-e '/^ *<\/testsuite/d' \
+			<"$junit_xml_path" >"$junit_xml_path.new"
+		mv "$junit_xml_path.new" "$junit_xml_path"
+
+		write_junit_xml "  </testsuite>" "</testsuites>"
+		write_junit_xml=
+	fi
+}
+
+write_junit_xml () {
+	case "$1" in
+	--truncate)
+		>"$junit_xml_path"
+		junit_have_testcase=
+		shift
+		;;
+	esac
+	printf '%s\n' "$@" >>"$junit_xml_path"
+}
+
+xml_attr_encode () {
+	printf '%s\n' "$@" | test-tool xml-encode
+}
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 8ba5ca15345..27e17c569d4 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -137,6 +137,12 @@ mark_option_requires_arg () {
 	store_arg_to=$2
 }
 
+# These functions can be overridden e.g. to output JUnit XML
+start_test_output () { :; }
+start_test_case_output () { :; }
+finalize_test_case_output () { :; }
+finalize_test_output () { :; }
+
 parse_option () {
 	local opt="$1"
 
@@ -196,7 +202,7 @@ parse_option () {
 		tee=t
 		;;
 	--write-junit-xml)
-		write_junit_xml=t
+		. "$TEST_DIRECTORY/test-lib-junit.sh"
 		;;
 	--stress)
 		stress=t ;;
@@ -664,7 +670,7 @@ exec 6<&0
 exec 7>&2
 
 _error_exit () {
-	finalize_junit_xml
+	finalize_test_output
 	GIT_EXIT_OK=t
 	exit 1
 }
@@ -774,35 +780,13 @@ trap '{ code=$?; set +x; } 2>/dev/null; exit $code' INT TERM HUP
 # the test_expect_* functions instead.
 
 test_ok_ () {
-	if test -n "$write_junit_xml"
-	then
-		write_junit_xml_testcase "$*"
-	fi
+	finalize_test_case_output ok "$@"
 	test_success=$(($test_success + 1))
 	say_color "" "ok $test_count - $@"
 }
 
 test_failure_ () {
-	if test -n "$write_junit_xml"
-	then
-		junit_insert="<failure message=\"not ok $test_count -"
-		junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
-		junit_insert="$junit_insert $(xml_attr_encode \
-			"$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
-			   then
-				test-tool path-utils skip-n-bytes \
-					"$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET
-			   else
-				printf '%s\n' "$@" | sed 1d
-			   fi)")"
-		junit_insert="$junit_insert</failure>"
-		if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
-		then
-			junit_insert="$junit_insert<system-err>$(xml_attr_encode \
-				"$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")</system-err>"
-		fi
-		write_junit_xml_testcase "$1" "      $junit_insert"
-	fi
+	finalize_test_case_output failure "$@"
 	test_failure=$(($test_failure + 1))
 	say_color error "not ok $test_count - $1"
 	shift
@@ -815,19 +799,13 @@ test_failure_ () {
 }
 
 test_known_broken_ok_ () {
-	if test -n "$write_junit_xml"
-	then
-		write_junit_xml_testcase "$* (breakage fixed)"
-	fi
+	finalize_test_case_output fixed "$@"
 	test_fixed=$(($test_fixed+1))
 	say_color error "ok $test_count - $@ # TODO known breakage vanished"
 }
 
 test_known_broken_failure_ () {
-	if test -n "$write_junit_xml"
-	then
-		write_junit_xml_testcase "$* (known breakage)"
-	fi
+	finalize_test_case_output broken "$@"
 	test_broken=$(($test_broken+1))
 	say_color warn "not ok $test_count - $@ # TODO known breakage"
 }
@@ -1104,10 +1082,7 @@ test_start_ () {
 	test_count=$(($test_count+1))
 	maybe_setup_verbose
 	maybe_setup_valgrind
-	if test -n "$write_junit_xml"
-	then
-		junit_start=$(test-tool date getnanos)
-	fi
+	start_test_case_output
 }
 
 test_finish_ () {
@@ -1158,12 +1133,7 @@ test_skip () {
 
 	case "$to_skip" in
 	t)
-		if test -n "$write_junit_xml"
-		then
-			message="$(xml_attr_encode "$skipped_reason")"
-			write_junit_xml_testcase "$1" \
-				"      <skipped message=\"$message\" />"
-		fi
+		finalize_test_case_output skip "$@"
 
 		say_color skip "ok $test_count # skip $1 ($skipped_reason)"
 		: true
@@ -1179,53 +1149,6 @@ test_at_end_hook_ () {
 	:
 }
 
-write_junit_xml () {
-	case "$1" in
-	--truncate)
-		>"$junit_xml_path"
-		junit_have_testcase=
-		shift
-		;;
-	esac
-	printf '%s\n' "$@" >>"$junit_xml_path"
-}
-
-xml_attr_encode () {
-	printf '%s\n' "$@" | test-tool xml-encode
-}
-
-write_junit_xml_testcase () {
-	junit_attrs="name=\"$(xml_attr_encode "$this_test.$test_count $1")\""
-	shift
-	junit_attrs="$junit_attrs classname=\"$this_test\""
-	junit_attrs="$junit_attrs time=\"$(test-tool \
-		date getnanos $junit_start)\""
-	write_junit_xml "$(printf '%s\n' \
-		"    <testcase $junit_attrs>" "$@" "    </testcase>")"
-	junit_have_testcase=t
-}
-
-finalize_junit_xml () {
-	if test -n "$write_junit_xml" && test -n "$junit_xml_path"
-	then
-		test -n "$junit_have_testcase" || {
-			junit_start=$(test-tool date getnanos)
-			write_junit_xml_testcase "all tests skipped"
-		}
-
-		# adjust the overall time
-		junit_time=$(test-tool date getnanos $junit_suite_start)
-		sed -e "s/\(<testsuite.*\) time=\"[^\"]*\"/\1/" \
-			-e "s/<testsuite [^>]*/& time=\"$junit_time\"/" \
-			-e '/^ *<\/testsuite/d' \
-			<"$junit_xml_path" >"$junit_xml_path.new"
-		mv "$junit_xml_path.new" "$junit_xml_path"
-
-		write_junit_xml "  </testsuite>" "</testsuites>"
-		write_junit_xml=
-	fi
-}
-
 test_atexit_cleanup=:
 test_atexit_handler () {
 	# In a succeeding test script 'test_atexit_handler' is invoked
@@ -1248,7 +1171,7 @@ test_done () {
 	# removed, so the commands can access pidfiles and socket files.
 	test_atexit_handler
 
-	finalize_junit_xml
+	finalize_test_output
 
 	if test -z "$HARNESS_ACTIVE"
 	then
@@ -1539,22 +1462,7 @@ fi
 # in subprocesses like git equals our $PWD (for pathname comparisons).
 cd -P "$TRASH_DIRECTORY" || exit 1
 
-if test -n "$write_junit_xml"
-then
-	junit_xml_dir="$TEST_OUTPUT_DIRECTORY/out"
-	mkdir -p "$junit_xml_dir"
-	junit_xml_base=${0##*/}
-	junit_xml_path="$junit_xml_dir/TEST-${junit_xml_base%.sh}.xml"
-	junit_attrs="name=\"${junit_xml_base%.sh}\""
-	junit_attrs="$junit_attrs timestamp=\"$(TZ=UTC \
-		date +%Y-%m-%dT%H:%M:%S)\""
-	write_junit_xml --truncate "<testsuites>" "  <testsuite $junit_attrs>"
-	junit_suite_start=$(test-tool date getnanos)
-	if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
-	then
-		GIT_TEST_TEE_OFFSET=0
-	fi
-fi
+start_test_output "$0"
 
 # Convenience
 # A regexp to match 5 and 35 hexdigits
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 05/14] test(junit): avoid line feeds in XML attributes

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

In the test case's output, we do want newline characters, but in the XML
attributes we do not want them.

However, the `xml_attr_encode` function always adds a Line Feed at the
end (which are then encoded as `&#x0a;`, even for XML attributes.

This seems not to faze Azure Pipelines' XML parser, but it still is
incorrect, so let's fix it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib-junit.sh | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/t/test-lib-junit.sh b/t/test-lib-junit.sh
index 9d55d74d764..c959183c7e2 100644
--- a/t/test-lib-junit.sh
+++ b/t/test-lib-junit.sh
@@ -50,7 +50,7 @@ finalize_test_case_output () {
 		;;
 	failure)
 		junit_insert="<failure message=\"not ok $test_count -"
-		junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
+		junit_insert="$junit_insert $(xml_attr_encode --no-lf "$1")\">"
 		junit_insert="$junit_insert $(xml_attr_encode \
 			"$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
 			   then
@@ -74,12 +74,12 @@ finalize_test_case_output () {
 		set "$* (known breakage)"
 		;;
 	skip)
-		message="$(xml_attr_encode "$skipped_reason")"
+		message="$(xml_attr_encode --no-lf "$skipped_reason")"
 		set "$1" "      <skipped message=\"$message\" />"
 		;;
 	esac
 
-	junit_attrs="name=\"$(xml_attr_encode "$this_test.$test_count $1")\""
+	junit_attrs="name=\"$(xml_attr_encode --no-lf "$this_test.$test_count $1")\""
 	shift
 	junit_attrs="$junit_attrs classname=\"$this_test\""
 	junit_attrs="$junit_attrs time=\"$(test-tool \
@@ -122,5 +122,11 @@ write_junit_xml () {
 }
 
 xml_attr_encode () {
-	printf '%s\n' "$@" | test-tool xml-encode
+	if test "x$1" = "x--no-lf"
+	then
+		shift
+		printf '%s' "$*" | test-tool xml-encode
+	else
+		printf '%s\n' "$@" | test-tool xml-encode
+	fi
 }
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 06/14] ci: optionally mark up output in the GitHub workflow

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

A couple of commands exist to spruce up the output in GitHub workflows:
https://docs.github.com/en/actions/learn-github-actions/workflow-commands-for-github-actions

In addition to the `::group::<label>`/`::endgroup::` commands (which we
already use to structure the output of the build step better), we also
use `::error::`/`::notice::` to draw the attention to test failures and
to test cases that were expected to fail but didn't.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib-functions.sh              |  4 +--
 t/test-lib-github-workflow-markup.sh | 50 ++++++++++++++++++++++++++++
 t/test-lib.sh                        |  5 ++-
 3 files changed, 56 insertions(+), 3 deletions(-)
 create mode 100644 t/test-lib-github-workflow-markup.sh

diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index 93c03380d44..af4831a54c6 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -795,7 +795,7 @@ test_verify_prereq () {
 }
 
 test_expect_failure () {
-	test_start_
+	test_start_ "$@"
 	test "$#" = 3 && { test_prereq=$1; shift; } || test_prereq=
 	test "$#" = 2 ||
 	BUG "not 2 or 3 parameters to test-expect-failure"
@@ -815,7 +815,7 @@ test_expect_failure () {
 }
 
 test_expect_success () {
-	test_start_
+	test_start_ "$@"
 	test "$#" = 3 && { test_prereq=$1; shift; } || test_prereq=
 	test "$#" = 2 ||
 	BUG "not 2 or 3 parameters to test-expect-success"
diff --git a/t/test-lib-github-workflow-markup.sh b/t/test-lib-github-workflow-markup.sh
new file mode 100644
index 00000000000..d8dc969df4a
--- /dev/null
+++ b/t/test-lib-github-workflow-markup.sh
@@ -0,0 +1,50 @@
+# Library of functions to mark up test scripts' output suitable for
+# pretty-printing it in GitHub workflows.
+#
+# Copyright (c) 2022 Johannes Schindelin
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see http://www.gnu.org/licenses/ .
+#
+# The idea is for `test-lib.sh` to source this file when run in GitHub
+# workflows; these functions will then override (empty) functions
+# that are are called at the appropriate times during the test runs.
+
+start_test_output () {
+	test -n "$GIT_TEST_TEE_OUTPUT_FILE" ||
+	die "--github-workflow-markup requires --verbose-log"
+	github_markup_output="${GIT_TEST_TEE_OUTPUT_FILE%.out}.markup"
+	>$github_markup_output
+	GIT_TEST_TEE_OFFSET=0
+}
+
+# No need to override start_test_case_output
+
+finalize_test_case_output () {
+	test_case_result=$1
+	shift
+	case "$test_case_result" in
+	failure)
+		echo >>$github_markup_output "::error::failed: $this_test.$test_count $1"
+		;;
+	fixed)
+		echo >>$github_markup_output "::notice::fixed: $this_test.$test_count $1"
+		;;
+	esac
+	echo >>$github_markup_output "::group::$test_case_result: $this_test.$test_count $*"
+	test-tool >>$github_markup_output path-utils skip-n-bytes \
+		"$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET
+	echo >>$github_markup_output "::endgroup::"
+}
+
+# No need to override finalize_test_output
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 27e17c569d4..0a7e067ebdb 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -204,6 +204,9 @@ parse_option () {
 	--write-junit-xml)
 		. "$TEST_DIRECTORY/test-lib-junit.sh"
 		;;
+	--github-workflow-markup)
+		. "$TEST_DIRECTORY/test-lib-github-workflow-markup.sh"
+		;;
 	--stress)
 		stress=t ;;
 	--stress=*)
@@ -1082,7 +1085,7 @@ test_start_ () {
 	test_count=$(($test_count+1))
 	maybe_setup_verbose
 	maybe_setup_valgrind
-	start_test_case_output
+	start_test_case_output "$@"
 }
 
 test_finish_ () {
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 07/14] ci: make it easier to find failed tests' logs in the GitHub workflow

[Ævar Arnfjörð Bjarmason]

When investigating a test failure, the time that matters most is the
time it takes from getting aware of the failure to displaying the output
of the failing test case.

You currently have to know a lot of implementation details when
investigating test failures in the CI runs. The first step is easy: the
failed job is marked quite clearly, but when opening it, the failed step
is expanded, which in our case is the one running
`make` or 'make test`. When failing in the latter we only show a
high-level view of what went wrong: it prints the output of `prove`
which merely tells the reader which test script failed, along with the failed
test numbers.

To diagnose those failures in detail the user has then had to know
that they should expand the subsequent "Run ci/print-test-failures.sh"
step. Because that step always exits successfully it's not expanded by
default (the GitHub CI only expands the last failing step), making it
easy to miss that there is useful information hidden in there.

This change does a couple of things to improve the situation:

 * We now make use of the "exit code" feature of
   ci/print-test-failures.sh added in a preceding commit, thus if "make
   test" fails we'll pick up that we have failing tests, and fail in
   ci/print-test-failures.sh too.

   Note that we still exit successfully from ci/print-test-failures.sh
   if we don't have logs of any failed tests in t/test-results. Thus
   if we fail in "make" (i.e. when building), or even in "make test"
   in a way that doesn't yield a failed test (e.g. the t/Makefile
   itself failing) we'll correctly "zoom in" on the (hopefully
   failing) "make test" instead.

   As before if there was no previously failed step the
   ci/print-test-failures.sh step is skipped entirely (see
   .github/workflows/main.yml)

   To make the script DWYM the ci/print-test-failures.sh defaults to
   an "auto" mode. Thus since we turn on --github-workflow-markup in
   ci/lib.sh we'll write *.markup files, and will pick up on that in
   ci/print-test-failures.sh.

   The "$github_workflow_markup" being "auto" (or "t") on failure is
   also what makes us exit non-zero. We could also add
   --[no-]exit-code arguments, but at this point we only need this for
   the "*.markup" mode, so let's leave that for now.

   By having this be "auto" well pick up on whatever the GIT_TEST_OPTS
   setting is, which as we'll see in a subsequent commit will be
   useful if we'd like to configure the CI to emit it only some of the
   time.

 * The ci/print-test-failures.sh now makes use of the GitHub markdown
   introduced for t/test-lib.sh in the preceding commit. I.e. in GitHub
   CI we'll now run with --github-workflow-markup in GIT_TEST_OPTS,
   which causes our "--verbose" output in
   "t/test-results/$TEST_OUT.markup" to be in the GitHub CI-specific
   Markdown format.

The new output is easier to read since it makes use of collapsible grouping.
Initially, readers will see the high-level view of what actually happened.
To drill down, the "group" for the specific failing test(s) can be expanded.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                 |  2 +-
 ci/print-test-failures.sh | 62 ++++++++++++++++++++++++++++++++++++---
 2 files changed, 59 insertions(+), 5 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 80e89f89b7f..422399b1305 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -189,7 +189,7 @@ MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
 case "$CI_TYPE" in
 github-actions)
 	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
-	GIT_TEST_OPTS="--verbose-log -x"
+	GIT_TEST_OPTS="--verbose-log -x --github-workflow-markup"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 8d4d2225371..f45e36bdefc 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -8,25 +8,70 @@ set -e
 . ${0%/*}/lib-ci-type.sh
 . ${0%/*}/lib-tput.sh
 
+github_workflow_markup=auto
+case "$CI_TYPE" in
+github-actions)
+	github_workflow_markup=t
+	;;
+esac
+
+while test $# != 0
+do
+	case "$1" in
+	--github-workflow-markup)
+		github_workflow_markup=t
+		;;
+	--no-github-workflow-markup)
+		github_workflow_markup=
+		;;
+	*)
+		echo "BUG: invalid $0 argument: $1" >&2
+		exit 1
+		;;
+	esac
+	shift
+done
+
 if ! ls t/test-results/*.exit >/dev/null 2>/dev/null
 then
 	echo "Build job failed before the tests could have been run"
 	exit
 fi
 
+failed=
 for TEST_EXIT in t/test-results/*.exit
 do
 	if [ "$(cat "$TEST_EXIT")" != "0" ]
 	then
+		failed=t
 		TEST_NAME="${TEST_EXIT%.exit}"
 		TEST_NAME="${TEST_NAME##*/}"
 		TEST_OUT="${TEST_NAME}.out"
 		TEST_MARKUP="${TEST_NAME}.markup"
 
-		echo "------------------------------------------------------------------------"
-		echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
-		echo "------------------------------------------------------------------------"
-		cat "t/test-results/${TEST_OUT}"
+		do_markup=
+		case "$github_workflow_markup" in
+		t)
+			do_markup=t
+			;;
+		auto)
+			if test -f "t/test-results/$TEST_MARKUP"
+			then
+				do_markup=t
+			fi
+			;;
+		esac
+
+		if test -n "$do_markup"
+		then
+			printf "\\e[33m\\e[1m=== Failed test: ${TEST_NAME} ===\\e[m\\n"
+			cat "t/test-results/$TEST_MARKUP"
+		else
+			echo "------------------------------------------------------------------------"
+			echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
+			echo "------------------------------------------------------------------------"
+			cat "t/test-results/${TEST_OUT}"
+		fi
 
 		trash_dir="trash directory.$TEST_NAME"
 		case "$CI_TYPE" in
@@ -48,4 +93,13 @@ then
 	then
 		echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
 	fi
+
+	case "$github_workflow_markup" in
+	t|auto)
+		exit 1
+		;;
+	'')
+		exit 0
+		;;
+	esac
 fi
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 08/14] ci(github): skip the logs of the successful test cases

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

In most instances, looking at the log of failed test cases is enough to
identify the problem.

In some (rare?) instances, a previous test case that was marked as
successful actually has information pertaining to a later test case that
fails.

In some (rare?) instances, a previous test case that was marked as
successful actually has information pertaining to a later test case that
fails.

To allow the page to load relatively quickly, let's only show the logs
of the failed test cases to be shown. The full logs are available for
download as artifacts, should a deeper investigation become necessary.

Co-authored-by: Victoria Dye <vdye@github.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib-github-workflow-markup.sh | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/t/test-lib-github-workflow-markup.sh b/t/test-lib-github-workflow-markup.sh
index d8dc969df4a..1ef0fd5ba87 100644
--- a/t/test-lib-github-workflow-markup.sh
+++ b/t/test-lib-github-workflow-markup.sh
@@ -40,6 +40,10 @@ finalize_test_case_output () {
 	fixed)
 		echo >>$github_markup_output "::notice::fixed: $this_test.$test_count $1"
 		;;
+	ok)
+		# Exit without printing the "ok" tests
+		return
+		;;
 	esac
 	echo >>$github_markup_output "::group::$test_case_result: $this_test.$test_count $*"
 	test-tool >>$github_markup_output path-utils skip-n-bytes \
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 09/14] ci(github): skip "skip" tests in --github-workflow-markup

[Ævar Arnfjörð Bjarmason]

As in the preceding commit's change to "ok" tests omit the "skip"
tests in the --github-workflow-markup. Before this we'd emit this for
e.g. t0026-eol-config.sh:

	$ rm -rf test-results; ./t0026-eol-config.sh --verbose-log -x --github-workflow-markup; echo; cat test-results/*markup
	ok 1 - setup
	ok 2 - eol=lf puts LFs in normalized file
	ok 3 - eol=crlf puts CRLFs in normalized file
	ok 4 - autocrlf=true overrides eol=lf
	ok 5 - autocrlf=true overrides unset eol
	ok 6 # skip eol native is crlf (missing NATIVE_CRLF)
	# passed all 6 test(s)
	1..6

	::group::skip: t0026.6 eol native is crlf

	        rm -rf native_eol && mkdir native_eol &&
	        (
	                cd native_eol &&
	                printf "*.txt text
	" >.gitattributes &&
	                printf "one
	two
	three
	" >filedos.txt &&
	                printf "one
	two
	three
	" >fileunix.txt &&
	                git init &&
	                git config core.autocrlf false &&
	                git config core.eol native &&
	                git add filedos.txt fileunix.txt &&
	                git commit -m "first" &&
	                rm file*.txt &&
	                git reset --hard HEAD &&
	                has_cr filedos.txt &&
	                has_cr fileunix.txt
	        )

	::endgroup::

Now we'll instead emit this, i.e. an empty *.markup file:

	$ rm -rf test-results; ./t0026-eol-config.sh --verbose-log -x --github-workflow-markup; echo; cat test-results/*markup
	ok 1 - setup
	ok 2 - eol=lf puts LFs in normalized file
	ok 3 - eol=crlf puts CRLFs in normalized file
	ok 4 - autocrlf=true overrides eol=lf
	ok 5 - autocrlf=true overrides unset eol
	ok 6 # skip eol native is crlf (missing NATIVE_CRLF)
	# passed all 6 test(s)
	1..6

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib-github-workflow-markup.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/t/test-lib-github-workflow-markup.sh b/t/test-lib-github-workflow-markup.sh
index 1ef0fd5ba87..bad3ebafdb7 100644
--- a/t/test-lib-github-workflow-markup.sh
+++ b/t/test-lib-github-workflow-markup.sh
@@ -40,8 +40,8 @@ finalize_test_case_output () {
 	fixed)
 		echo >>$github_markup_output "::notice::fixed: $this_test.$test_count $1"
 		;;
-	ok)
-		# Exit without printing the "ok" tests
+	ok|skip)
+		# Exit without printing the "ok" or "skip" tests
 		return
 		;;
 	esac
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 10/14] ci(github): avoid printing test case preamble twice

[Ævar Arnfjörð Bjarmason]

From: Victoria Dye <vdye@github.com>

We want to mark up the test case preamble when presenting test output in
Git's GitHub workflow. Let's suppress the non-marked-up version in that
case. Any information it would contain is included in the marked-up
variant already.

This change has the side-effect of not making the full logs available
to us in the *.out files anymore. E.g. in the case of running a test
where all tests are successful, we ran run this sequence of commands
in t/:

	test=t0013-sha1dc.sh;
	rm -rf test-results*;
	./$test --verbose-log -x;
	mv test-results{,.old};
	./$test --verbose-log -x --github-workflow-markup;
	diff -ru test-results{.old,}; file test-results/*markup

Which will emit:

	ok 1 - test-sha1 detects shattered pdf
	# passed all 1 test(s)
	1..1
	ok 1 - test-sha1 detects shattered pdf
	# passed all 1 test(s)
	1..1
	Only in test-results: t0013-sha1dc.markup
	diff -ru test-results.old/t0013-sha1dc.out test-results/t0013-sha1dc.out
	--- test-results.old/t0013-sha1dc.out   2022-05-24 19:12:14.106339385 +0200
	+++ test-results/t0013-sha1dc.out       2022-05-24 19:12:14.154338965 +0200
	@@ -1,9 +1,4 @@
	 Initialized empty Git repository in /home/avar/g/git/t/trash directory.t0013-sha1dc/.git/
	-expecting success of 0013.1 'test-sha1 detects shattered pdf':
	-       test_must_fail test-tool sha1 <"$TEST_DATA/shattered-1.pdf" 2>err &&
	-       test_i18ngrep collision err &&
	-       grep 38762cf7f55934b34d179ae6a4c80cadccbb7f0a err
	-
	 + test_must_fail test-tool sha1
	 + _test_ok=
	 + test_must_fail_acceptable test-tool sha1
	test-results/t0013-sha1dc.markup: empty

This is because the the "test_skip_test_preamble" variable being set
here has a global effect. Perhaps it's acceptable to munge the *.out
output for now, as this will only kick in under
--github-workflow-markup, but ideally we'd implement
--github-workflow-markup in such a way that it doesn't change the
normal --verbose-log output.

Because of this change it's now harder to make sense of the
--verbose-log output when viewing the full log output, as some of the
context for test boundaries will be missing. Note that we shill have
the "ok" or "not ok" TAP lines, so we can still infer the test
boundaries. The output that suffers the most from this is the
"--verbose-log -x" output, i.e. we won't have the test source to go
along with the "-x" output.

Signed-off-by: Victoria Dye <vdye@github.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Tested-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib-functions.sh              | 2 ++
 t/test-lib-github-workflow-markup.sh | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index af4831a54c6..89a5e146b7a 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -803,6 +803,7 @@ test_expect_failure () {
 	export test_prereq
 	if ! test_skip "$@"
 	then
+		test -n "$test_skip_test_preamble" ||
 		say >&3 "checking known breakage of $TEST_NUMBER.$test_count '$1': $2"
 		if test_run_ "$2" expecting_failure
 		then
@@ -823,6 +824,7 @@ test_expect_success () {
 	export test_prereq
 	if ! test_skip "$@"
 	then
+		test -n "$test_skip_test_preamble" ||
 		say >&3 "expecting success of $TEST_NUMBER.$test_count '$1': $2"
 		if test_run_ "$2"
 		then
diff --git a/t/test-lib-github-workflow-markup.sh b/t/test-lib-github-workflow-markup.sh
index bad3ebafdb7..2267fba947c 100644
--- a/t/test-lib-github-workflow-markup.sh
+++ b/t/test-lib-github-workflow-markup.sh
@@ -20,6 +20,8 @@
 # workflows; these functions will then override (empty) functions
 # that are are called at the appropriate times during the test runs.
 
+test_skip_test_preamble=t
+
 start_test_output () {
 	test -n "$GIT_TEST_TEE_OUTPUT_FILE" ||
 	die "--github-workflow-markup requires --verbose-log"
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 11/14] ci(github): mention where the full logs can be found

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The full logs are contained in the `failed-tests-*.zip` artifacts that
are attached to the failed CI run. Since this is not immediately
obvious to the well-disposed reader, let's mention it explicitly.

Suggested-by: Victoria Dye <vdye@github.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/print-test-failures.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index f45e36bdefc..72fae2b0cc4 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -65,6 +65,7 @@ do
 		if test -n "$do_markup"
 		then
 			printf "\\e[33m\\e[1m=== Failed test: ${TEST_NAME} ===\\e[m\\n"
+			echo "The full logs are in the artifacts attached to this run."
 			cat "t/test-results/$TEST_MARKUP"
 		else
 			echo "------------------------------------------------------------------------"
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 12/14] ci: call `finalize_test_case_output` a little later

[Ævar Arnfjörð Bjarmason]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

We used to call that function already before printing the final verdict.
However, now that we added grouping to the GitHub workflow output, we
will want to include even that part in the collapsible group for that
test case.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib.sh | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/t/test-lib.sh b/t/test-lib.sh
index 0a7e067ebdb..736c6447ecf 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -783,13 +783,13 @@ trap '{ code=$?; set +x; } 2>/dev/null; exit $code' INT TERM HUP
 # the test_expect_* functions instead.
 
 test_ok_ () {
-	finalize_test_case_output ok "$@"
 	test_success=$(($test_success + 1))
 	say_color "" "ok $test_count - $@"
+	finalize_test_case_output ok "$@"
 }
 
 test_failure_ () {
-	finalize_test_case_output failure "$@"
+	failure_label=$1
 	test_failure=$(($test_failure + 1))
 	say_color error "not ok $test_count - $1"
 	shift
@@ -799,18 +799,19 @@ test_failure_ () {
 		say_color error "1..$test_count"
 		_error_exit
 	fi
+	finalize_test_case_output failure "$failure_label" "$@"
 }
 
 test_known_broken_ok_ () {
-	finalize_test_case_output fixed "$@"
 	test_fixed=$(($test_fixed+1))
 	say_color error "ok $test_count - $@ # TODO known breakage vanished"
+	finalize_test_case_output fixed "$@"
 }
 
 test_known_broken_failure_ () {
-	finalize_test_case_output broken "$@"
 	test_broken=$(($test_broken+1))
 	say_color warn "not ok $test_count - $@ # TODO known breakage"
+	finalize_test_case_output broken "$@"
 }
 
 test_debug () {
@@ -1136,10 +1137,10 @@ test_skip () {
 
 	case "$to_skip" in
 	t)
-		finalize_test_case_output skip "$@"
 
 		say_color skip "ok $test_count # skip $1 ($skipped_reason)"
 		: true
+		finalize_test_case_output skip "$@"
 		;;
 	*)
 		false
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 13/14] CI: make --github-workflow-markup ci-config, off by default

[Ævar Arnfjörð Bjarmason]

In preceding commits the --github-workflow-markup output was made the
default under GitHub CI.

There's a few outstanding issues with that new output target however:

A. As noted in [1] it's much slower in some common cases, i.e. the
   page rendering time (under Firefox & Chrome debugging) goes from ~20s
   to on the order of 60-80s (less pronounced when not debugging, but
   still in that ballpark). See also [2] for tests on an earlier iteration
   of the series.

B. Preceding commits in this series to omit non-failing test output
   may have mitigated that somewhat, i.e. skipping "ok" and "skip" tests
   in the *.markup files (although [1] is a benchmark with that "ok"
   change).

   Per [3] there are some CI users that prefer to see only the full
   verbose output, i.e. we can't in the general case assume that we
   can emit only the trace output for the failing test, and that's
   going to give us the information we need to debug the test. It
   might be failing because of earlier setup, or from trace output
   outside of any test (i.e. in the main body of the file).

   Users such as [3] could still just consult the *.out output, but
   then we'd make them download the archive instead of viewing the
   output in the web UX.

C. Per B above and [3] an earlier change in this series modified
   the *.out output to change the *.out output inadvertently while trying
   to modify the *.markup output. See the commit message of "ci(github):
   avoid printing test case preamble twice" earlier in this series.

It thus make sense for now to make the "ci/print-test-failures.sh"
output be the "raw" one. I.e. the *.out output we'd get before
--github-workflow-markup was implemented.

To make it use ci/print-test-failures.sh we can create a
"ci/config/print-test-failures-output-type" script on the "ci-config"
branch that does e.g.:

	#!/bin/sh
	echo github

That script will (like the existing "check-ref") get the ref as an
argument, so we can in the future turn this on experimentally for
"seen", and flip the default in the future. The script can also emit
nothing to get whatever our idea is of the default (currently "raw").

Note the corresponding change to ci/print-test-failures.sh, i.e. we'll
do the right thing vis-a-vis what we "zoom in" to depending on the
setting. If it's "raw" we'd like a failed "make test" to "zoom in" to
that step, if it's "github" we'd like the to open the subsequent
"ci/print-test-failures.sh" step instead.

1. https://lore.kernel.org/git/220523.86ee0kzjix.gmgdl@evledraar.gmail.com/
2. https://lore.kernel.org/git/220222.86tucr6kz5.gmgdl@evledraar.gmail.com/
3. https://lore.kernel.org/git/20210309175249.GE3590451@szeder.dev/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 24 ++++++++++++++++++++++++
 ci/lib.sh                  | 14 +++++++++++++-
 ci/print-test-failures.sh  |  7 +++++--
 3 files changed, 42 insertions(+), 3 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 6835e942280..4195f37c1ba 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -8,6 +8,7 @@ jobs:
     runs-on: ubuntu-latest
     outputs:
       enabled: ${{ steps.check-ref.outputs.enabled }}${{ steps.skip-if-redundant.outputs.enabled }}
+      print-test-failures-output-type: ${{ steps.print-test-failures-output-type.outputs.type }}
     steps:
       - name: try to clone ci-config branch
         run: |
@@ -22,6 +23,21 @@ jobs:
             config-repo &&
           cd config-repo &&
           git checkout HEAD -- ci/config || : ignore
+      - id: print-test-failures-output-type
+        name: check what output type ci/print-test-failures.sh uses
+        run: |
+          type_default=raw
+          type=$type_default
+
+          if test -x config-repo/ci/config/print-test-failures-output-type
+          then
+            type=$(config-repo/ci/config/print-test-failures-output-type '${{ github.ref }}')
+            if test -z "$type"
+            then
+              type=$type_default
+            fi
+          fi
+          echo "::set-output name=type::$type"
       - id: check-ref
         name: check whether CI is enabled for ref
         run: |
@@ -124,6 +140,8 @@ jobs:
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
     - run: ci/lib.sh --test
       shell: bash
+      env:
+        GIT_CI_PTF_OUTPUT_TYPE: ${{needs.ci-config.outputs.print-test-failures-output-type}}
     - name: select tests
       run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
       shell: bash
@@ -213,6 +231,8 @@ jobs:
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
     - run: ci/lib.sh --test
       shell: bash
+      env:
+        GIT_CI_PTF_OUTPUT_TYPE: ${{needs.ci-config.outputs.print-test-failures-output-type}}
     - name: select tests
       run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
       shell: bash
@@ -265,6 +285,8 @@ jobs:
     - run: ci/lib.sh --build
     - run: make
     - run: ci/lib.sh --test
+      env:
+        GIT_CI_PTF_OUTPUT_TYPE: ${{needs.ci-config.outputs.print-test-failures-output-type}}
     - run: make test
       if: success()
     - run: ci/print-test-failures.sh
@@ -302,6 +324,8 @@ jobs:
     - run: make
     - run: ci/lib.sh --test
       if: success() && matrix.vector.skip-tests != 'yes'
+      env:
+        GIT_CI_PTF_OUTPUT_TYPE: ${{needs.ci-config.outputs.print-test-failures-output-type}}
     - run: make test
       if: success() && matrix.vector.skip-tests != 'yes'
     - run: ci/print-test-failures.sh
diff --git a/ci/lib.sh b/ci/lib.sh
index 422399b1305..b148d3f209f 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -95,6 +95,7 @@ if test -n "$GITHUB_ENV"
 then
 	echo "CONFIG: GITHUB_ENV=$GITHUB_ENV" >&2
 fi
+echo "CONFIG: GIT_CI_PTF_OUTPUT_TYPE=$GIT_CI_PTF_OUTPUT_TYPE" >&2
 echo >&2
 
 # Helper functions
@@ -189,7 +190,18 @@ MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
 case "$CI_TYPE" in
 github-actions)
 	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
-	GIT_TEST_OPTS="--verbose-log -x --github-workflow-markup"
+	GIT_TEST_OPTS="--verbose-log -x"
+	if test -n "$GIT_CI_PTF_OUTPUT_TYPE"
+	then
+		# For later use in ci/print-test-failures.sh
+		setenv --test GIT_CI_PTF_OUTPUT_TYPE "$GIT_CI_PTF_OUTPUT_TYPE"
+
+		case "$GIT_CI_PTF_OUTPUT_TYPE" in
+		github)
+			GIT_TEST_OPTS="$GIT_TEST_OPTS --github-workflow-markup"
+			;;
+		esac
+	fi
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 72fae2b0cc4..1e0738e0058 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -9,10 +9,13 @@ set -e
 . ${0%/*}/lib-tput.sh
 
 github_workflow_markup=auto
-case "$CI_TYPE" in
-github-actions)
+case "$GIT_CI_PTF_OUTPUT_TYPE" in
+github)
 	github_workflow_markup=t
 	;;
+raw)
+	github_workflow_markup=
+	;;
 esac
 
 while test $# != 0
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 14/14] CI: make the --github-workflow-markup "github" output the default

[Ævar Arnfjörð Bjarmason]

As noted in the preceding commit there's outstanding issues with this
output target over the "raw" one, but let's flip the default so that
the end-state of this series matches that of [1]. If we'd like we can
now easily revert back.

My ci/config/print-test-failures-output-type on my "ci-config" branch
is currently the following script, which allows for naming branches
with _CI_OUT_{GITHUB,RAW} suffixes to force a given output type:

	#!/bin/sh

	set -ex

	ref=$1
	shift
	echo "ref: $ref" >&2
	echo "arguments: $@" >&2

	case "$ref" in
	*_CI_OUT_GITHUB)
		echo github
		;;
	*_CI_OUT_RAW)
		echo raw
		;;
	esac

1. https://lore.kernel.org/git/pull.1117.v3.git.1653171536.gitgitgadget@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 4195f37c1ba..39675cbdd60 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -26,7 +26,7 @@ jobs:
       - id: print-test-failures-output-type
         name: check what output type ci/print-test-failures.sh uses
         run: |
-          type_default=raw
+          type_default=github
           type=$type_default
 
           if test -x config-repo/ci/config/print-test-failures-output-type
-- 
2.36.1.1045.gf356b5617dd

Re: [PATCH v2 00/25] CI: run "make [test]" directly, use $GITHUB_ENV

[Victoria Dye]

Ævar Arnfjörð Bjarmason wrote:
> A re-roll of my improvements my series to simplify the CI setup a lot
> (see diffstat), much of it was dealing with constraints that went away
> with Travis et al. CI for this series (OSX runners failing for
> unrelated reasons):
> 
>     https://github.com/avar/git/actions/runs/2040223909
> 
> For a much more detailed summary of how the output looks before/after
> see v1[].
> 
> This series heavily conflicts with Johannes's
> js/ci-github-workflow-markup in "seen", but in the v1 I suggested
> basing that series on top of this one, because it can benefit a lot
> from these simplifications.
> 
> I'll reply to this series with a proposed rebasing of that series on
> top of this one, which allows for removing almost all of its changes
> to "ci/" with no harm to its end-goals, i.e. the splitting up of
> "make" and "make test" output is something it'll get for free from
> this series.
> 
> Junio: Since that series has been stalled on still-outstanding
> performance issues for a couple of months I was hoping we could queue
> this instead, and perhaps in addition if Johannes approves of the
> proposed re-roll on top of his.
> 
> There's some forward progress on the performance issues (this[2] reply
> of Victoria Dye's from yesterday), but fully resolving those will
> probably take a bit...
> 
> Whereas even though this one is relatively large I don't think there's
> anything controversial here. The one concern that's been raised has
> been Johannes's objection to removing some of the dead Azure code
> (which was needed to move forward here). I asked how he'd prefer to
> move forward with that in [3], but there hasn't been a reply to that
> in >1 month.
> 

While the largeness of a series shouldn't necessarily block it, the lack of
overarching structure or purpose in this one makes it really difficult for
me to review with much confidence (I can't speak for everyone, but it may be
one of the reasons for the general lack of feedback). If you believe all of
these patches as thematically-related enough to warrant being in a single
series, then it would help a lot if you could:

1. Clearly describe the purpose of the series (yes they're all CI
   improvements, but *why* these particular improvements, and why do they
   all need to go together?) 
2. Outline the "path" these commits take to accomplishing that purpose ("The
   first 3 commits do X because Y. Then, the next 4 commits do A because B."
   etc. or whatever format fits your writing style, as long as the
   information is there).
3. Reorganize commits as necessary to keep the above outline from jumping
   back and forth between topics. 

Personally, I think this could (should?) be split into at least two series:
one that breaks up 'run-build-and-tests.sh' (and is more directly relevant
to dscho's series), and one that does the cleanup/flag change/other work.
The two appear to be independent, and the resulting two series would be a
much more manageable 10-15 commits each. 

> I think just removing it is OK, we can always bring it back if needed,
> and doing so is actually going to be simpler on top of this since the
> CI is now less special, and leans more heavily on the logic of our
> normal build process.
> 
> 1. https://lore.kernel.org/git/cover-00.25-00000000000-20220221T143936Z-avarab@gmail.com
> 2. https://lore.kernel.org/git/6b83bb83-32b9-20c9-fa02-c1c3170351c3@github.com/
> 3. https://lore.kernel.org/git/220222.86y2236ndp.gmgdl@evledraar.gmail.com/
> 
> Ævar Arnfjörð Bjarmason (25):
>   CI: run "set -ex" early in ci/lib.sh
>   CI: make "$jobname" explicit, remove fallback
>   CI: remove more dead Travis CI support
>   CI: remove dead "tree skipping" code
>   CI: remove unused Azure ci/* code
>   CI: don't have "git grep" invoke a pager in tree content check
>   CI: have "static-analysis" run a "make ci-static-analysis" target
>   CI: have "static-analysis" run "check-builtins", not "documentation"
>   CI: move p4 and git-lfs variables to ci/install-dependencies.sh
>   CI: consistently use "export" in ci/lib.sh
>   CI: export variables via a wrapper
>   CI: remove "run-build-and-tests.sh", run "make [test]" directly
>   CI: check ignored unignored build artifacts in "win[+VS] build" too
>   CI: invoke "make artifacts-tar" directly in windows-build
>   CI: split up and reduce "ci/test-documentation.sh"
>   CI: combine ci/install{,-docker}-dependencies.sh
>   CI: move "env" definitions into ci/lib.sh
>   ci/run-test-slice.sh: replace shelling out with "echo"
>   CI: pre-select test slice in Windows & VS tests
>   CI: only invoke ci/lib.sh as "steps" in main.yml
>   CI: narrow down variable definitions in --build and --test
>   CI: add more variables to MAKEFLAGS, except under vs-build
>   CI: set CC in MAKEFLAGS directly, don't add it to the environment
>   CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case
>   CI: don't use "set -x" in "ci/lib.sh" output
> 
>  .github/workflows/main.yml            | 100 +++++---
>  Makefile                              |  31 ++-
>  ci/check-directional-formatting.bash  |   2 +-
>  ci/check-unignored-build-artifacts.sh |  20 ++
>  ci/install-dependencies.sh            |  53 ++++-
>  ci/install-docker-dependencies.sh     |  22 --
>  ci/lib-ci-type.sh                     |  10 +
>  ci/lib-tput.sh                        |   2 +
>  ci/lib.sh                             | 316 +++++++++++++-------------
>  ci/make-test-artifacts.sh             |  12 -
>  ci/mount-fileshare.sh                 |  25 --
>  ci/print-test-failures.sh             |  16 +-
>  ci/run-build-and-tests.sh             |  54 -----
>  ci/run-docker-build.sh                |  66 ------
>  ci/run-docker.sh                      |  47 ----
>  ci/run-static-analysis.sh             |  32 ---
>  ci/run-test-slice.sh                  |  17 --
>  ci/select-test-slice.sh               |  10 +
>  ci/test-documentation.sh              |  37 +--
>  ci/util/extract-trash-dirs.sh         |  50 ----
>  shared.mak                            |   1 +
>  21 files changed, 346 insertions(+), 577 deletions(-)
>  create mode 100755 ci/check-unignored-build-artifacts.sh
>  delete mode 100755 ci/install-docker-dependencies.sh
>  create mode 100644 ci/lib-ci-type.sh
>  create mode 100644 ci/lib-tput.sh
>  delete mode 100755 ci/make-test-artifacts.sh
>  delete mode 100755 ci/mount-fileshare.sh
>  delete mode 100755 ci/run-build-and-tests.sh
>  delete mode 100755 ci/run-docker-build.sh
>  delete mode 100755 ci/run-docker.sh
>  delete mode 100755 ci/run-static-analysis.sh
>  delete mode 100755 ci/run-test-slice.sh
>  create mode 100755 ci/select-test-slice.sh
>  delete mode 100755 ci/util/extract-trash-dirs.sh
> 
> Range-diff against v1:
>  1:  970849a227f =  1:  4addbd70213 CI: run "set -ex" early in ci/lib.sh
>  2:  eda7fb18064 =  2:  b23aa99fd37 CI: make "$jobname" explicit, remove fallback
>  3:  3ee32815cf3 =  3:  eec15a95879 CI: remove more dead Travis CI support
>  4:  c81c1b3f504 =  4:  73c894f1665 CI: remove dead "tree skipping" code
>  5:  4738a22a36d =  5:  b5e6d538554 CI: remove unused Azure ci/* code
>  6:  59e4f41e86c =  6:  a4b9febbdca CI: don't have "git grep" invoke a pager in tree content check
>  7:  836ef20fdcc !  7:  5d46d5b34c9 CI: have "static-analysis" run a "make ci-static-analysis" target
>     @@ .github/workflows/main.yml: jobs:
>           if: needs.ci-config.outputs.enabled == 'yes'
>      
>       ## Makefile ##
>     -@@ Makefile: ifndef V
>     - 	QUIET_HDR      = @echo '   ' HDR $(<:hcc=h);
>     - 	QUIET_RC       = @echo '   ' RC $@;
>     - 	QUIET_SPATCH   = @echo '   ' SPATCH $<;
>     -+	QUIET_CHECK    = @echo '   ' CHECK $@;
>     - 	QUIET_SUBDIR0  = +@subdir=
>     - 	QUIET_SUBDIR1  = ;$(NO_SUBDIR) echo '   ' SUBDIR $$subdir; \
>     - 			 $(MAKE) $(PRINT_DIR) -C $$subdir
>      @@ Makefile: coccicheck: $(addsuffix .patch,$(filter-out %.pending.cocci,$(wildcard contrib/c
>       # See contrib/coccinelle/README
>       coccicheck-pending: $(addsuffix .patch,$(wildcard contrib/coccinelle/*.pending.cocci))
>     @@ ci/run-static-analysis.sh (deleted)
>      -
>      -make hdr-check ||
>      -exit 1
>     +
>     + ## shared.mak ##
>     +@@ shared.mak: ifndef V
>     + 	QUIET_HDR      = @echo '   ' HDR $(<:hcc=h);
>     + 	QUIET_RC       = @echo '   ' RC $@;
>     + 	QUIET_SPATCH   = @echo '   ' SPATCH $<;
>     ++	QUIET_CHECK    = @echo '   ' CHECK $@;
>     + 
>     + ## Used in "Documentation/Makefile"
>     + 	QUIET_ASCIIDOC	= @echo '   ' ASCIIDOC $@;
>  8:  95cd496868e =  8:  81e06f13d84 CI: have "static-analysis" run "check-builtins", not "documentation"
>  9:  a1d0796259e =  9:  3be91c26d44 CI: move p4 and git-lfs variables to ci/install-dependencies.sh
> 10:  b6a61a786c5 = 10:  9dc148341ba CI: consistently use "export" in ci/lib.sh
> 11:  f2fcee5d6e4 = 11:  e9c7ba492e8 CI: export variables via a wrapper
> 12:  dfd823f2e7d = 12:  86d5a48d59a CI: remove "run-build-and-tests.sh", run "make [test]" directly
> 13:  46459fff296 = 13:  649ad1ae249 CI: check ignored unignored build artifacts in "win[+VS] build" too
> 14:  aecd3ebaafe = 14:  b1b5b083389 CI: invoke "make artifacts-tar" directly in windows-build
> 15:  4f1564af70f = 15:  dfa91ac8938 CI: split up and reduce "ci/test-documentation.sh"
> 16:  868613a5b06 = 16:  ba4ed216769 CI: combine ci/install{,-docker}-dependencies.sh
> 17:  5d854e8ff36 = 17:  b5e89a33340 CI: move "env" definitions into ci/lib.sh
> 18:  a6106525b7f = 18:  571f4d0f441 ci/run-test-slice.sh: replace shelling out with "echo"
> 19:  e9c6c4dd293 = 19:  2edea06ee4d CI: pre-select test slice in Windows & VS tests
> 20:  40d86e8c1dc = 20:  ef9daa6882f CI: only invoke ci/lib.sh as "steps" in main.yml
> 21:  abf9c504740 = 21:  44e3ace5fbe CI: narrow down variable definitions in --build and --test
> 22:  9f4c2798a82 = 22:  5f56b922e08 CI: add more variables to MAKEFLAGS, except under vs-build
> 23:  8a8b7ecf16b ! 23:  b45b7cec94e CI: stop over-setting the $CC variable
>     @@ Metadata
>      Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>      
>       ## Commit message ##
>     -    CI: stop over-setting the $CC variable
>     +    CI: set CC in MAKEFLAGS directly, don't add it to the environment
>      
>     -    As detailed in 2c8921db2b8 (travis-ci: build with the right compiler,
>     -    2019-01-17) the reason we started using $CC in $MAKEFLAGS as opposed
>     -    to setting it in the environment was due to Travis CI clobbering $CC
>     -    in the environment.
>     +    Rather than pass a "$CC" and "$CC_PACKAGE" in the environment to be
>     +    picked up in ci/lib.sh let's instead have ci/lib.sh itself add it
>     +    directly to MAKEFLAGS.
>      
>     -    We don't need to set it unconditionally to accomplish that, but rather
>     -    just have it set for those jobs that need them. E.g. the "win+VS
>     -    build" job confusingly has CC=gcc set, even though it builds with
>     -    MSVC.
>     +    Setting CC=gcc by default made for confusing trace output, and since a
>     +    preceding change to carry it and others over across "steps" in the
>     +    GitHub CI it's been even more misleading.  E.g. the "win+VS build" job
>     +    confusingly has CC=gcc set, even though it builds with MSVC.
>     +
>     +    Let's instead reply on the Makefile default of CC=cc, and only
>     +    override it for those jobs where it's needed. This does mean that
>     +    we'll need to set it for the "pedantic" job, which previously relied
>     +    on the default CC=gcc in case "clang" become the default on that
>     +    platform.
>      
>          This partially reverts my 707d2f2fe86 (CI: use "$runs_on_pool", not
>          "$jobname" to select packages & config, 2021-11-23), i.e. we're now
>     @@ ci/lib.sh: linux-TEST-vars)
>       	setenv --test GIT_TEST_DEFAULT_HASH sha256
>       	;;
>       pedantic)
>     ++	CC=gcc
>     + 	# Don't run the tests; we only care about whether Git can be
>     + 	# built.
>     + 	setenv --build DEVOPTS pedantic
>      @@ ci/lib.sh: linux-musl)
>       	MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8"
>       	;;
> 24:  d7b472b4a52 = 24:  06bf8d9f61b CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case
> 25:  08a9776c259 = 25:  6dc96ba8b82 CI: don't use "set -x" in "ci/lib.sh" output

Re: [PATCH v2 00/25] CI: run "make [test]" directly, use $GITHUB_ENV

[Ævar Arnfjörð Bjarmason]

On Fri, Mar 25 2022, Victoria Dye wrote:

> Ævar Arnfjörð Bjarmason wrote:
>> A re-roll of my improvements my series to simplify the CI setup a lot
>> (see diffstat), much of it was dealing with constraints that went away
>> with Travis et al. CI for this series (OSX runners failing for
>> unrelated reasons):
>> 
>>     https://github.com/avar/git/actions/runs/2040223909
>> 
>> For a much more detailed summary of how the output looks before/after
>> see v1[].
>> 
>> This series heavily conflicts with Johannes's
>> js/ci-github-workflow-markup in "seen", but in the v1 I suggested
>> basing that series on top of this one, because it can benefit a lot
>> from these simplifications.
>> 
>> I'll reply to this series with a proposed rebasing of that series on
>> top of this one, which allows for removing almost all of its changes
>> to "ci/" with no harm to its end-goals, i.e. the splitting up of
>> "make" and "make test" output is something it'll get for free from
>> this series.
>> 
>> Junio: Since that series has been stalled on still-outstanding
>> performance issues for a couple of months I was hoping we could queue
>> this instead, and perhaps in addition if Johannes approves of the
>> proposed re-roll on top of his.
>> 
>> There's some forward progress on the performance issues (this[2] reply
>> of Victoria Dye's from yesterday), but fully resolving those will
>> probably take a bit...
>> 
>> Whereas even though this one is relatively large I don't think there's
>> anything controversial here. The one concern that's been raised has
>> been Johannes's objection to removing some of the dead Azure code
>> (which was needed to move forward here). I asked how he'd prefer to
>> move forward with that in [3], but there hasn't been a reply to that
>> in >1 month.
>> 
>
> While the largeness of a series shouldn't necessarily block it, the lack of
> overarching structure or purpose in this one makes it really difficult for
> me to review with much confidence (I can't speak for everyone, but it may be
> one of the reasons for the general lack of feedback). If you believe all of
> these patches as thematically-related enough to warrant being in a single
> series, then it would help a lot if you could:
>
> 1. Clearly describe the purpose of the series (yes they're all CI
>    improvements, but *why* these particular improvements, and why do they
>    all need to go together?) 
> 2. Outline the "path" these commits take to accomplishing that purpose ("The
>    first 3 commits do X because Y. Then, the next 4 commits do A because B."
>    etc. or whatever format fits your writing style, as long as the
>    information is there).
> 3. Reorganize commits as necessary to keep the above outline from jumping
>    back and forth between topics. 

The v1 summary described it about as clearly as I was able to:
https://lore.kernel.org/git/cover-00.25-00000000000-20220221T143936Z-avarab@gmail.com/

> Personally, I think this could (should?) be split into at least two series:
> one that breaks up 'run-build-and-tests.sh' (and is more directly relevant
> to dscho's series), and one that does the cleanup/flag change/other work.
> The two appear to be independent, and the resulting two series would be a
> much more manageable 10-15 commits each. 

To rephrase that a bit, every commit here passes CI and is atomic, so it
could be split up into 25 parts (at least).

But it's really not doing different things, it's a single-topic series:
It's changing CI "step" targets that are shellscripts that do N things
to instead be single command invocations at the "step" level, driven by
the CI recipe itself.

To do that we need to pass state that we previously re-setup for every
"step" via $GITHUB_ENV, whose state we then helpfully show (this is just
a standard GitHub CI feature) in a drop-down at the start of every
"step".

So yes, it could be split up in the sense that we could get partway
there and continue with the rest some other time, but I really think the
UX experience is *much better* if it's not a partial conversion.

I.e. at the tip of this series you can reliably look at that $GITHUB_ENV
view to see what the full and relevant environment was for that "make",
"make test" or whatever.

If we just do that partially you might get that for "make" if it failed,
but if your tests failed we'd have a failure in the proverbial
ci/load-lib.sh-do-setup-stuff-and-run-make-test-at-some-point.sh.

Which I think makes the UX much less useful, i.e. you really want to
*always* find this information in the same place.

Re: [PATCH v2 00/25] CI: run "make [test]" directly, use $GITHUB_ENV

[Phillip Wood]

Hi Ævar

On 25/03/2022 20:43, Victoria Dye wrote:
> Ævar Arnfjörð Bjarmason wrote:
>> A re-roll of my improvements my series to simplify the CI setup a lot
>> (see diffstat), much of it was dealing with constraints that went away
>> with Travis et al. CI for this series (OSX runners failing for
>> unrelated reasons):
>>
>>      https://github.com/avar/git/actions/runs/2040223909
>>
>> For a much more detailed summary of how the output looks before/after
>> see v1[].
>>
>> This series heavily conflicts with Johannes's
>> js/ci-github-workflow-markup in "seen", but in the v1 I suggested
>> basing that series on top of this one, because it can benefit a lot
>> from these simplifications.
>>
>> I'll reply to this series with a proposed rebasing of that series on
>> top of this one, which allows for removing almost all of its changes
>> to "ci/" with no harm to its end-goals, i.e. the splitting up of
>> "make" and "make test" output is something it'll get for free from
>> this series.
>>
>> Junio: Since that series has been stalled on still-outstanding
>> performance issues for a couple of months I was hoping we could queue
>> this instead, and perhaps in addition if Johannes approves of the
>> proposed re-roll on top of his.
>>
>> There's some forward progress on the performance issues (this[2] reply
>> of Victoria Dye's from yesterday), but fully resolving those will
>> probably take a bit...
>>
>> Whereas even though this one is relatively large I don't think there's
>> anything controversial here. The one concern that's been raised has
>> been Johannes's objection to removing some of the dead Azure code
>> (which was needed to move forward here). I asked how he'd prefer to
>> move forward with that in [3], but there hasn't been a reply to that
>> in >1 month.
>>
> 
> While the largeness of a series shouldn't necessarily block it, the lack of
> overarching structure or purpose in this one makes it really difficult for
> me to review with much confidence (I can't speak for everyone, but it may be
> one of the reasons for the general lack of feedback).

That applies to me, I think this would be easier to follow if it was 
split up. As far as I can tell there are several groups of changes and I 
think it would be easier for reviewers if they were split into separate 
shorter series.

1. Split building git and running the tests into separate steps. This is
    I think the only user facing change. I think it is worthwhile and
    could be done in a single patch that just moved make test into a
    separate script without a lot of refactoring. In particular I do not
    see the need for separate steps that setup the environment for the
    build and tests (especially as the build environment variables are
    still present when the tests are run).
    Without the refactoring anyone resurrecting the azure pipelines for
    testing security fixes would easily be able to update those build
    definitions.

2. Moving the static analysis ci job into the Makefile. I'm not sure
    what the practical benefit of this is but it could be a self
    contained series.

3. Being able to run the ci scripts locally. Does this have a use beyond
    debugging them locally? Could this be implemented with a wrapper
    script rather than a large refactoring?

4. Cleanups. To be honest I think we could happily live without a some
    of these, especially patches 24-26 of v3 that rework code introduced
    in this series.

I see v4 has been posted while I was writing this and is even longer 
than v3.

Best Wishes

Phillip

> If you believe all of
> these patches as thematically-related enough to warrant being in a single
> series, then it would help a lot if you could:
> 
> 1. Clearly describe the purpose of the series (yes they're all CI
>     improvements, but *why* these particular improvements, and why do they
>     all need to go together?)
> 2. Outline the "path" these commits take to accomplishing that purpose ("The
>     first 3 commits do X because Y. Then, the next 4 commits do A because B."
>     etc. or whatever format fits your writing style, as long as the
>     information is there).
> 3. Reorganize commits as necessary to keep the above outline from jumping
>     back and forth between topics.
> 
> Personally, I think this could (should?) be split into at least two series:
> one that breaks up 'run-build-and-tests.sh' (and is more directly relevant
> to dscho's series), and one that does the cleanup/flag change/other work.
> The two appear to be independent, and the resulting two series would be a
> much more manageable 10-15 commits each.
> 
>> I think just removing it is OK, we can always bring it back if needed,
>> and doing so is actually going to be simpler on top of this since the
>> CI is now less special, and leans more heavily on the logic of our
>> normal build process.
>>
>> 1. https://lore.kernel.org/git/cover-00.25-00000000000-20220221T143936Z-avarab@gmail.com
>> 2. https://lore.kernel.org/git/6b83bb83-32b9-20c9-fa02-c1c3170351c3@github.com/
>> 3. https://lore.kernel.org/git/220222.86y2236ndp.gmgdl@evledraar.gmail.com/
>>
>> Ævar Arnfjörð Bjarmason (25):
>>    CI: run "set -ex" early in ci/lib.sh
>>    CI: make "$jobname" explicit, remove fallback
>>    CI: remove more dead Travis CI support
>>    CI: remove dead "tree skipping" code
>>    CI: remove unused Azure ci/* code
>>    CI: don't have "git grep" invoke a pager in tree content check
>>    CI: have "static-analysis" run a "make ci-static-analysis" target
>>    CI: have "static-analysis" run "check-builtins", not "documentation"
>>    CI: move p4 and git-lfs variables to ci/install-dependencies.sh
>>    CI: consistently use "export" in ci/lib.sh
>>    CI: export variables via a wrapper
>>    CI: remove "run-build-and-tests.sh", run "make [test]" directly
>>    CI: check ignored unignored build artifacts in "win[+VS] build" too
>>    CI: invoke "make artifacts-tar" directly in windows-build
>>    CI: split up and reduce "ci/test-documentation.sh"
>>    CI: combine ci/install{,-docker}-dependencies.sh
>>    CI: move "env" definitions into ci/lib.sh
>>    ci/run-test-slice.sh: replace shelling out with "echo"
>>    CI: pre-select test slice in Windows & VS tests
>>    CI: only invoke ci/lib.sh as "steps" in main.yml
>>    CI: narrow down variable definitions in --build and --test
>>    CI: add more variables to MAKEFLAGS, except under vs-build
>>    CI: set CC in MAKEFLAGS directly, don't add it to the environment
>>    CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case
>>    CI: don't use "set -x" in "ci/lib.sh" output
>>
>>   .github/workflows/main.yml            | 100 +++++---
>>   Makefile                              |  31 ++-
>>   ci/check-directional-formatting.bash  |   2 +-
>>   ci/check-unignored-build-artifacts.sh |  20 ++
>>   ci/install-dependencies.sh            |  53 ++++-
>>   ci/install-docker-dependencies.sh     |  22 --
>>   ci/lib-ci-type.sh                     |  10 +
>>   ci/lib-tput.sh                        |   2 +
>>   ci/lib.sh                             | 316 +++++++++++++-------------
>>   ci/make-test-artifacts.sh             |  12 -
>>   ci/mount-fileshare.sh                 |  25 --
>>   ci/print-test-failures.sh             |  16 +-
>>   ci/run-build-and-tests.sh             |  54 -----
>>   ci/run-docker-build.sh                |  66 ------
>>   ci/run-docker.sh                      |  47 ----
>>   ci/run-static-analysis.sh             |  32 ---
>>   ci/run-test-slice.sh                  |  17 --
>>   ci/select-test-slice.sh               |  10 +
>>   ci/test-documentation.sh              |  37 +--
>>   ci/util/extract-trash-dirs.sh         |  50 ----
>>   shared.mak                            |   1 +
>>   21 files changed, 346 insertions(+), 577 deletions(-)
>>   create mode 100755 ci/check-unignored-build-artifacts.sh
>>   delete mode 100755 ci/install-docker-dependencies.sh
>>   create mode 100644 ci/lib-ci-type.sh
>>   create mode 100644 ci/lib-tput.sh
>>   delete mode 100755 ci/make-test-artifacts.sh
>>   delete mode 100755 ci/mount-fileshare.sh
>>   delete mode 100755 ci/run-build-and-tests.sh
>>   delete mode 100755 ci/run-docker-build.sh
>>   delete mode 100755 ci/run-docker.sh
>>   delete mode 100755 ci/run-static-analysis.sh
>>   delete mode 100755 ci/run-test-slice.sh
>>   create mode 100755 ci/select-test-slice.sh
>>   delete mode 100755 ci/util/extract-trash-dirs.sh
>>
>> Range-diff against v1:
>>   1:  970849a227f =  1:  4addbd70213 CI: run "set -ex" early in ci/lib.sh
>>   2:  eda7fb18064 =  2:  b23aa99fd37 CI: make "$jobname" explicit, remove fallback
>>   3:  3ee32815cf3 =  3:  eec15a95879 CI: remove more dead Travis CI support
>>   4:  c81c1b3f504 =  4:  73c894f1665 CI: remove dead "tree skipping" code
>>   5:  4738a22a36d =  5:  b5e6d538554 CI: remove unused Azure ci/* code
>>   6:  59e4f41e86c =  6:  a4b9febbdca CI: don't have "git grep" invoke a pager in tree content check
>>   7:  836ef20fdcc !  7:  5d46d5b34c9 CI: have "static-analysis" run a "make ci-static-analysis" target
>>      @@ .github/workflows/main.yml: jobs:
>>            if: needs.ci-config.outputs.enabled == 'yes'
>>       
>>        ## Makefile ##
>>      -@@ Makefile: ifndef V
>>      - 	QUIET_HDR      = @echo '   ' HDR $(<:hcc=h);
>>      - 	QUIET_RC       = @echo '   ' RC $@;
>>      - 	QUIET_SPATCH   = @echo '   ' SPATCH $<;
>>      -+	QUIET_CHECK    = @echo '   ' CHECK $@;
>>      - 	QUIET_SUBDIR0  = +@subdir=
>>      - 	QUIET_SUBDIR1  = ;$(NO_SUBDIR) echo '   ' SUBDIR $$subdir; \
>>      - 			 $(MAKE) $(PRINT_DIR) -C $$subdir
>>       @@ Makefile: coccicheck: $(addsuffix .patch,$(filter-out %.pending.cocci,$(wildcard contrib/c
>>        # See contrib/coccinelle/README
>>        coccicheck-pending: $(addsuffix .patch,$(wildcard contrib/coccinelle/*.pending.cocci))
>>      @@ ci/run-static-analysis.sh (deleted)
>>       -
>>       -make hdr-check ||
>>       -exit 1
>>      +
>>      + ## shared.mak ##
>>      +@@ shared.mak: ifndef V
>>      + 	QUIET_HDR      = @echo '   ' HDR $(<:hcc=h);
>>      + 	QUIET_RC       = @echo '   ' RC $@;
>>      + 	QUIET_SPATCH   = @echo '   ' SPATCH $<;
>>      ++	QUIET_CHECK    = @echo '   ' CHECK $@;
>>      +
>>      + ## Used in "Documentation/Makefile"
>>      + 	QUIET_ASCIIDOC	= @echo '   ' ASCIIDOC $@;
>>   8:  95cd496868e =  8:  81e06f13d84 CI: have "static-analysis" run "check-builtins", not "documentation"
>>   9:  a1d0796259e =  9:  3be91c26d44 CI: move p4 and git-lfs variables to ci/install-dependencies.sh
>> 10:  b6a61a786c5 = 10:  9dc148341ba CI: consistently use "export" in ci/lib.sh
>> 11:  f2fcee5d6e4 = 11:  e9c7ba492e8 CI: export variables via a wrapper
>> 12:  dfd823f2e7d = 12:  86d5a48d59a CI: remove "run-build-and-tests.sh", run "make [test]" directly
>> 13:  46459fff296 = 13:  649ad1ae249 CI: check ignored unignored build artifacts in "win[+VS] build" too
>> 14:  aecd3ebaafe = 14:  b1b5b083389 CI: invoke "make artifacts-tar" directly in windows-build
>> 15:  4f1564af70f = 15:  dfa91ac8938 CI: split up and reduce "ci/test-documentation.sh"
>> 16:  868613a5b06 = 16:  ba4ed216769 CI: combine ci/install{,-docker}-dependencies.sh
>> 17:  5d854e8ff36 = 17:  b5e89a33340 CI: move "env" definitions into ci/lib.sh
>> 18:  a6106525b7f = 18:  571f4d0f441 ci/run-test-slice.sh: replace shelling out with "echo"
>> 19:  e9c6c4dd293 = 19:  2edea06ee4d CI: pre-select test slice in Windows & VS tests
>> 20:  40d86e8c1dc = 20:  ef9daa6882f CI: only invoke ci/lib.sh as "steps" in main.yml
>> 21:  abf9c504740 = 21:  44e3ace5fbe CI: narrow down variable definitions in --build and --test
>> 22:  9f4c2798a82 = 22:  5f56b922e08 CI: add more variables to MAKEFLAGS, except under vs-build
>> 23:  8a8b7ecf16b ! 23:  b45b7cec94e CI: stop over-setting the $CC variable
>>      @@ Metadata
>>       Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>>       
>>        ## Commit message ##
>>      -    CI: stop over-setting the $CC variable
>>      +    CI: set CC in MAKEFLAGS directly, don't add it to the environment
>>       
>>      -    As detailed in 2c8921db2b8 (travis-ci: build with the right compiler,
>>      -    2019-01-17) the reason we started using $CC in $MAKEFLAGS as opposed
>>      -    to setting it in the environment was due to Travis CI clobbering $CC
>>      -    in the environment.
>>      +    Rather than pass a "$CC" and "$CC_PACKAGE" in the environment to be
>>      +    picked up in ci/lib.sh let's instead have ci/lib.sh itself add it
>>      +    directly to MAKEFLAGS.
>>       
>>      -    We don't need to set it unconditionally to accomplish that, but rather
>>      -    just have it set for those jobs that need them. E.g. the "win+VS
>>      -    build" job confusingly has CC=gcc set, even though it builds with
>>      -    MSVC.
>>      +    Setting CC=gcc by default made for confusing trace output, and since a
>>      +    preceding change to carry it and others over across "steps" in the
>>      +    GitHub CI it's been even more misleading.  E.g. the "win+VS build" job
>>      +    confusingly has CC=gcc set, even though it builds with MSVC.
>>      +
>>      +    Let's instead reply on the Makefile default of CC=cc, and only
>>      +    override it for those jobs where it's needed. This does mean that
>>      +    we'll need to set it for the "pedantic" job, which previously relied
>>      +    on the default CC=gcc in case "clang" become the default on that
>>      +    platform.
>>       
>>           This partially reverts my 707d2f2fe86 (CI: use "$runs_on_pool", not
>>           "$jobname" to select packages & config, 2021-11-23), i.e. we're now
>>      @@ ci/lib.sh: linux-TEST-vars)
>>        	setenv --test GIT_TEST_DEFAULT_HASH sha256
>>        	;;
>>        pedantic)
>>      ++	CC=gcc
>>      + 	# Don't run the tests; we only care about whether Git can be
>>      + 	# built.
>>      + 	setenv --build DEVOPTS pedantic
>>       @@ ci/lib.sh: linux-musl)
>>        	MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8"
>>        	;;
>> 24:  d7b472b4a52 = 24:  06bf8d9f61b CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case
>> 25:  08a9776c259 = 25:  6dc96ba8b82 CI: don't use "set -x" in "ci/lib.sh" output
> 

Re: [PATCH v2 00/25] CI: run "make [test]" directly, use $GITHUB_ENV

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> A re-roll of my improvements my series to simplify the CI setup a lot
> (see diffstat), much of it was dealing with constraints that went away
> with Travis et al. CI for this series (OSX runners failing for
> unrelated reasons):
>
>     https://github.com/avar/git/actions/runs/2040223909
>
> For a much more detailed summary of how the output looks before/after
> see v1[].
>
> This series heavily conflicts with Johannes's
> js/ci-github-workflow-markup in "seen", but in the v1 I suggested
> basing that series on top of this one, because it can benefit a lot
> from these simplifications.
>
> I'll reply to this series with a proposed rebasing of that series on
> top of this one, which allows for removing almost all of its changes
> to "ci/" with no harm to its end-goals, i.e. the splitting up of
> "make" and "make test" output is something it'll get for free from
> this series.

A sample run for this can be seen at

  https://github.com/git/git/runs/5715128999?check_suite_focus=true

With the output, I can point at a specific line, e.g.

  https://github.com/git/git/runs/5715128999?check_suite_focus=true#step:7:1150

The URLs with Dscho's one that correspond to the above two are

  https://github.com/git/git/runs/5699946885?check_suite_focus=true
  https://github.com/git/git/runs/5699946885?check_suite_focus=true#step:4:1826

The specific breakage (which has little to do with the comparison
between CI error pages) is that diff_setup() is called and populate
diff_options.parseopts member but diff_setup_done() does not seem to
be called and ends up leaking it.

I wonder why options->parseopts is not freed immediately after
diff_opt_parse() calls parse_options(), but need to be kept until
diff_setup_done().

Re: [PATCH v2 00/25] CI: run "make [test]" directly, use $GITHUB_ENV

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 2747 bytes --]

Hi Ævar,

On Fri, 25 Mar 2022, Ævar Arnfjörð Bjarmason wrote:

> A re-roll of my improvements my series to simplify the CI setup a lot
> (see diffstat), much of it was dealing with constraints that went away
> with Travis et al.

This type of work causes me a lot of follow-up work e.g. many merge
conflicts in the latest Git for Windows rebases.

Perhaps it is worth taking a step back and evaluating the return on this
investment in the CI setup. While this can be characterized as a
simplification taking the diffstat as proof, one could challenge that the
diffstat does not actually measure whether the code is simple or not, it
just measures whether there are less lines in the end.

If the diffstat was a good measure, then the optimum would be one 0-byte
`.c` file (which some C compilers compile without error). Another obvious
way to optimize the diffstat would be to remove all code comments. Would I
suggest to do either? Of course not.

The reduction in code size of this patch series also comes at quite a
steep cost: After all, the way Lars and Gábor set things up was already
easy to reuse with Azure Pipeline and GitHub Actions.

Removing this type of generic, easily-to-adapt code can remove a lot of
lines at the expense of making the code less generic and harder to adapt,
and leads us directly to CI lock-in.

A better approach would be to use the already-generic code and adapt it
e.g. to extend to the CirrusCI definition we have.

Even if you do not care about extending our FreeBSD coverage, I would like
to ask to slow down on refactoring as it is done in this patch series. As
indicated in my comment above, these types of refactorings lead to a lot
of complications in Git for Windows's processes, which are time-consuming
to resolve. I understand your motivation, but if you wouldn't mind taking
some time to weigh the criticality of these changes against the overhead
incurred for maintainers, it would be appreciated.

> I think just removing it is OK, we can always bring it back if needed,
> and doing so is actually going to be simpler on top of this since the CI
> is now less special, and leans more heavily on the logic of our normal
> build process.

Removing and re-adding things does take time, though. Again, I think it
would be helpful to step back and try to understand the value of this
removal versus the projected time it would take (from all involved) to
re-add.

Besides, given how much is shuffled around in this patch series (e.g. some
files in ci/ are removed altogether and their equivalent code is moved
into various other places), doubt must be cast on the idea that it would
be simple to bring back anything here.

Ciao,
Johannes

Re: [PATCH v2 00/25] CI: run "make [test]" directly, use $GITHUB_ENV

[Ævar Arnfjörð Bjarmason]

On Tue, Apr 05 2022, Johannes Schindelin wrote:

> Hi Ævar,
>
> On Fri, 25 Mar 2022, Ævar Arnfjörð Bjarmason wrote:
>
>> A re-roll of my improvements my series to simplify the CI setup a lot
>> (see diffstat), much of it was dealing with constraints that went away
>> with Travis et al.
>
> This type of work causes me a lot of follow-up work e.g. many merge
> conflicts in the latest Git for Windows rebases.

Can you elaborate a bit on why? These patches are currently in "seen"
and not "next", and I thought GFW merged git.git's "master" for new
releases, and e.g. in:

    git diff --stat -p v2.36.0-rc0..v2.36.0-rc0.windows.1 -- ci t

I don't see anything related to those sorts of CI changes.

I.e. I understand that you may have something out-of-git.git that may
conflict with this, I'm confused about the GFW reference in particular.

> Perhaps it is worth taking a step back and evaluating the return on this
> investment in the CI setup. While this can be characterized as a
> simplification taking the diffstat as proof, one could challenge that the
> diffstat does not actually measure whether the code is simple or not, it
> just measures whether there are less lines in the end.
>
> If the diffstat was a good measure, then the optimum would be one 0-byte
> `.c` file (which some C compilers compile without error). Another obvious
> way to optimize the diffstat would be to remove all code comments. Would I
> suggest to do either? Of course not.

This is a misunderstanding I really didn't expect, but no. I'm not
suggesting that the diffstat being shorter is a value in and of itself.

Clearly that's ridiculous, as we could make the code "better" by
removing subsequent newlines or whatever.

I thought it was clear in context that the v2 CL's reference to this is
a shorthand for us being able to generally "do more with less"
summarized in the v1:
https://lore.kernel.org/git/cover-00.25-00000000000-20220221T143936Z-avarab@gmail.com

> The reduction in code size of this patch series also comes at quite a
> steep cost: After all, the way Lars and Gábor set things up was already
> easy to reuse with Azure Pipeline and GitHub Actions.
>
> Removing this type of generic, easily-to-adapt code can remove a lot of
> lines at the expense of making the code less generic and harder to adapt,
> and leads us directly to CI lock-in.

It's really hard to try to come to some sort of landing with changes in
ci/ when it looks as though you haven't read my side of the patches or
why these changes were made.

So in liue of some long summary of that I'll just say briefly that the
reason for why I made those changes is covered in detail in commits you
haven't replied to, and which argue the opposite of what you do here.

Briefly: The entire reason I changed those bits is exactly to avoid the
sort of CI lock-in you're talking about, to th point where this series
effectively adds another CI target: You can run everything it's doing
with a normal "make" invocation.

> A better approach would be to use the already-generic code and adapt it
> e.g. to extend to the CirrusCI definition we have.

This series doesn't change .cirrus.yml or how it functions, but it just
does:

  build_script:
    - su git -c gmake
  test_script:
    - su git -c 'gmake test'

Which, after this series is exactly what your "main" CI does. So we're
set up to make it easier to unify the two.

> Even if you do not care about extending our FreeBSD coverage,

FWIW I do care, and I've sent in various portability patches for
FreeBSD, but I haven't used that CI in particular.

> I would like
> to ask to slow down on refactoring as it is done in this patch series. As
> indicated in my comment above, these types of refactorings lead to a lot
> of complications in Git for Windows's processes, which are time-consuming
> to resolve. I understand your motivation, but if you wouldn't mind taking
> some time to weigh the criticality of these changes against the overhead
> incurred for maintainers, it would be appreciated.

Sure, I'm happy to accommodate that.

As this v2 CL notes I'd asked you a month before submitting it in [1]
what I could do to make the parts of it you seemed to find most
objectionable easier or different for you.

>> I think just removing it is OK, we can always bring it back if needed,
>> and doing so is actually going to be simpler on top of this since the CI
>> is now less special, and leans more heavily on the logic of our normal
>> build process.
>
> Removing and re-adding things does take time, though. Again, I think it
> would be helpful to step back and try to understand the value of this
> removal versus the projected time it would take (from all involved) to
> re-add.

Having prepped this v2 + the RFC of your (then smaller) patches on top
I'm pretty sure it would be trivial for either of us compared to
continuing this verbal back & forth.

So, to that end I re-rolled the two in combination, I think it would be
much more helpful to comment on the substance of the changes here.

E.g. we can now see the sum of my patches and yours in "seen", they both
change the CI UI in what I think are complimentary ways (issues of
e.g. added slowness in your patches aside, which has been discussed).

For easy comparison I prepared this the other day, and committed the
same "make CI fail" commit on top to all of them:

 * Your original patches, just rebased on master:
   https://github.com/avar/git/tree/avar-pr-1117/dscho/use-grouping-in-ci-v2-FAIL-TEST
 * Just my v2 series here:
   https://github.com/avar/git/tree/avar/ci-unroll-make-commands-to-ci-recipe-FAIL-TEST
 * The sum of the two, i.e. the v2 + your "RFC" (the RFC label being mine) on top:
   https://github.com/avar/git/tree/avar-dscho/use-grouping-in-ci-v2-FAIL-TEST

If you've got some conflicts or whatever with something out of tree
you'd like me to accommodate or even resolve for you I'd be happy to do
so, just send me the specifics.

> Besides, given how much is shuffled around in this patch series (e.g. some
> files in ci/ are removed altogether and their equivalent code is moved
> into various other places), doubt must be cast on the idea that it would
> be simple to bring back anything here.

I'm assuming you're talking about resurrecting Azure CI, i.e. (mostly)
this would semantically conflict:

    git grep ci/ 6081d3898fe^ -- azure-pipelines.yml

One thing I can see right away that would need adjusting is that I "git
rm" the "ci/mount-fileshare.sh" script as unused. It's part of what I
asked about in [1], but I could just leave that in place, would you like
me to do that for a re-roll & would that help?

For the rest "ci/install-dependencies.sh" is the same,
"ci/run-build-and-tests.sh" would be an easy matter of making it run
"make test" instead after invoking ci/lib.sh.

We're really just talking about a few lines of boilerplate here, I
really don't see what the big deal is.

As my patches also note there's things that the azure-pipelines.yml
assumed that are no longer true (before any changes I made), so some of
this would need adjusting in any case.

It also does a test run followed by ci/print-test-failures.sh, which as
I understand you've been maintaining quite adamantly insisting is pretty
much useless compared to the new GitHub-specific output your series
adds, and which is CI-specific. Presumably most of the work of getting
Azure running again would be (in your version) to adjust that to
something Azure-specific.

Whereas on the CI portability front I noted in [2] and [3] that we could
do pretty much the same without adding CI-specific output targets.

Anyway, I really hope we can find some way past what seems to be an
impasse with these various CI changes. All the best.

1. https://lore.kernel.org/git/220222.86y2236ndp.gmgdl@evledraar.gmail.com/
2. https://lore.kernel.org/git/220324.8635j7nyvw.gmgdl@evledraar.gmail.com/
3. https://lore.kernel.org/git/220302.86mti87cj2.gmgdl@evledraar.gmail.com/

Re: [PATCH v2 00/25] CI: run "make [test]" directly, use $GITHUB_ENV

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> Briefly: The entire reason I changed those bits is exactly to avoid the
> sort of CI lock-in you're talking about, to th point where this series
> effectively adds another CI target: You can run everything it's doing
> with a normal "make" invocation.

I'll comment only on this part.  

FWIW, the above matches my impression fromt he base part of the
rewrite.  As long as a particular CI integration runs shell
scriptlets that invoke "make", moving the code that is shared common
among CI platforms from the shell scriptlets to the recipe in
Makefile should not make it harder to support it, and ...

>> A better approach would be to use the already-generic code and adapt it
>> e.g. to extend to the CirrusCI definition we have.
>
> This series doesn't change .cirrus.yml or how it functions, but it just
> does:
>
>   build_script:
>     - su git -c gmake
>   test_script:
>     - su git -c 'gmake test'
>
> Which, after this series is exactly what your "main" CI does. So we're
> set up to make it easier to unify the two.

... BSDs or any POSIXy system I would expect to fall into the same
"shell scriptlets eventually driving make" pattern.

The same argument may not apply if another CI platform uses shell
scriptlets but does not want to use "make" at all.  The "do more in
'make'" movement could be hurting such a user.  I do not know if
Windows' preference to use cmake plays a role there, or if Dscho's
resistance comes from there, though.

> Anyway, I really hope we can find some way past what seems to be an
> impasse with these various CI changes. All the best.

Yup.

Thanks.

[PATCH v3 00/29] CI: run "make" in CI "steps", improve UX

[Ævar Arnfjörð Bjarmason]

This series changes CI "step" targets that are shellscripts that do N
things to instead be single command invocations at the "step" level,
driven by the CI recipe itself.

To do that we need to pass state that we previously re-setup for every
"step" via $GITHUB_ENV, whose state we then helpfully show (this is just
a standard GitHub CI feature) in a drop-down at the start of every
"step".

I.e. at the tip of this series you can reliably look at that
$GITHUB_ENV view to see what the full and relevant environment was for
that "make", "make test" or whatever.

Changes since v2:

 * A new CL summary, based on
   https://lore.kernel.org/git/220326.861qypldwn.gmgdl@evledraar.gmail.com/

 * It's now 29 patches instead of 25, but 3/4 of those are trivial
   (see range-diff), it just made sense to split them up for
   readability.

 * The rather minor in code but rather big in UX terms is that I've
   added 29/29, which explicitly documents and shows examples of using
   the CI tooling in a way where you can locally run a CI target as if
   though it were a given $jobname, e.g.:

	# run "make all test" like the "linux-leaks" job
	(eval $(jobname=linux-leaks ci/lib.sh --all) && make test)

	# run "make test" like the "linux-TEST-vars" job (uses various GIT_TEST_* modes)
	make && (eval $(jobname=linux-TEST-vars ci/lib.sh --test) && make test)

   The rest of the changes in this series are all here to make that
   nicer, i.e. ci/lib.sh has been taught to not panic if it's not
   under a known CI platform, but instead to fall back on doing as
   well as it can, and suggesting that you provide it with a "jobname"
   so it can be used as in the examples above.

   Likewise the ci/print-test-failures.sh script can now be run
   locally without any extra mock-ups, e.g.:

	make test || ci/print-test-failures.sh

The CI run for this series can be seen at:
https://github.com/avar/git/tree/avar/ci-unroll-make-commands-to-ci-recipe-3

 .github/workflows/main.yml            | 100 ++++---
 Makefile                              |  31 ++-
 ci/check-directional-formatting.bash  |   2 +-
 ci/check-unignored-build-artifacts.sh |  20 ++
 ci/install-dependencies.sh            |  53 +++-
 ci/install-docker-dependencies.sh     |  22 --
 ci/lib-ci-type.sh                     |   6 +
 ci/lib-tput.sh                        |   5 +
 ci/lib.sh                             | 387 +++++++++++++++-----------
 ci/make-test-artifacts.sh             |  12 -
 ci/mount-fileshare.sh                 |  25 --
 ci/print-test-failures.sh             |  25 +-
 ci/run-build-and-tests.sh             |  54 ----
 ci/run-docker-build.sh                |  66 -----
 ci/run-docker.sh                      |  47 ----
 ci/run-static-analysis.sh             |  32 ---
 ci/run-test-slice.sh                  |  17 --
 ci/select-test-slice.sh               |  10 +
 ci/test-documentation.sh              |  37 +--
 ci/util/extract-trash-dirs.sh         |  50 ----
 shared.mak                            |   1 +
 21 files changed, 420 insertions(+), 582 deletions(-)
 create mode 100755 ci/check-unignored-build-artifacts.sh
 delete mode 100755 ci/install-docker-dependencies.sh
 create mode 100644 ci/lib-ci-type.sh
 create mode 100644 ci/lib-tput.sh
 delete mode 100755 ci/make-test-artifacts.sh
 delete mode 100755 ci/mount-fileshare.sh
 delete mode 100755 ci/run-build-and-tests.sh
 delete mode 100755 ci/run-docker-build.sh
 delete mode 100755 ci/run-docker.sh
 delete mode 100755 ci/run-static-analysis.sh
 delete mode 100755 ci/run-test-slice.sh
 create mode 100755 ci/select-test-slice.sh
 delete mode 100755 ci/util/extract-trash-dirs.sh

Range-diff against v2:
 1:  4addbd70213 =  1:  3cb4749c5ea CI: run "set -ex" early in ci/lib.sh
 2:  b23aa99fd37 !  2:  7de95ff437e CI: make "$jobname" explicit, remove fallback
    @@ ci/lib.sh
     +# Starting assertions
     +if test -z "$jobname"
     +then
    -+	echo "must set a CI jobname" >&2
    ++	echo "must set a CI jobname in the environment" >&2
     +	exit 1
     +fi
     +
 3:  eec15a95879 =  3:  820c4e551d9 CI: remove more dead Travis CI support
 4:  73c894f1665 =  4:  914e81118b8 CI: remove dead "tree skipping" code
 5:  b5e6d538554 !  5:  b2e456f9a9b CI: remove unused Azure ci/* code
    @@ ci/mount-fileshare.sh (deleted)
     
      ## ci/print-test-failures.sh ##
     @@ ci/print-test-failures.sh: do
    - 		test_name="${TEST_EXIT%.exit}"
      		test_name="${test_name##*/}"
      		trash_dir="trash directory.$test_name"
    -+		mkdir -p failed-test-artifacts
      		case "$CI_TYPE" in
     -		azure-pipelines)
     -			mkdir -p failed-test-artifacts
    @@ ci/print-test-failures.sh: do
     -			continue
     -			;;
      		github-actions)
    --			mkdir -p failed-test-artifacts
    + 			mkdir -p failed-test-artifacts
      			echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
    - 			cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
    - 			tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
 -:  ----------- >  6:  30968e36bdd CI/lib.sh: stop adding leading whitespace to $MAKEFLAGS
 6:  a4b9febbdca =  7:  cc81b9fe37e CI: don't have "git grep" invoke a pager in tree content check
 7:  5d46d5b34c9 =  8:  2387b0c5842 CI: have "static-analysis" run a "make ci-static-analysis" target
 8:  81e06f13d84 =  9:  54a4d79bf8d CI: have "static-analysis" run "check-builtins", not "documentation"
 9:  3be91c26d44 = 10:  9a31b7d5011 CI: move p4 and git-lfs variables to ci/install-dependencies.sh
10:  9dc148341ba ! 11:  8ab4e81e1ca CI: consistently use "export" in ci/lib.sh
    @@ Commit message
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## ci/lib.sh ##
    -@@ ci/lib.sh: check_unignored_build_artifacts ()
    - export TERM=${TERM:-dumb}
    +@@ ci/lib.sh: export TERM=${TERM:-dumb}
    + NPROC=10
      
      # Clear MAKEFLAGS that may come from the outside world.
    --export MAKEFLAGS=
    -+MAKEFLAGS=
    +-export MAKEFLAGS=--jobs=$NPROC
    ++MAKEFLAGS=--jobs=$NPROC
      
      if test "$GITHUB_ACTIONS" = "true"
      then
     @@ ci/lib.sh: then
      	CC="${CC:-gcc}"
      
    - 	export GIT_PROVE_OPTS="--timer --jobs 10"
    + 	export GIT_PROVE_OPTS="--timer --jobs $NPROC"
     -	export GIT_TEST_OPTS="--verbose-log -x"
     +	GIT_TEST_OPTS="--verbose-log -x"
    - 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
      	test Windows != "$RUNNER_OS" ||
      	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
    -+
     +	export GIT_TEST_OPTS
      else
      	echo "Could not identify CI type" >&2
11:  e9c7ba492e8 ! 12:  9fe00566767 CI: export variables via a wrapper
    @@ Commit message
         "--test" and "--all" parameters to indicate whether they're needed for
         building, testing or both.
     
    -    We currently ignore this parameter, but will make use of it in a
    +    We currently ignore the "varname" parameter, but will make use of it in a
         subsequent commit. Adding it here reduces the needless churn, as we
         won't need to change all these lines again for that subsequent change.
     
    @@ ci/lib.sh: then
      
     +# Helper functions
     +setenv () {
    -+	varmode=
    -+	case "$1" in
    -+	--*)
    -+		varmode=$1
    ++	while test $# != 0
    ++	do
    ++		case "$1" in
    ++		--build)
    ++			;;
    ++		--test)
    ++			;;
    ++		--all)
    ++			;;
    ++		-*)
    ++			echo "BUG: bad setenv() option '$1'" >&2
    ++			exit 1
    ++			;;
    ++		*)
    ++			break
    ++			;;
    ++		esac
     +		shift
    -+		;;
    -+	esac
    ++	done
     +
     +	key=$1
     +	val=$2
    @@ ci/lib.sh: check_unignored_build_artifacts ()
     -export TERM=${TERM:-dumb}
     +setenv TERM ${TERM:-dumb}
      
    - # Clear MAKEFLAGS that may come from the outside world.
    - MAKEFLAGS=
    + # How many jobs to run in parallel?
    + NPROC=10
     @@ ci/lib.sh: then
      	CI_TYPE=github-actions
      	CC="${CC:-gcc}"
      
    --	export GIT_PROVE_OPTS="--timer --jobs 10"
    -+	setenv --test GIT_PROVE_OPTS "--timer --jobs 10"
    +-	export GIT_PROVE_OPTS="--timer --jobs $NPROC"
    ++	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
      	GIT_TEST_OPTS="--verbose-log -x"
    - 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
      	test Windows != "$RUNNER_OS" ||
      	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
    - 
     -	export GIT_TEST_OPTS
     +	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
      else
12:  86d5a48d59a ! 13:  13feda050c0 CI: remove "run-build-and-tests.sh", run "make [test]" directly
    @@ ci/lib.sh: setenv () {
     +	if test -n "$GITHUB_ENV"
     +	then
     +		echo "$key=$val" >>"$GITHUB_ENV"
    -+	else
    -+		# For local debugging. Not used by the GitHub CI
    -+		# itself.
    -+		eval "export $key=\"$val\""
     +	fi
      }
      
 -:  ----------- > 14:  d3909b8e896 ci/lib.sh: use "test" instead of "["
13:  649ad1ae249 = 15:  d63a689eae0 CI: check ignored unignored build artifacts in "win[+VS] build" too
14:  b1b5b083389 = 16:  ac1ffe9c642 CI: invoke "make artifacts-tar" directly in windows-build
15:  dfa91ac8938 = 17:  8837bfa5433 CI: split up and reduce "ci/test-documentation.sh"
16:  ba4ed216769 = 18:  aa491990c1a CI: combine ci/install{,-docker}-dependencies.sh
17:  b5e89a33340 = 19:  39b13298425 CI: move "env" definitions into ci/lib.sh
18:  571f4d0f441 = 20:  0a92c478d28 ci/run-test-slice.sh: replace shelling out with "echo"
19:  2edea06ee4d = 21:  16ac7f5c210 CI: pre-select test slice in Windows & VS tests
20:  ef9daa6882f ! 22:  939ef573b75 CI: only invoke ci/lib.sh as "steps" in main.yml
    @@ ci/install-dependencies.sh
     
      ## ci/lib-ci-type.sh (new) ##
     @@
    ++#!/bin/sh
    ++
     +if test "$GITHUB_ACTIONS" = "true"
     +then
     +	CI_TYPE=github-actions
     +else
     +	echo "Could not identify CI type" >&2
    -+	env >&2
     +	exit 1
     +fi
     
      ## ci/lib-tput.sh (new) ##
     @@
    ++#!/bin/sh
    ++
     +# GitHub Action doesn't set TERM, which is required by tput
    -+export TERM=${TERM:-dumb}
    ++TERM=${TERM:-dumb}
    ++export TERM
     
      ## ci/lib.sh ##
     @@
    @@ ci/lib.sh: setenv () {
     -# GitHub Action doesn't set TERM, which is required by tput
     -setenv TERM ${TERM:-dumb}
     -
    + # How many jobs to run in parallel?
    + NPROC=10
    + 
      # Clear MAKEFLAGS that may come from the outside world.
    - MAKEFLAGS=
    + MAKEFLAGS=--jobs=$NPROC
      
     -if test "$GITHUB_ACTIONS" = "true"
     -then
    @@ ci/lib.sh: setenv () {
     +github-actions)
      	CC="${CC:-gcc}"
      
    - 	setenv --test GIT_PROVE_OPTS "--timer --jobs 10"
    + 	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
     @@ ci/lib.sh: then
    + 	test Windows != "$RUNNER_OS" ||
      	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
    - 
      	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
     -else
     -	echo "Could not identify CI type" >&2
21:  44e3ace5fbe ! 23:  8e79c3e39e0 CI: narrow down variable definitions in --build and --test
    @@ Commit message
         fewer, which makes diagnosing anything going on there easier, as we
         won't have to look at a deluge of e.g. GIT_TEST_* variables.
     
    +    Since we'll now always run "--build" for "make" and "--test" for "make
    +    test" we can stop setting the "fat" MAKEFLAGS entirely on the "test"
    +    steps, i.e. the one with PYTHON_PATH, CC etc. These will all be
    +    carried over from the earlier --build step.
    +
    +    That won't be true in the case of the "windows-test" and "vs-test"
    +    jobs, since they run separately from the corresponding "build"
    +    step. So we'll need to make sure that we have the --jobs=N argument
    +    for those.
    +
    +    This doesn't matter that much, as we'll still have --jobs=N in
    +    GIT_PROVE_OPTS. So the only thing we'll use it for is parallelism in
    +    the t/Makefile before we get to running "prove". Still, it's good to
    +    be consistent for good measure, and to run the t/Makefile itself in
    +    parallel.
    +
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## .github/workflows/main.yml ##
    @@ ci/lib.sh: set -ex
     +mode=$1
     +if test -z "$mode"
     +then
    -+	echo "need a $0 mode, e.g. --build or --test"
    ++	echo "need a $0 mode, e.g. --build or --test" >&2
     +	exit 1
     +fi
     +
      if test -z "$jobname"
      then
    - 	echo "must set a CI jobname" >&2
    -@@ ci/lib.sh: fi
    - 
    - # Helper functions
    - setenv () {
    -+	skip=
    - 	varmode=
    - 	case "$1" in
    - 	--*)
    -+		if test "$1" != "$mode" && test "$1" != "--all"
    -+		then
    -+			skip=t
    -+		fi
    - 		varmode=$1
    - 		shift
    - 		;;
    + 	echo "must set a CI jobname in the environment" >&2
     @@ ci/lib.sh: setenv () {
    - 	val=$2
    - 	shift 2
    + 	while test $# != 0
    + 	do
    + 		case "$1" in
    +-		--build)
    +-			;;
    +-		--test)
    +-			;;
    +-		--all)
    ++		--build | --test)
    ++			if test "$1" != "$mode"
    ++			then
    ++				return 0
    ++			fi
    ++
    + 			;;
    + 		-*)
    + 			echo "BUG: bad setenv() option '$1'" >&2
    +@@ ci/lib.sh: setenv () {
    + # How many jobs to run in parallel?
    + NPROC=10
      
    -+	if test -n "$skip"
    -+	then
    -+		return 0
    -+	fi
    ++# For "--test" we carry the MAKEFLAGS over from earlier steps, except
    ++# in stand-alone jobs which will use $COMMON_MAKEFLAGS.
    ++COMMON_MAKEFLAGS=--jobs=$NPROC
     +
    - 	if test -n "$GITHUB_ENV"
    - 	then
    - 		echo "$key=$val" >>"$GITHUB_ENV"
    + # Clear MAKEFLAGS that may come from the outside world.
    +-MAKEFLAGS=--jobs=$NPROC
    ++MAKEFLAGS=$COMMON_MAKEFLAGS
    + 
    + case "$CI_TYPE" in
    + github-actions)
    +@@ ci/lib.sh: windows-build)
    + 	setenv --build NO_PERL NoThanks
    + 	setenv --build ARTIFACTS_DIRECTORY artifacts
    + 	;;
    ++windows-test)
    ++	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
    ++	;;
    + vs-build)
    + 	setenv --build NO_PERL NoThanks
    + 	setenv --build NO_GETTEXT NoThanks
    +@@ ci/lib.sh: vs-build)
    + 	;;
    + vs-test)
    + 	setenv --test NO_SVN_TESTS YesPlease
    ++	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
    + 	;;
    + linux-gcc)
    + 	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
    +@@ ci/lib.sh: linux-leaks)
    + 	;;
    + esac
    + 
    +-setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
    ++setenv --build MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
22:  5f56b922e08 ! 24:  d1d5c1e2f65 CI: add more variables to MAKEFLAGS, except under vs-build
    @@ Commit message
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## ci/lib.sh ##
    -@@ ci/lib.sh: setenv () {
    +@@ ci/lib.sh: COMMON_MAKEFLAGS=--jobs=$NPROC
      # Clear MAKEFLAGS that may come from the outside world.
    - MAKEFLAGS=
    + MAKEFLAGS=$COMMON_MAKEFLAGS
      
    -+# Common make and cmake build options
    ++# Use common options for "make" (cmake in "vs-build" below uses the
    ++# intermediate variables directly)
     +DEVELOPER=1
    ++MAKEFLAGS="$MAKEFLAGS DEVELOPER=$DEVELOPER"
     +SKIP_DASHED_BUILT_INS=YesPlease
    -+
    -+# Use common options for "make" (cmake in "vs-build" below)
    -+MAKEFLAGS="DEVELOPER=$DEVELOPER SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
    ++MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
     +
      case "$CI_TYPE" in
      github-actions)
    @@ ci/lib.sh: github-actions)
      
      case "$runs_on_pool" in
      ubuntu-latest)
    -@@ ci/lib.sh: windows-build)
    - 	setenv --build ARTIFACTS_DIRECTORY artifacts
    +@@ ci/lib.sh: windows-test)
    + 	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
      	;;
      vs-build)
     +	setenv --build DEVELOPER $DEVELOPER
23:  b45b7cec94e ! 25:  1268792233f CI: set CC in MAKEFLAGS directly, don't add it to the environment
    @@ ci/lib.sh: setenv () {
      	fi
      }
      
    --# Clear MAKEFLAGS that may come from the outside world.
     +# Clear variables that may come from the outside world.
     +CC=
     +CC_PACKAGE=
    - MAKEFLAGS=
    ++
    + # How many jobs to run in parallel?
    + NPROC=10
      
    - # Common make and cmake build options
    -@@ ci/lib.sh: MAKEFLAGS="DEVELOPER=$DEVELOPER SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
    +@@ ci/lib.sh: MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
      
      case "$CI_TYPE" in
      github-actions)
     -	CC="${CC:-gcc}"
     -
    - 	setenv --test GIT_PROVE_OPTS "--timer --jobs 10"
    + 	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
      	GIT_TEST_OPTS="--verbose-log -x"
    - 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
    + 	test Windows != "$RUNNER_OS" ||
     @@ ci/lib.sh: vs-test)
    - 	setenv --test NO_SVN_TESTS YesPlease
    + 	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
      	;;
      linux-gcc)
     +	CC=gcc
    @@ ci/lib.sh: linux-musl)
      	;;
      esac
      
    --setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
    +-setenv --build MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
     +MAKEFLAGS="$MAKEFLAGS${CC:+ CC=$CC}"
    -+setenv --all MAKEFLAGS "$MAKEFLAGS"
    ++setenv --build MAKEFLAGS "$MAKEFLAGS"
 -:  ----------- > 26:  83138dacd3e CI: set SANITIZE=leak in MAKEFLAGS directly
24:  06bf8d9f61b ! 27:  0508777c68e CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case
    @@ ci/lib.sh: ubuntu-latest)
      	setenv --test GIT_TEST_HTTPD true
      	;;
     -macos-latest)
    --	if [ "$jobname" = osx-gcc ]
    +-	if test "$jobname" = osx-gcc
     -	then
     -		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
     -	else
25:  6dc96ba8b82 ! 28:  c0ce0fd3a80 CI: don't use "set -x" in "ci/lib.sh" output
    @@ Commit message
     
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
    - ## ci/lib-ci-type.sh ##
    -@@ ci/lib-ci-type.sh: else
    - 	env >&2
    - 	exit 1
    - fi
    -+
    -+echo "CONFIG: CI_TYPE=$CI_TYPE" >&2
    -
      ## ci/lib.sh ##
     @@
      #!/bin/sh
    @@ ci/lib.sh
      # Helper libraries
      . ${0%/*}/lib-ci-type.sh
     @@ ci/lib.sh: then
    - 	echo "need a $0 mode, e.g. --build or --test"
      	exit 1
      fi
    -+echo "CONFIG: mode=$mode" >&2
      
    - if test -z "$jobname"
    - then
    - 	echo "must set a CI jobname" >&2
    - 	exit 1
    - fi
    ++# Show our configuration
    ++echo "CONFIG: CI_TYPE=$CI_TYPE" >&2
     +echo "CONFIG: jobname=$jobname" >&2
     +echo "CONFIG: runs_on_pool=$runs_on_pool" >&2
     +echo "CONFIG: GITHUB_ENV=$GITHUB_ENV" >&2
    - 
    ++if test -n "$GITHUB_ENV"
    ++then
    ++	echo "CONFIG: GITHUB_ENV=$GITHUB_ENV" >&2
    ++fi
    ++
      # Helper functions
      setenv () {
    ++	local skip=
    + 	while test $# != 0
    + 	do
    + 		case "$1" in
    + 		--build | --test)
    + 			if test "$1" != "$mode"
    + 			then
    +-				return 0
    ++				skip=t
    + 			fi
    + 
    + 			;;
     @@ ci/lib.sh: setenv () {
    + 	val=$2
    + 	shift 2
      
    - 	if test -n "$skip"
    - 	then
    ++	if test -n "$skip"
    ++	then
     +		echo "SKIP '$key=$val'" >&2
    - 		return 0
    - 	fi
    - 
    -@@ ci/lib.sh: setenv () {
    - 		# itself.
    - 		eval "export $key=\"$val\""
    ++		return 0
    ++	fi
    ++
    + 	if test -n "$GITHUB_ENV"
    + 	then
    + 		echo "$key=$val" >>"$GITHUB_ENV"
      	fi
     +
     +	echo "SET: '$key=$val'" >&2
 -:  ----------- > 29:  2e3c02fa0df CI: make it easy to use ci/*.sh outside of CI
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 01/29] CI: run "set -ex" early in ci/lib.sh

[Ævar Arnfjörð Bjarmason]

Change ci/lib.sh to run "set -ex" before it does anything else, before
this we were omitting a few "export" commands in our trace output that
happened before we enabled tracing.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index cbc2f8f1caa..57141d38a85 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,5 +1,11 @@
 # Library of functions shared by all CI scripts
 
+# Set 'exit on error' for all CI scripts to let the caller know that
+# something went wrong.
+# Set tracing executed commands, primarily setting environment variables
+# and installing dependencies.
+set -ex
+
 skip_branch_tip_with_tag () {
 	# Sometimes, a branch is pushed at the same time the tag that points
 	# at the same commit as the tip of the branch is pushed, and building
@@ -85,12 +91,6 @@ export TERM=${TERM:-dumb}
 # Clear MAKEFLAGS that may come from the outside world.
 export MAKEFLAGS=
 
-# Set 'exit on error' for all CI scripts to let the caller know that
-# something went wrong.
-# Set tracing executed commands, primarily setting environment variables
-# and installing dependencies.
-set -ex
-
 if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
 then
 	CI_TYPE=azure-pipelines
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 02/29] CI: make "$jobname" explicit, remove fallback

[Ævar Arnfjörð Bjarmason]

Remove the $jobname fallback last adjusted in b011fabd6e0 (ci/lib.sh:
encapsulate Travis-specific things, 2019-01-27).

This makes all of our CI jobs have an explicit $jobname, and removes
the special-case of the Windows jobs implicitly setting one (that they
don't currently use). In subsequent commits we'll use this new
$jobname explicitly.

This is a follow-up to my 25715419bf4 (CI: don't run "make test" twice
in one job, 2021-11-23), since then all the jobs except the Windows
jobs have had an explicit $jobname.

In the case of the Windows jobs we were using a jobname of windows-gcc
for both, due to picking "gcc" as a default fallback for "$CC" in
ci/lib.sh.

Since we'll need this to run ci/lib.sh it makes sense to move the
check to the top of the file. Subsequent commits will add more such
assertions to it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml |  8 ++++++++
 ci/lib.sh                  | 12 +++++++-----
 2 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index c35200defb9..309b9141249 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -79,6 +79,8 @@ jobs:
 
   windows-build:
     name: win build
+    env:
+      jobname: windows-build
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
     runs-on: windows-latest
@@ -100,6 +102,8 @@ jobs:
         path: artifacts
   windows-test:
     name: win test
+    env:
+      jobname: windows-test
     runs-on: windows-latest
     needs: [windows-build]
     strategy:
@@ -131,6 +135,8 @@ jobs:
         path: ${{env.FAILED_TEST_ARTIFACTS}}
   vs-build:
     name: win+VS build
+    env:
+      jobname: vs-build
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
     env:
@@ -183,6 +189,8 @@ jobs:
         path: artifacts
   vs-test:
     name: win+VS test
+    env:
+      jobname: vs-test
     runs-on: windows-latest
     needs: vs-build
     strategy:
diff --git a/ci/lib.sh b/ci/lib.sh
index 57141d38a85..5fda166ad47 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -6,6 +6,13 @@
 # and installing dependencies.
 set -ex
 
+# Starting assertions
+if test -z "$jobname"
+then
+	echo "must set a CI jobname in the environment" >&2
+	exit 1
+fi
+
 skip_branch_tip_with_tag () {
 	# Sometimes, a branch is pushed at the same time the tag that points
 	# at the same commit as the tip of the branch is pushed, and building
@@ -146,11 +153,6 @@ test -n "${DONT_SKIP_TAGS-}" ||
 skip_branch_tip_with_tag
 skip_good_tree
 
-if test -z "$jobname"
-then
-	jobname="$CI_OS_NAME-$CC"
-fi
-
 export DEVELOPER=1
 export DEFAULT_TEST_TARGET=prove
 export GIT_TEST_CLONE_2GB=true
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 03/29] CI: remove more dead Travis CI support

[Ævar Arnfjörð Bjarmason]

Remove code that I missed in 4a6e4b96026 (CI: remove Travis CI
support, 2021-11-23). This code was only called from or used by the
now-removed .travis.yml, or needed by the Travis CI environment.

For the symlinking in ci/run-build-and-tests.sh: Back when
3c93b829205 (travis-ci: build Git during the 'script' phase,
2018-01-08) and 4b060a4d973 (ci: use a junction on Windows instead of
a symlink, 2019-01-27) adjusted this "ln" command, the Windows build
would use ci/run-build-and-tests.sh.

As seen in 889cacb6897 (ci: configure GitHub Actions for CI/PR,
2020-04-11) the current windows build uses a different entry point
under the GitHub CI, which doesn't use this .prove caching. Namely
"ci/run-test-slice.sh".

We can be certain that it's never used in "ci/run-test-slice.sh"
because to have a ".prove" file we'd need to use "--state=save", which
we only do in the dead Azure codepath in ci/lib.sh. If it were used it
would do the wrong thing, because the different test slices would each
try to clobber the same "t/.prove" file.

If a subsequent run then used the -"-state=failed,slow,save" it would
defeat the purpose of "ci/run-test-slice.sh", since all slices would
then run all tests. I.e. behavior of prove's "--state" options is to
select tests to run from the provided "--state" file, in addition to
those specified on the command-line.

For ci/run-docker{,-build}.sh: It was likewise last referenced in the
.travis.yml removed in my 4a6e4b96026. The current "dockerized" run in
".github/workflows/main.yml" calls the same entry points as the main
"regular" job.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                     | 10 ++----
 ci/print-test-failures.sh     |  3 --
 ci/run-build-and-tests.sh     |  5 ---
 ci/run-docker-build.sh        | 66 -----------------------------------
 ci/run-docker.sh              | 47 -------------------------
 ci/run-test-slice.sh          |  5 ---
 ci/util/extract-trash-dirs.sh | 50 --------------------------
 7 files changed, 2 insertions(+), 184 deletions(-)
 delete mode 100755 ci/run-docker-build.sh
 delete mode 100755 ci/run-docker.sh
 delete mode 100755 ci/util/extract-trash-dirs.sh

diff --git a/ci/lib.sh b/ci/lib.sh
index 5fda166ad47..510b20726ab 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -106,9 +106,6 @@ then
 	CI_COMMIT="$BUILD_SOURCEVERSION"
 	CI_JOB_ID="$BUILD_BUILDID"
 	CI_JOB_NUMBER="$BUILD_BUILDNUMBER"
-	CI_OS_NAME="$(echo "$AGENT_OS" | tr A-Z a-z)"
-	test darwin != "$CI_OS_NAME" || CI_OS_NAME=osx
-	CI_REPO_SLUG="$(expr "$BUILD_REPOSITORY_URI" : '.*/\([^/]*/[^/]*\)$')"
 	CC="${CC:-gcc}"
 
 	# use a subdirectory of the cache dir (because the file share is shared
@@ -118,16 +115,13 @@ then
 	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
-	test windows_nt != "$CI_OS_NAME" ||
+	test Windows_NT != "$AGENT_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 elif test true = "$GITHUB_ACTIONS"
 then
 	CI_TYPE=github-actions
 	CI_BRANCH="$GITHUB_REF"
 	CI_COMMIT="$GITHUB_SHA"
-	CI_OS_NAME="$(echo "$RUNNER_OS" | tr A-Z a-z)"
-	test macos != "$CI_OS_NAME" || CI_OS_NAME=osx
-	CI_REPO_SLUG="$GITHUB_REPOSITORY"
 	CI_JOB_ID="$GITHUB_RUN_ID"
 	CC="${CC:-gcc}"
 	DONT_SKIP_TAGS=t
@@ -137,7 +131,7 @@ then
 	export GIT_PROVE_OPTS="--timer --jobs 10"
 	export GIT_TEST_OPTS="--verbose-log -x"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
-	test windows != "$CI_OS_NAME" ||
+	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 else
 	echo "Could not identify CI type" >&2
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 57277eefcd0..00fcf27b1b2 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -77,9 +77,6 @@ do
 			fi
 			combined_trash_size=$new_combined_trash_size
 
-			# DO NOT modify these two 'echo'-ed strings below
-			# without updating 'ci/util/extract-trash-dirs.sh'
-			# as well.
 			echo "$(tput setaf 1)Start of trash directory of '$test_name':$(tput sgr0)"
 			cat "$trash_tgz_b64"
 			echo "$(tput setaf 1)End of trash directory of '$test_name'$(tput sgr0)"
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 280dda7d285..0a1ec8c2bae 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -5,11 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-case "$CI_OS_NAME" in
-windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
-*) ln -s "$cache_dir/.prove" t/.prove;;
-esac
-
 export MAKE_TARGETS="all test"
 
 case "$jobname" in
diff --git a/ci/run-docker-build.sh b/ci/run-docker-build.sh
deleted file mode 100755
index 6cd832efb9c..00000000000
--- a/ci/run-docker-build.sh
+++ /dev/null
@@ -1,66 +0,0 @@
-#!/bin/sh
-#
-# Build and test Git inside container
-#
-# Usage:
-#   run-docker-build.sh <host-user-id>
-#
-
-set -ex
-
-if test $# -ne 1 || test -z "$1"
-then
-	echo >&2 "usage: run-docker-build.sh <host-user-id>"
-	exit 1
-fi
-
-case "$jobname" in
-linux32)
-	switch_cmd="linux32 --32bit i386"
-	;;
-linux-musl)
-	switch_cmd=
-	useradd () { adduser -D "$@"; }
-	;;
-*)
-	exit 1
-	;;
-esac
-
-"${0%/*}/install-docker-dependencies.sh"
-
-# If this script runs inside a docker container, then all commands are
-# usually executed as root. Consequently, the host user might not be
-# able to access the test output files.
-# If a non 0 host user id is given, then create a user "ci" with that
-# user id to make everything accessible to the host user.
-HOST_UID=$1
-if test $HOST_UID -eq 0
-then
-	# Just in case someone does want to run the test suite as root.
-	CI_USER=root
-else
-	CI_USER=ci
-	if test "$(id -u $CI_USER 2>/dev/null)" = $HOST_UID
-	then
-		echo "user '$CI_USER' already exists with the requested ID $HOST_UID"
-	else
-		useradd -u $HOST_UID $CI_USER
-	fi
-fi
-
-# Build and test
-command $switch_cmd su -m -l $CI_USER -c "
-	set -ex
-	export DEVELOPER='$DEVELOPER'
-	export DEFAULT_TEST_TARGET='$DEFAULT_TEST_TARGET'
-	export GIT_PROVE_OPTS='$GIT_PROVE_OPTS'
-	export GIT_TEST_OPTS='$GIT_TEST_OPTS'
-	export GIT_TEST_CLONE_2GB='$GIT_TEST_CLONE_2GB'
-	export MAKEFLAGS='$MAKEFLAGS'
-	export cache_dir='$cache_dir'
-	cd /usr/src/git
-	test -n '$cache_dir' && ln -s '$cache_dir/.prove' t/.prove
-	make
-	make test
-"
diff --git a/ci/run-docker.sh b/ci/run-docker.sh
deleted file mode 100755
index af89d1624a4..00000000000
--- a/ci/run-docker.sh
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/bin/sh
-#
-# Download and run Docker image to build and test Git
-#
-
-. ${0%/*}/lib.sh
-
-case "$jobname" in
-linux32)
-	CI_CONTAINER="daald/ubuntu32:xenial"
-	;;
-linux-musl)
-	CI_CONTAINER=alpine
-	;;
-*)
-	exit 1
-	;;
-esac
-
-docker pull "$CI_CONTAINER"
-
-# Use the following command to debug the docker build locally:
-# <host-user-id> must be 0 if podman is used as drop-in replacement for docker
-# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/sh "$CI_CONTAINER"
-# root@container:/# export jobname=<jobname>
-# root@container:/# /usr/src/git/ci/run-docker-build.sh <host-user-id>
-
-container_cache_dir=/tmp/container-cache
-
-docker run \
-	--interactive \
-	--env DEVELOPER \
-	--env DEFAULT_TEST_TARGET \
-	--env GIT_PROVE_OPTS \
-	--env GIT_TEST_OPTS \
-	--env GIT_TEST_CLONE_2GB \
-	--env MAKEFLAGS \
-	--env jobname \
-	--env cache_dir="$container_cache_dir" \
-	--volume "${PWD}:/usr/src/git" \
-	--volume "$cache_dir:$container_cache_dir" \
-	"$CI_CONTAINER" \
-	/usr/src/git/ci/run-docker-build.sh $(id -u $USER)
-
-check_unignored_build_artifacts
-
-save_good_tree
diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index f8c2c3106a2..b9a682b4bcd 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -5,11 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-case "$CI_OS_NAME" in
-windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
-*) ln -s "$cache_dir/.prove" t/.prove;;
-esac
-
 make --quiet -C t T="$(cd t &&
 	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
 	tr '\n' ' ')"
diff --git a/ci/util/extract-trash-dirs.sh b/ci/util/extract-trash-dirs.sh
deleted file mode 100755
index 8e67bec21a2..00000000000
--- a/ci/util/extract-trash-dirs.sh
+++ /dev/null
@@ -1,50 +0,0 @@
-#!/bin/sh
-
-error () {
-	echo >&2 "error: $@"
-	exit 1
-}
-
-find_embedded_trash () {
-	while read -r line
-	do
-		case "$line" in
-		*Start\ of\ trash\ directory\ of\ \'t[0-9][0-9][0-9][0-9]-*\':*)
-			test_name="${line#*\'}"
-			test_name="${test_name%\'*}"
-
-			return 0
-		esac
-	done
-
-	return 1
-}
-
-extract_embedded_trash () {
-	while read -r line
-	do
-		case "$line" in
-		*End\ of\ trash\ directory\ of\ \'$test_name\'*)
-			return
-			;;
-		*)
-			printf '%s\n' "$line"
-			;;
-		esac
-	done
-
-	error "unexpected end of input"
-}
-
-# Raw logs from Linux build jobs have CRLF line endings, while OSX
-# build jobs mostly have CRCRLF, except an odd line every now and
-# then that has CRCRCRLF.  'base64 -d' from 'coreutils' doesn't like
-# CRs and complains about "invalid input", so remove all CRs at the
-# end of lines.
-sed -e 's/\r*$//' | \
-while find_embedded_trash
-do
-	echo "Extracting trash directory of '$test_name'"
-
-	extract_embedded_trash |base64 -d |tar xzp
-done
-- 
2.36.0.rc2.843.g193535c2aa7

Re: [PATCH v3 03/29] CI: remove more dead Travis CI support

[Eric Sunshine]

On Wed, Apr 13, 2022 at 5:13 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> Remove code that I missed in 4a6e4b96026 (CI: remove Travis CI
> support, 2021-11-23). This code was only called from or used by the
> now-removed .travis.yml, or needed by the Travis CI environment.
>
> For the symlinking in ci/run-build-and-tests.sh: Back when
> 3c93b829205 (travis-ci: build Git during the 'script' phase,
> 2018-01-08) and 4b060a4d973 (ci: use a junction on Windows instead of
> a symlink, 2019-01-27) adjusted this "ln" command, the Windows build
> would use ci/run-build-and-tests.sh.
>
> As seen in 889cacb6897 (ci: configure GitHub Actions for CI/PR,
> 2020-04-11) the current windows build uses a different entry point
> under the GitHub CI, which doesn't use this .prove caching. Namely
> "ci/run-test-slice.sh".
>
> We can be certain that it's never used in "ci/run-test-slice.sh"
> because to have a ".prove" file we'd need to use "--state=save", which
> we only do in the dead Azure codepath in ci/lib.sh. If it were used it
> would do the wrong thing, because the different test slices would each
> try to clobber the same "t/.prove" file.
>
> If a subsequent run then used the -"-state=failed,slow,save" it would

s/-"-/--"/

> defeat the purpose of "ci/run-test-slice.sh", since all slices would
> then run all tests. I.e. behavior of prove's "--state" options is to
> select tests to run from the provided "--state" file, in addition to
> those specified on the command-line.
>
> For ci/run-docker{,-build}.sh: It was likewise last referenced in the
> .travis.yml removed in my 4a6e4b96026. The current "dockerized" run in
> ".github/workflows/main.yml" calls the same entry points as the main
> "regular" job.
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

[PATCH v3 04/29] CI: remove dead "tree skipping" code

[Ævar Arnfjörð Bjarmason]

Remove the code related to skipping CI runs if the tree was
successfully tested before. With the GitHub CI this is done with the
"ci-config" step, see e76eec35540 (ci: allow per-branch config for
GitHub Actions, 2020-05-07) and 7d78d5fc1a9 (ci: skip GitHub workflow
runs for already-tested commits/trees, 2020-10-08).

This code hasn't been used since 4a6e4b96026 (CI: remove Travis CI
support, 2021-11-23), and before that for the retired Azure pipeline
support removed in 6081d3898fe (ci: retire the Azure Pipelines
definition, 2020-04-11).

This change is needed because this and subsequent commits are turning
"ci/lib.sh" into a dumber library that'll only be tasked with setting
variables for CI jobs, or for specific steps within those jobs. Now we
don't need to worry about it potentially skipping the run on its own.

This change also removes a subtle potential logic error introduced in
0e7696c64db (ci: disallow directional formatting, 2021-11-04). The
"ci/check-directional-formatting.bash" script would have been made to
run after the "save_good_tree" in invoked in
"ci/run-static-analysis.sh". So if e.g. the "make coccicheck" failed
we'd still mark the tree as good.

I.e. the addition of "ci/check-directional-formatting.bash" didn't
take into account that the various users of the "save_good_tree"
function in ci/*.sh made a hard assumption that they're the only
ci/*.sh script that's being run (and they use "set -e").

This is entirely academic since we weren't actually running this code,
but is something to be careful of if anyone ever needs to resurrect
parts of this.

Finally, the cache_dir="$HOME/none" added in a3f2eec862b (ci/lib:
allow running in GitHub Actions, 2020-04-08) for the GitHub CI can be
removed. It appears to have been added to appease the subsequent
'mkdir -p "$cache_dir"', which wasn't needed in GitHub CI.

1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                 | 91 ---------------------------------------
 ci/run-build-and-tests.sh |  2 -
 ci/run-static-analysis.sh |  2 -
 ci/test-documentation.sh  |  2 -
 4 files changed, 97 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 510b20726ab..d12eaf338f9 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -13,75 +13,6 @@ then
 	exit 1
 fi
 
-skip_branch_tip_with_tag () {
-	# Sometimes, a branch is pushed at the same time the tag that points
-	# at the same commit as the tip of the branch is pushed, and building
-	# both at the same time is a waste.
-	#
-	# When the build is triggered by a push to a tag, $CI_BRANCH will
-	# have that tagname, e.g. v2.14.0.  Let's see if $CI_BRANCH is
-	# exactly at a tag, and if so, if it is different from $CI_BRANCH.
-	# That way, we can tell if we are building the tip of a branch that
-	# is tagged and we can skip the build because we won't be skipping a
-	# build of a tag.
-
-	if TAG=$(git describe --exact-match "$CI_BRANCH" 2>/dev/null) &&
-		test "$TAG" != "$CI_BRANCH"
-	then
-		echo "$(tput setaf 2)Tip of $CI_BRANCH is exactly at $TAG$(tput sgr0)"
-		exit 0
-	fi
-}
-
-# Save some info about the current commit's tree, so we can skip the build
-# job if we encounter the same tree again and can provide a useful info
-# message.
-save_good_tree () {
-	echo "$(git rev-parse $CI_COMMIT^{tree}) $CI_COMMIT $CI_JOB_NUMBER $CI_JOB_ID" >>"$good_trees_file"
-	# limit the file size
-	tail -1000 "$good_trees_file" >"$good_trees_file".tmp
-	mv "$good_trees_file".tmp "$good_trees_file"
-}
-
-# Skip the build job if the same tree has already been built and tested
-# successfully before (e.g. because the branch got rebased, changing only
-# the commit messages).
-skip_good_tree () {
-	if test true = "$GITHUB_ACTIONS"
-	then
-		return
-	fi
-
-	if ! good_tree_info="$(grep "^$(git rev-parse $CI_COMMIT^{tree}) " "$good_trees_file")"
-	then
-		# Haven't seen this tree yet, or no cached good trees file yet.
-		# Continue the build job.
-		return
-	fi
-
-	echo "$good_tree_info" | {
-		read tree prev_good_commit prev_good_job_number prev_good_job_id
-
-		if test "$CI_JOB_ID" = "$prev_good_job_id"
-		then
-			cat <<-EOF
-			$(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0)
-			This commit has already been built and tested successfully by this build job.
-			To force a re-build delete the branch's cache and then hit 'Restart job'.
-			EOF
-		else
-			cat <<-EOF
-			$(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0)
-			This commit's tree has already been built and tested successfully in build job $prev_good_job_number for commit $prev_good_commit.
-			The log of that build job is available at $SYSTEM_TASKDEFINITIONSURI$SYSTEM_TEAMPROJECT/_build/results?buildId=$prev_good_job_id
-			To force a re-build delete the branch's cache and then hit 'Restart job'.
-			EOF
-		fi
-	}
-
-	exit 0
-}
-
 check_unignored_build_artifacts ()
 {
 	! git ls-files --other --exclude-standard --error-unmatch \
@@ -102,16 +33,8 @@ if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
 then
 	CI_TYPE=azure-pipelines
 	# We are running in Azure Pipelines
-	CI_BRANCH="$BUILD_SOURCEBRANCH"
-	CI_COMMIT="$BUILD_SOURCEVERSION"
-	CI_JOB_ID="$BUILD_BUILDID"
-	CI_JOB_NUMBER="$BUILD_BUILDNUMBER"
 	CC="${CC:-gcc}"
 
-	# use a subdirectory of the cache dir (because the file share is shared
-	# among *all* phases)
-	cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME"
-
 	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
@@ -120,13 +43,7 @@ then
 elif test true = "$GITHUB_ACTIONS"
 then
 	CI_TYPE=github-actions
-	CI_BRANCH="$GITHUB_REF"
-	CI_COMMIT="$GITHUB_SHA"
-	CI_JOB_ID="$GITHUB_RUN_ID"
 	CC="${CC:-gcc}"
-	DONT_SKIP_TAGS=t
-
-	cache_dir="$HOME/none"
 
 	export GIT_PROVE_OPTS="--timer --jobs 10"
 	export GIT_TEST_OPTS="--verbose-log -x"
@@ -139,14 +56,6 @@ else
 	exit 1
 fi
 
-good_trees_file="$cache_dir/good-trees"
-
-mkdir -p "$cache_dir"
-
-test -n "${DONT_SKIP_TAGS-}" ||
-skip_branch_tip_with_tag
-skip_good_tree
-
 export DEVELOPER=1
 export DEFAULT_TEST_TARGET=prove
 export GIT_TEST_CLONE_2GB=true
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 0a1ec8c2bae..e5a21985b44 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -45,5 +45,3 @@ esac
 # start running tests.
 make $MAKE_TARGETS
 check_unignored_build_artifacts
-
-save_good_tree
diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh
index 65bcebda41a..5a87b1cac96 100755
--- a/ci/run-static-analysis.sh
+++ b/ci/run-static-analysis.sh
@@ -28,5 +28,3 @@ fi
 
 make hdr-check ||
 exit 1
-
-save_good_tree
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index de41888430a..9e0652c30dd 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -41,5 +41,3 @@ grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
 
 rm -f stdout.log stderr.log stderr.raw
 check_unignored_build_artifacts
-
-save_good_tree
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 05/29] CI: remove unused Azure ci/* code

[Ævar Arnfjörð Bjarmason]

Remove Azure-specific code that's been unused since 6081d3898fe (ci:
retire the Azure Pipelines definition, 2020-04-11). As noted in a
larger removal of all of the Azure-supporting code in [1] (although
that missed some of this) there is more of it in-tree, but let's focus
on only the ci/* code for now.

This is needed because in subsequent commits this unused code would
either need to be changed to accommodate changes in ci/*, or be
removed.

As we'll see in those subsequent commits the end-state we're heading
towards will actually make it easier to add new CI types in the
future, even though the only one we're left with now is the GitHub
CI. I.e. the "ci/*" framework will be made to do less, not more. We'll
be offloading more of what it does to our generic build and test
system.

While I'm at it (since the line needs to be touched anyway) change an
odd 'if test true == "$GITHUB_ACTIONS"' to the more usual style used
in other places of 'if test "$GITHUB_ACTIONS" = "true"'.

1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/install-dependencies.sh |  3 ---
 ci/lib.sh                  | 13 +------------
 ci/mount-fileshare.sh      | 25 -------------------------
 ci/print-test-failures.sh  |  5 -----
 4 files changed, 1 insertion(+), 45 deletions(-)
 delete mode 100755 ci/mount-fileshare.sh

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index dbcebad2fb2..e7ea8799411 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -34,8 +34,6 @@ macos-latest)
 	export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
 	# Uncomment this if you want to run perf tests:
 	# brew install gnu-time
-	test -z "$BREW_INSTALL_PACKAGES" ||
-	brew install $BREW_INSTALL_PACKAGES
 	brew link --force gettext
 	brew install --cask --no-quarantine perforce || {
 		# Update the definitions and try again
@@ -69,7 +67,6 @@ Documentation)
 	sudo apt-get -q update
 	sudo apt-get -q -y install asciidoc xmlto docbook-xsl-ns make
 
-	test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
 	sudo gem install --version 1.5.8 asciidoctor
 	;;
 linux-gcc-default)
diff --git a/ci/lib.sh b/ci/lib.sh
index d12eaf338f9..c4b829c1eed 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -29,18 +29,7 @@ export TERM=${TERM:-dumb}
 # Clear MAKEFLAGS that may come from the outside world.
 export MAKEFLAGS=
 
-if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
-then
-	CI_TYPE=azure-pipelines
-	# We are running in Azure Pipelines
-	CC="${CC:-gcc}"
-
-	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
-	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
-	MAKEFLAGS="$MAKEFLAGS --jobs=10"
-	test Windows_NT != "$AGENT_OS" ||
-	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
-elif test true = "$GITHUB_ACTIONS"
+if test "$GITHUB_ACTIONS" = "true"
 then
 	CI_TYPE=github-actions
 	CC="${CC:-gcc}"
diff --git a/ci/mount-fileshare.sh b/ci/mount-fileshare.sh
deleted file mode 100755
index 26b58a80960..00000000000
--- a/ci/mount-fileshare.sh
+++ /dev/null
@@ -1,25 +0,0 @@
-#!/bin/sh
-
-die () {
-	echo "$*" >&2
-	exit 1
-}
-
-test $# = 4 ||
-die "Usage: $0 <share> <username> <password> <mountpoint>"
-
-mkdir -p "$4" || die "Could not create $4"
-
-case "$(uname -s)" in
-Linux)
-	sudo mount -t cifs -o vers=3.0,username="$2",password="$3",dir_mode=0777,file_mode=0777,serverino "$1" "$4"
-	;;
-Darwin)
-	pass="$(echo "$3" | sed -e 's/\//%2F/g' -e 's/+/%2B/g')" &&
-	mount -t smbfs,soft "smb://$2:$pass@${1#//}" "$4"
-	;;
-*)
-	die "No support for $(uname -s)"
-	;;
-esac ||
-die "Could not mount $4"
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 00fcf27b1b2..a878cca67e4 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -39,11 +39,6 @@ do
 		test_name="${test_name##*/}"
 		trash_dir="trash directory.$test_name"
 		case "$CI_TYPE" in
-		azure-pipelines)
-			mkdir -p failed-test-artifacts
-			mv "$trash_dir" failed-test-artifacts
-			continue
-			;;
 		github-actions)
 			mkdir -p failed-test-artifacts
 			echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 06/29] CI/lib.sh: stop adding leading whitespace to $MAKEFLAGS

[Ævar Arnfjörð Bjarmason]

In a8c51f77d16 (ci: clear and mark MAKEFLAGS exported just once,
2019-02-07) the assignment of MAKEFLAGS was done up-front to
future-proof this code for the bug fixed in 406f93ae488 (ci: make sure
we build Git parallel, 2019-02-07).

But since 4a6e4b96026 (CI: remove Travis CI support, 2021-11-23) the
--jobs=10 has been an invariant here, so let's assign --jobs=$NPROC to
the MAKEFLAGS right away, this gets rid of the leading whitespace from
the $MAKEFLAGS variable, as it would previously be empty on the first
assignment.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index c4b829c1eed..21c009757f4 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -26,17 +26,19 @@ check_unignored_build_artifacts ()
 # GitHub Action doesn't set TERM, which is required by tput
 export TERM=${TERM:-dumb}
 
+# How many jobs to run in parallel?
+NPROC=10
+
 # Clear MAKEFLAGS that may come from the outside world.
-export MAKEFLAGS=
+export MAKEFLAGS=--jobs=$NPROC
 
 if test "$GITHUB_ACTIONS" = "true"
 then
 	CI_TYPE=github-actions
 	CC="${CC:-gcc}"
 
-	export GIT_PROVE_OPTS="--timer --jobs 10"
+	export GIT_PROVE_OPTS="--timer --jobs $NPROC"
 	export GIT_TEST_OPTS="--verbose-log -x"
-	MAKEFLAGS="$MAKEFLAGS --jobs=10"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 else
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 07/29] CI: don't have "git grep" invoke a pager in tree content check

[Ævar Arnfjörð Bjarmason]

Fix the omission of not adding "-P" (a.k.a. "--no-pager") to the "git"
invocation that spaws "git grep", added in 0e7696c64db (ci: disallow
directional formatting, 2021-11-04).

This didn't matter for the GitHub CI, but does when running the check
locally. A subsequent commit will start invoking it via a new check
from the top-level Makefile, and without this we'd visibly spawn a
pager when doing so.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/check-directional-formatting.bash | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ci/check-directional-formatting.bash b/ci/check-directional-formatting.bash
index e6211b141aa..de8e6e486f0 100755
--- a/ci/check-directional-formatting.bash
+++ b/ci/check-directional-formatting.bash
@@ -23,5 +23,5 @@
 # U+2066..U+2069: LRI, RLI, FSI and PDI
 regex='(\u202a|\u202b|\u202c|\u202d|\u202e|\u2066|\u2067|\u2068|\u2069)'
 
-! LC_CTYPE=C git grep -El "$(LC_CTYPE=C.UTF-8 printf "$regex")" \
+! LC_CTYPE=C git -P grep -El "$(LC_CTYPE=C.UTF-8 printf "$regex")" \
 	-- ':(exclude,attr:binary)' ':(exclude)*.po'
-- 
2.36.0.rc2.843.g193535c2aa7

Re: [PATCH v3 07/29] CI: don't have "git grep" invoke a pager in tree content check

[Eric Sunshine]

On Wed, Apr 13, 2022 at 5:33 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> Fix the omission of not adding "-P" (a.k.a. "--no-pager") to the "git"
> invocation that spaws "git grep", added in 0e7696c64db (ci: disallow
> directional formatting, 2021-11-04).

s/spaws/spawns/

> This didn't matter for the GitHub CI, but does when running the check
> locally. A subsequent commit will start invoking it via a new check
> from the top-level Makefile, and without this we'd visibly spawn a
> pager when doing so.
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

[PATCH v3 08/29] CI: have "static-analysis" run a "make ci-static-analysis" target

[Ævar Arnfjörð Bjarmason]

Make it easier to run the CI logic locally by creating
"ci-static-analysis" target.

Now the "check-coccicheck" logic added in 0860a7641ba (travis-ci: fail
if Coccinelle static analysis found something to transform,
2018-07-23) can be easily tested outside of CI, and likewise the
sanity check added in 0e7696c64db (ci: disallow directional
formatting, 2021-11-04).

These targets could be improved, the "hdr-check" target will re-do all
of its work every time it's run, and could with a change similar to my
c234e8a0ecf (Makefile: make the "sparse" target non-.PHONY,
2021-09-23). The same goes for the new
"ci-check-directional-formatting" target.

But without those improvements being able to easily run a 1=1
equivalent of the checks we do in CI during a local build is already
an improvement. Subsequent commits will expand on this ability.

This change will also make the CI check faster, since we can take
advantage of parallelism. The "make coccicheck" takes a very long
time, and at the tail end we might only have one job pegging 100% of
one CPU core for a long time. Now any extra cores will be free to run
the rest of the targets under "ci-static-analysis".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml |  3 +--
 Makefile                   | 28 ++++++++++++++++++++++++++++
 ci/run-static-analysis.sh  | 30 ------------------------------
 shared.mak                 |  1 +
 4 files changed, 30 insertions(+), 32 deletions(-)
 delete mode 100755 ci/run-static-analysis.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 309b9141249..ad752010102 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -317,8 +317,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/run-static-analysis.sh
-    - run: ci/check-directional-formatting.bash
+    - run: make ci-static-analysis
   sparse:
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
diff --git a/Makefile b/Makefile
index f8bccfab5e9..8717ea7e745 100644
--- a/Makefile
+++ b/Makefile
@@ -3031,6 +3031,20 @@ coccicheck: $(addsuffix .patch,$(filter-out %.pending.cocci,$(wildcard contrib/c
 # See contrib/coccinelle/README
 coccicheck-pending: $(addsuffix .patch,$(wildcard contrib/coccinelle/*.pending.cocci))
 
+.PHONY: check-coccicheck
+check-coccicheck: coccicheck
+	$(QUIET_CHECK)for cocci_patch in contrib/coccinelle/*.patch; do \
+		if test -s "$$cocci_patch"; then \
+			echo "Coccinelle suggests the following changes in '$$cocci_patch':"; \
+			cat "$$cocci_patch"; \
+			fail=UnfortunatelyYes; \
+		fi \
+	done; \
+	if test -n "$$fail"; then \
+		echo "error: Coccinelle suggested some changes"; \
+		exit 1; \
+	fi
+
 .PHONY: coccicheck coccicheck-pending
 
 ### Installation rules
@@ -3466,3 +3480,17 @@ $(FUZZ_PROGRAMS): all
 		$(XDIFF_OBJS) $(EXTLIBS) git.o $@.o $(LIB_FUZZING_ENGINE) -o $@
 
 fuzz-all: $(FUZZ_PROGRAMS)
+
+### CI "check" targets
+#
+# These targets are run from the CI, see .github/workflows/main.yml,
+# but can also be run manually to run the same assertions locally.
+
+.PHONY: ci-check-directional-formatting
+ci-check-directional-formatting:
+	$(QUIET_CHECK)ci/check-directional-formatting.bash
+
+.PHONY: ci-static-analysis
+ci-static-analysis: ci-check-directional-formatting
+ci-static-analysis: check-coccicheck
+ci-static-analysis: hdr-check
diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh
deleted file mode 100755
index 5a87b1cac96..00000000000
--- a/ci/run-static-analysis.sh
+++ /dev/null
@@ -1,30 +0,0 @@
-#!/bin/sh
-#
-# Perform various static code analysis checks
-#
-
-. ${0%/*}/lib.sh
-
-make coccicheck
-
-set +x
-
-fail=
-for cocci_patch in contrib/coccinelle/*.patch
-do
-	if test -s "$cocci_patch"
-	then
-		echo "$(tput setaf 1)Coccinelle suggests the following changes in '$cocci_patch':$(tput sgr0)"
-		cat "$cocci_patch"
-		fail=UnfortunatelyYes
-	fi
-done
-
-if test -n "$fail"
-then
-	echo "$(tput setaf 1)error: Coccinelle suggested some changes$(tput sgr0)"
-	exit 1
-fi
-
-make hdr-check ||
-exit 1
diff --git a/shared.mak b/shared.mak
index 50d4596f0d6..83e0aa16e7d 100644
--- a/shared.mak
+++ b/shared.mak
@@ -68,6 +68,7 @@ ifndef V
 	QUIET_HDR      = @echo '   ' HDR $(<:hcc=h);
 	QUIET_RC       = @echo '   ' RC $@;
 	QUIET_SPATCH   = @echo '   ' SPATCH $<;
+	QUIET_CHECK    = @echo '   ' CHECK $@;
 
 ## Used in "Documentation/Makefile"
 	QUIET_ASCIIDOC	= @echo '   ' ASCIIDOC $@;
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 09/29] CI: have "static-analysis" run "check-builtins", not "documentation"

[Ævar Arnfjörð Bjarmason]

Move the "make check-builtins" check from the "documentation" job to
the "static-analysis" job.

The "check-builtins" target added in c74390e4a1d (cherry is built-in,
do not ship git-cherry.sh, 2006-11-05) is unrelated to the
documentation, so it's odd that b98712b9aa9 (travis-ci: build
documentation, 2016-05-04) added it to the "documentation" job.

Let's just move it to the "ci-static-analysis" target, and while we're
at it improve the output with $(QUIET_CHECK).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile                 | 3 ++-
 ci/test-documentation.sh | 1 -
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 8717ea7e745..2e1ae6c6067 100644
--- a/Makefile
+++ b/Makefile
@@ -3402,7 +3402,7 @@ check-docs::
 ### Make sure built-ins do not have dups and listed in git.c
 #
 check-builtins::
-	./check-builtins.sh
+	$(QUIET_CHECK)./check-builtins.sh
 
 ### Test suite coverage testing
 #
@@ -3492,5 +3492,6 @@ ci-check-directional-formatting:
 
 .PHONY: ci-static-analysis
 ci-static-analysis: ci-check-directional-formatting
+ci-static-analysis: check-builtins
 ci-static-analysis: check-coccicheck
 ci-static-analysis: hdr-check
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 9e0652c30dd..41e2b126311 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -14,7 +14,6 @@ filter_log () {
 	    "$1"
 }
 
-make check-builtins
 make check-docs
 
 # Build docs with AsciiDoc
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 10/29] CI: move p4 and git-lfs variables to ci/install-dependencies.sh

[Ævar Arnfjörð Bjarmason]

Move the declaration of variables that are only used by the
"ubuntu-latest" block in "ci/install-dependencies.sh" there from
"ci/lib.sh".

This makes the code easier to follow, and changes "ci/lib.sh" to a
library that only exports CI variables for general use, we didn't need
to export these $P4_PATH and $GIT_LFS_PATH variables.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/install-dependencies.sh | 17 +++++++++++++++--
 ci/lib.sh                  | 12 ------------
 2 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index e7ea8799411..34b4400f38b 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -5,14 +5,27 @@
 
 . ${0%/*}/lib.sh
 
-P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
-LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
 UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
  tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
  libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl"
 
 case "$runs_on_pool" in
 ubuntu-latest)
+	# The Linux build installs the defined dependency versions below.
+	# The OS X build installs much more recent versions, whichever
+	# were recorded in the Homebrew database upon creating the OS X
+	# image.
+	# Keep that in mind when you encounter a broken OS X build!
+	LINUX_P4_VERSION="16.2"
+	LINUX_GIT_LFS_VERSION="1.5.2"
+
+	P4_PATH="$HOME/custom/p4"
+	GIT_LFS_PATH="$HOME/custom/git-lfs"
+	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
+
+	P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
+	LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
+
 	sudo apt-get -q update
 	sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \
 		$UBUNTU_COMMON_PKGS $CC_PACKAGE
diff --git a/ci/lib.sh b/ci/lib.sh
index 21c009757f4..5736a4d2e04 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -67,18 +67,6 @@ ubuntu-latest)
 	fi
 
 	export GIT_TEST_HTTPD=true
-
-	# The Linux build installs the defined dependency versions below.
-	# The OS X build installs much more recent versions, whichever
-	# were recorded in the Homebrew database upon creating the OS X
-	# image.
-	# Keep that in mind when you encounter a broken OS X build!
-	export LINUX_P4_VERSION="16.2"
-	export LINUX_GIT_LFS_VERSION="1.5.2"
-
-	P4_PATH="$HOME/custom/p4"
-	GIT_LFS_PATH="$HOME/custom/git-lfs"
-	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
 	;;
 macos-latest)
 	if [ "$jobname" = osx-gcc ]
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 11/29] CI: consistently use "export" in ci/lib.sh

[Ævar Arnfjörð Bjarmason]

Change the "ci/lib.sh" script co consistently use "export", for
e.g. MAKEFLAGS we were exporting it, and then assigning to it, let's
do it the other way around.

Right now this doesn't matter, since we in
e.g. "ci/install-dependencies.sh" source this file, and don't use
something like "env(1)" to retrieve these variables. But in a
subsequent commit we'll "export" these variables through a wrapper (to
additionally write them to a GitHub CI-specific $GITHUB_ENV
file). This change makes that subsequent change easier to read, as it
won't need to do any control flow refactoring.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 5736a4d2e04..5783eed1b20 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -30,7 +30,7 @@ export TERM=${TERM:-dumb}
 NPROC=10
 
 # Clear MAKEFLAGS that may come from the outside world.
-export MAKEFLAGS=--jobs=$NPROC
+MAKEFLAGS=--jobs=$NPROC
 
 if test "$GITHUB_ACTIONS" = "true"
 then
@@ -38,9 +38,10 @@ then
 	CC="${CC:-gcc}"
 
 	export GIT_PROVE_OPTS="--timer --jobs $NPROC"
-	export GIT_TEST_OPTS="--verbose-log -x"
+	GIT_TEST_OPTS="--verbose-log -x"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
+	export GIT_TEST_OPTS
 else
 	echo "Could not identify CI type" >&2
 	env >&2
@@ -94,4 +95,4 @@ linux-leaks)
 	;;
 esac
 
-MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
+export MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 12/29] CI: export variables via a wrapper

[Ævar Arnfjörð Bjarmason]

Add and use a "setenv" wrapper to export the variables set in
ci/lib.sh.

For now this merely adds debugging information about what variables
are set, but in a subsequent commit we'll expand this to set variables
in $GITHUB_ENV instead of the environment, and thus carry variable
settings between steps in a job.

We'll also eventually be interested in what type of variable we're
setting, and when it needs to be used, because we'll prominently
expose what variables we set in what CI "step", so having only the
ones we actually use will improve readability.

To that end call the "setenv" function with optional "--build",
"--test" and "--all" parameters to indicate whether they're needed for
building, testing or both.

We currently ignore the "varname" parameter, but will make use of it in a
subsequent commit. Adding it here reduces the needless churn, as we
won't need to change all these lines again for that subsequent change.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                 | 51 ++++++++++++++++++++++++++++++---------
 ci/run-build-and-tests.sh | 34 +++++++++++++-------------
 2 files changed, 57 insertions(+), 28 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 5783eed1b20..3727e6ec2f9 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -13,6 +13,35 @@ then
 	exit 1
 fi
 
+# Helper functions
+setenv () {
+	while test $# != 0
+	do
+		case "$1" in
+		--build)
+			;;
+		--test)
+			;;
+		--all)
+			;;
+		-*)
+			echo "BUG: bad setenv() option '$1'" >&2
+			exit 1
+			;;
+		*)
+			break
+			;;
+		esac
+		shift
+	done
+
+	key=$1
+	val=$2
+	shift 2
+
+	eval "export $key=\"$val\""
+}
+
 check_unignored_build_artifacts ()
 {
 	! git ls-files --other --exclude-standard --error-unmatch \
@@ -24,7 +53,7 @@ check_unignored_build_artifacts ()
 }
 
 # GitHub Action doesn't set TERM, which is required by tput
-export TERM=${TERM:-dumb}
+setenv TERM ${TERM:-dumb}
 
 # How many jobs to run in parallel?
 NPROC=10
@@ -37,21 +66,21 @@ then
 	CI_TYPE=github-actions
 	CC="${CC:-gcc}"
 
-	export GIT_PROVE_OPTS="--timer --jobs $NPROC"
+	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
 	GIT_TEST_OPTS="--verbose-log -x"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
-	export GIT_TEST_OPTS
+	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
 else
 	echo "Could not identify CI type" >&2
 	env >&2
 	exit 1
 fi
 
-export DEVELOPER=1
-export DEFAULT_TEST_TARGET=prove
-export GIT_TEST_CLONE_2GB=true
-export SKIP_DASHED_BUILT_INS=YesPlease
+setenv --build DEVELOPER 1
+setenv --test DEFAULT_TEST_TARGET prove
+setenv --test GIT_TEST_CLONE_2GB true
+setenv --build SKIP_DASHED_BUILT_INS YesPlease
 
 case "$runs_on_pool" in
 ubuntu-latest)
@@ -67,7 +96,7 @@ ubuntu-latest)
 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python2"
 	fi
 
-	export GIT_TEST_HTTPD=true
+	setenv --test GIT_TEST_HTTPD true
 	;;
 macos-latest)
 	if [ "$jobname" = osx-gcc ]
@@ -90,9 +119,9 @@ linux-musl)
 	MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8"
 	;;
 linux-leaks)
-	export SANITIZE=leak
-	export GIT_TEST_PASSING_SANITIZE_LEAK=true
+	setenv --build SANITIZE leak
+	setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true
 	;;
 esac
 
-export MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
+setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index e5a21985b44..35d45a9373e 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -9,33 +9,33 @@ export MAKE_TARGETS="all test"
 
 case "$jobname" in
 linux-gcc)
-	export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
 	;;
 linux-TEST-vars)
-	export GIT_TEST_SPLIT_INDEX=yes
-	export GIT_TEST_MERGE_ALGORITHM=recursive
-	export GIT_TEST_FULL_IN_PACK_ARRAY=true
-	export GIT_TEST_OE_SIZE=10
-	export GIT_TEST_OE_DELTA_SIZE=5
-	export GIT_TEST_COMMIT_GRAPH=1
-	export GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS=1
-	export GIT_TEST_MULTI_PACK_INDEX=1
-	export GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP=1
-	export GIT_TEST_ADD_I_USE_BUILTIN=1
-	export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master
-	export GIT_TEST_WRITE_REV_INDEX=1
-	export GIT_TEST_CHECKOUT_WORKERS=2
+	setenv --test GIT_TEST_SPLIT_INDEX yes
+	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
+	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
+	setenv --test GIT_TEST_OE_SIZE 10
+	setenv --test GIT_TEST_OE_DELTA_SIZE 5
+	setenv --test GIT_TEST_COMMIT_GRAPH 1
+	setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
+	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
+	setenv --test GIT_TEST_WRITE_REV_INDEX 1
+	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
 	;;
 linux-clang)
-	export GIT_TEST_DEFAULT_HASH=sha1
+	setenv --test GIT_TEST_DEFAULT_HASH sha1
 	;;
 linux-sha256)
-	export GIT_TEST_DEFAULT_HASH=sha256
+	setenv --test GIT_TEST_DEFAULT_HASH sha256
 	;;
 pedantic)
 	# Don't run the tests; we only care about whether Git can be
 	# built.
-	export DEVOPTS=pedantic
+	setenv --build DEVOPTS pedantic
 	export MAKE_TARGETS=all
 	;;
 esac
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 13/29] CI: remove "run-build-and-tests.sh", run "make [test]" directly

[Ævar Arnfjörð Bjarmason]

Remove the already thin "ci/run-build-and-tests.sh" wrapper and
instead make the CI run "make" or "make test" directly. By doing this
we'll be able to easily see at a glance whether our failure was in the
compilation or testing, whether that's via human eyes or improve
machine readability.

We also need to run our new "ci/check-unignored-build-artifacts.sh" on
success() in the CI now, just like we already had a step conditional
on failure() running ci/print-test-failures.sh.

The reason we used a "ci/run-build-and-tests.sh" wrapper in the first
place had to do with Travis CI-specific constraints that no longer
apply to us, as the Travis CI support has been removed.

Instead we can configure the CI in an earlier step by running
"ci/lib.sh", which under GitHub CI will write the environment
variables we need to the "$GITHUB_ENV" file.

We'll then have access to them in subsequent steps, and crucially
those variables will be prominently visible at the start of each step
via an expandable drop-down in the UI.drop-do.

I.e. this changes the CI run from a top-down flow like (pseudocode):

 - job:
   - step1:
     - use ci/lib.sh to set env vars
     - run a script like ci/run-build-and-tests.sh
   - step2:
     - if: failure()
     - use ci/lib.sh to set env vars
     - run ci/print-test-failures.sh

To:

 - job:
   - step1:
     - set variables in $GITHUB_ENV using ci/lib.sh
   - step2:
     - make
   - step3:
     - make test
   - step4:
     - if: failure()
     - run ci/print-test-failures.sh
   - step5:
     - if: success()
     - run ci/check-unignored-build-artifacts.sh

There is a proposal[2] to get some of the benefits of this approach by
not re-arranging our variable setup in this way, but to instead use
the GitHub CI grouping syntax to focus on the relevant parts of "make"
or "make test" when we have failures.

Doing it this way makes for better looking GitHub CI UI, and lays much
better ground work for our CI going forward. Because:

 * The CI logic will be more portable to a future CI system, since a
   common feature of them is to run various commands in sequence, but
   a future system won't necessarily support the GitHub-specifics
   syntax of "grouping" output within a "step".

   Even if those systems don't support a "$GITHUB_ENV" emulating will
   be much easier than to deal with some CI-specific grouping syntax.

 * At the start of every step the GitHub CI presents an expandable
   list of environment variables from "$GITHUB_ENV". We'll now see
   exactly what variables affected that step (although we currently
   overshoot that a bit, and always define all variables).

 * CI failures will be easier to reproduce locally, as this makes the
   relevant ci/* scripts something that sets up our environment, but
   leaves "make" and "make test" working as they do locally.

   To reproduce a run the user only needs to set the variables
   discussed in the drop-down above, either manually or by running
   "ci/lib.sh".

 * The output will be less verbose. The "ci/lib.sh" script uses "set
   -x", and before this e.g. "ci/static-analysis.sh" would start with
   40 lines of trace output, culminating in using "export" to export
   the relevant environment variables.

   Now that verbosity is in the earlier "ci/lib.sh" step, and not in
   any subsequent one. The "make" targets then start out with the
   relevant output non-trace output right away.

 * If we do want to use the grouping syntax within a "step" it'll now
   be easier to do so. It doesn't support nesting, so we'd have to
   make a choice between using it for e.g. "make" v.s. "make test", or
   individual test failures. See "sadly" in [3].

1. https://lore.kernel.org/git/211120.86k0h30zuw.gmgdl@evledraar.gmail.com/
2. https://lore.kernel.org/git/pull.1117.git.1643050574.gitgitgadget@gmail.com/
3. https://lore.kernel.org/git/9333ba781b8240f704e739b00d274f8c3d887e39.1643050574.git.gitgitgadget@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml            | 14 ++++++--
 ci/check-unignored-build-artifacts.sh |  8 +++++
 ci/install-dependencies.sh            |  4 +++
 ci/lib.sh                             | 36 +++++++++++++++++++-
 ci/make-test-artifacts.sh             |  2 --
 ci/run-build-and-tests.sh             | 47 ---------------------------
 ci/run-test-slice.sh                  |  2 --
 ci/test-documentation.sh              |  1 -
 8 files changed, 58 insertions(+), 56 deletions(-)
 create mode 100755 ci/check-unignored-build-artifacts.sh
 delete mode 100755 ci/run-build-and-tests.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index ad752010102..c83f6a21ca3 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -268,7 +268,10 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/run-build-and-tests.sh
+    - run: ci/lib.sh
+    - run: make
+    - run: make test
+      if: success()
     - run: ci/print-test-failures.sh
       if: failure()
     - name: Upload failed tests' directories
@@ -292,6 +295,7 @@ jobs:
           image: daald/ubuntu32:xenial
         - jobname: pedantic
           image: fedora
+          skip-tests: no
     env:
       jobname: ${{matrix.vector.jobname}}
     runs-on: ubuntu-latest
@@ -299,9 +303,12 @@ jobs:
     steps:
     - uses: actions/checkout@v1
     - run: ci/install-docker-dependencies.sh
-    - run: ci/run-build-and-tests.sh
+    - run: ci/lib.sh
+    - run: make
+    - run: make test
+      if: success() && matrix.vector.skip-tests != 'no'
     - run: ci/print-test-failures.sh
-      if: failure()
+      if: failure() && matrix.vector.skip-tests != 'no'
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v1
@@ -317,6 +324,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
+    - run: ci/lib.sh
     - run: make ci-static-analysis
   sparse:
     needs: ci-config
diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh
new file mode 100755
index 00000000000..56d04b0db9a
--- /dev/null
+++ b/ci/check-unignored-build-artifacts.sh
@@ -0,0 +1,8 @@
+#!/bin/sh
+#
+# Check whether the build created anything not in our .gitignore
+#
+
+. ${0%/*}/lib.sh
+
+check_unignored_build_artifacts
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 34b4400f38b..01006379fd0 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -22,6 +22,10 @@ ubuntu-latest)
 	P4_PATH="$HOME/custom/p4"
 	GIT_LFS_PATH="$HOME/custom/git-lfs"
 	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
+	if test -n "$GITHUB_PATH"
+	then
+		echo "$PATH" >>"$GITHUB_PATH"
+	fi
 
 	P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
 	LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
diff --git a/ci/lib.sh b/ci/lib.sh
index 3727e6ec2f9..cc9a368be42 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,3 +1,5 @@
+#!/bin/sh
+
 # Library of functions shared by all CI scripts
 
 # Set 'exit on error' for all CI scripts to let the caller know that
@@ -39,7 +41,10 @@ setenv () {
 	val=$2
 	shift 2
 
-	eval "export $key=\"$val\""
+	if test -n "$GITHUB_ENV"
+	then
+		echo "$key=$val" >>"$GITHUB_ENV"
+	fi
 }
 
 check_unignored_build_artifacts ()
@@ -109,6 +114,35 @@ macos-latest)
 esac
 
 case "$jobname" in
+linux-gcc)
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
+	;;
+linux-TEST-vars)
+	setenv --test GIT_TEST_SPLIT_INDEX yes
+	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
+	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
+	setenv --test GIT_TEST_OE_SIZE 10
+	setenv --test GIT_TEST_OE_DELTA_SIZE 5
+	setenv --test GIT_TEST_COMMIT_GRAPH 1
+	setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
+	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
+	setenv --test GIT_TEST_WRITE_REV_INDEX 1
+	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
+	;;
+linux-clang)
+	setenv --test GIT_TEST_DEFAULT_HASH sha1
+	;;
+linux-sha256)
+	setenv --test GIT_TEST_DEFAULT_HASH sha256
+	;;
+pedantic)
+	# Don't run the tests; we only care about whether Git can be
+	# built.
+	setenv --build DEVOPTS pedantic
+	;;
 linux32)
 	CC=gcc
 	;;
diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh
index 646967481f6..45298562982 100755
--- a/ci/make-test-artifacts.sh
+++ b/ci/make-test-artifacts.sh
@@ -8,5 +8,3 @@ mkdir -p "$1" # in case ci/lib.sh decides to quit early
 . ${0%/*}/lib.sh
 
 make artifacts-tar ARTIFACTS_DIRECTORY="$1"
-
-check_unignored_build_artifacts
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
deleted file mode 100755
index 35d45a9373e..00000000000
--- a/ci/run-build-and-tests.sh
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/bin/sh
-#
-# Build and test Git
-#
-
-. ${0%/*}/lib.sh
-
-export MAKE_TARGETS="all test"
-
-case "$jobname" in
-linux-gcc)
-	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
-	;;
-linux-TEST-vars)
-	setenv --test GIT_TEST_SPLIT_INDEX yes
-	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
-	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
-	setenv --test GIT_TEST_OE_SIZE 10
-	setenv --test GIT_TEST_OE_DELTA_SIZE 5
-	setenv --test GIT_TEST_COMMIT_GRAPH 1
-	setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
-	setenv --test GIT_TEST_MULTI_PACK_INDEX 1
-	setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
-	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
-	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
-	setenv --test GIT_TEST_WRITE_REV_INDEX 1
-	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
-	;;
-linux-clang)
-	setenv --test GIT_TEST_DEFAULT_HASH sha1
-	;;
-linux-sha256)
-	setenv --test GIT_TEST_DEFAULT_HASH sha256
-	;;
-pedantic)
-	# Don't run the tests; we only care about whether Git can be
-	# built.
-	setenv --build DEVOPTS pedantic
-	export MAKE_TARGETS=all
-	;;
-esac
-
-# Any new "test" targets should not go after this "make", but should
-# adjust $MAKE_TARGETS. Otherwise compilation-only targets above will
-# start running tests.
-make $MAKE_TARGETS
-check_unignored_build_artifacts
diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index b9a682b4bcd..70326961454 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -8,5 +8,3 @@
 make --quiet -C t T="$(cd t &&
 	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
 	tr '\n' ' ')"
-
-check_unignored_build_artifacts
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 41e2b126311..6b5cce03bd7 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -39,4 +39,3 @@ test -s Documentation/git.html
 grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
 
 rm -f stdout.log stderr.log stderr.raw
-check_unignored_build_artifacts
-- 
2.36.0.rc2.843.g193535c2aa7

Re: [PATCH v3 13/29] CI: remove "run-build-and-tests.sh", run "make [test]" directly

[Eric Sunshine]

On 4/13/22 3:51 PM, Ævar Arnfjörð Bjarmason wrote:
> Remove the already thin "ci/run-build-and-tests.sh" wrapper and
> instead make the CI run "make" or "make test" directly. By doing this
> we'll be able to easily see at a glance whether our failure was in the
> compilation or testing, whether that's via human eyes or improve
> machine readability.
> 
> We also need to run our new "ci/check-unignored-build-artifacts.sh" on
> success() in the CI now, just like we already had a step conditional
> on failure() running ci/print-test-failures.sh.
> 
> The reason we used a "ci/run-build-and-tests.sh" wrapper in the first
> place had to do with Travis CI-specific constraints that no longer
> apply to us, as the Travis CI support has been removed.
> 
> Instead we can configure the CI in an earlier step by running
> "ci/lib.sh", which under GitHub CI will write the environment
> variables we need to the "$GITHUB_ENV" file.
> 
> We'll then have access to them in subsequent steps, and crucially
> those variables will be prominently visible at the start of each step
> via an expandable drop-down in the UI.drop-do.
> 
> I.e. this changes the CI run from a top-down flow like (pseudocode):
> 
>   - job:
>     - step1:
>       - use ci/lib.sh to set env vars
>       - run a script like ci/run-build-and-tests.sh
>     - step2:
>       - if: failure()
>       - use ci/lib.sh to set env vars
>       - run ci/print-test-failures.sh
> 
> To:
> 
>   - job:
>     - step1:
>       - set variables in $GITHUB_ENV using ci/lib.sh
>     - step2:
>       - make
>     - step3:
>       - make test
>     - step4:
>       - if: failure()
>       - run ci/print-test-failures.sh
>     - step5:
>       - if: success()
>       - run ci/check-unignored-build-artifacts.sh
> 
> There is a proposal[2] to get some of the benefits of this approach by
> not re-arranging our variable setup in this way, but to instead use
> the GitHub CI grouping syntax to focus on the relevant parts of "make"
> or "make test" when we have failures.
> 
> Doing it this way makes for better looking GitHub CI UI, and lays much
> better ground work for our CI going forward. Because:
> 
>   * The CI logic will be more portable to a future CI system, since a
>     common feature of them is to run various commands in sequence, but
>     a future system won't necessarily support the GitHub-specifics
>     syntax of "grouping" output within a "step".
> 
>     Even if those systems don't support a "$GITHUB_ENV" emulating will
>     be much easier than to deal with some CI-specific grouping syntax.
> 
>   * At the start of every step the GitHub CI presents an expandable
>     list of environment variables from "$GITHUB_ENV". We'll now see
>     exactly what variables affected that step (although we currently
>     overshoot that a bit, and always define all variables).
> 
>   * CI failures will be easier to reproduce locally, as this makes the
>     relevant ci/* scripts something that sets up our environment, but
>     leaves "make" and "make test" working as they do locally.
> 
>     To reproduce a run the user only needs to set the variables
>     discussed in the drop-down above, either manually or by running
>     "ci/lib.sh".
> 
>   * The output will be less verbose. The "ci/lib.sh" script uses "set
>     -x", and before this e.g. "ci/static-analysis.sh" would start with
>     40 lines of trace output, culminating in using "export" to export
>     the relevant environment variables.
> 
>     Now that verbosity is in the earlier "ci/lib.sh" step, and not in
>     any subsequent one. The "make" targets then start out with the
>     relevant output non-trace output right away.

s/output non-trace output/non-trace output/

>   * If we do want to use the grouping syntax within a "step" it'll now
>     be easier to do so. It doesn't support nesting, so we'd have to
>     make a choice between using it for e.g. "make" v.s. "make test", or
>     individual test failures. See "sadly" in [3].
> 
> 1. https://lore.kernel.org/git/211120.86k0h30zuw.gmgdl@evledraar.gmail.com/
> 2. https://lore.kernel.org/git/pull.1117.git.1643050574.gitgitgadget@gmail.com/
> 3. https://lore.kernel.org/git/9333ba781b8240f704e739b00d274f8c3d887e39.1643050574.git.gitgitgadget@gmail.com/
> 
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

[PATCH v3 14/29] ci/lib.sh: use "test" instead of "["

[Ævar Arnfjörð Bjarmason]

Change the ci/lib.sh script to use "if test" rather than "if [". Until
the preceding commit this script relied on the non-POSIX "export X=Y"
syntax, but now that it's POSIX portable let's use our usual style in
it, and not the bash-specific style that was used in it historically.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index cc9a368be42..ffc0e524aaa 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -94,7 +94,7 @@ ubuntu-latest)
 		break
 	fi
 
-	if [ "$jobname" = linux-gcc ]
+	if test "$jobname" = linux-gcc
 	then
 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python3"
 	else
@@ -104,7 +104,7 @@ ubuntu-latest)
 	setenv --test GIT_TEST_HTTPD true
 	;;
 macos-latest)
-	if [ "$jobname" = osx-gcc ]
+	if test "$jobname" = osx-gcc
 	then
 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
 	else
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 15/29] CI: check ignored unignored build artifacts in "win[+VS] build" too

[Ævar Arnfjörð Bjarmason]

Use the newly created "ci/check-unignored-build-artifacts.sh" script
in the "win build" and "win+VS build" steps too. These steps weren't
doing this sanity check of untracked files against .gitignore before,
but as the check passes for them let's keep doing that extra sanity
check.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index c83f6a21ca3..05c5771e7f6 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -93,6 +93,9 @@ jobs:
         HOME: ${{runner.workspace}}
         NO_PERL: 1
       run: . /etc/profile && ci/make-test-artifacts.sh artifacts
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
+      shell: bash
     - name: zip up tracked files
       run: git archive -o artifacts/tracked.tar.gz HEAD
     - name: upload tracked files and build artifacts
@@ -180,6 +183,9 @@ jobs:
       run: |
         mkdir -p artifacts &&
         eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts NO_GETTEXT=YesPlease 2>&1 | grep ^tar)"
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
+      shell: bash
     - name: zip up tracked files
       run: git archive -o artifacts/tracked.tar.gz HEAD
     - name: upload tracked files and build artifacts
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 16/29] CI: invoke "make artifacts-tar" directly in windows-build

[Ævar Arnfjörð Bjarmason]

Change the windows-build job to invoke the "make artifacts-tar" step
directly, instead of calling a "ci/make-test-artifacts.sh" script.

The script was needed because "ci/lib.sh" would set up various
environment variables for us, but now we can instead use the
"ci/lib.sh" in its script mode.

The "mkdir -p" added in b819f1d2cec (ci: parallelize testing on
Windows, 2019-01-29) isn't needed, the same commit added that "mkdir
-p" to the "artifacts-tar" rule itself, so we can have "make" create
the directory for us.

This also has the benefit of making the "build" step less chatty,
since it won't start with the verbose "set -x" output, that's now
contained in the "ci/lib.sh" step.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml |  5 ++++-
 ci/make-test-artifacts.sh  | 10 ----------
 2 files changed, 4 insertions(+), 11 deletions(-)
 delete mode 100755 ci/make-test-artifacts.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 05c5771e7f6..2c23a19eac2 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -87,12 +87,15 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
+    - run: ci/lib.sh
+      shell: bash
     - name: build
       shell: bash
       env:
         HOME: ${{runner.workspace}}
         NO_PERL: 1
-      run: . /etc/profile && ci/make-test-artifacts.sh artifacts
+      run: . /etc/profile && make artifacts-tar ARTIFACTS_DIRECTORY=artifacts
+      if: success()
     - run: ci/check-unignored-build-artifacts.sh
       if: success()
       shell: bash
diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh
deleted file mode 100755
index 45298562982..00000000000
--- a/ci/make-test-artifacts.sh
+++ /dev/null
@@ -1,10 +0,0 @@
-#!/bin/sh
-#
-# Build Git and store artifacts for testing
-#
-
-mkdir -p "$1" # in case ci/lib.sh decides to quit early
-
-. ${0%/*}/lib.sh
-
-make artifacts-tar ARTIFACTS_DIRECTORY="$1"
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 17/29] CI: split up and reduce "ci/test-documentation.sh"

[Ævar Arnfjörð Bjarmason]

Change the "ci/test-documentation.sh" script to run the bash-specific
parts in as one command in the CI job itself, and to run the two "make
doc" commands at the top-level.

It'll now be obvious from the title of the step if if we failed in the
asciidoc or asciidoctor step.

Since the "check_unignored_build_artifacts()" function is now only
used in "ci/check-unignored-build-artifacts.sh" move that function
there.

The recipe for the job in ".github/workflows/main.yml" is now a bit
verbose because it's effectively the same job twice, with a "make
clean" in-between. It would be better for the verbosity to run it via
a matrix as done in the alternate approach in [1] does, but then we'd
sacrifice overall CPU time for the brevity. It might still be worth
doing, but let's go for this simpler approach for now.

1. https://lore.kernel.org/git/patch-v2-6.6-7c423c8283d-20211120T030848Z-avarab@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml            | 16 +++++++++++++-
 ci/check-unignored-build-artifacts.sh | 10 +++++++++
 ci/lib.sh                             | 10 ---------
 ci/test-documentation.sh              | 31 +++++++++------------------
 4 files changed, 35 insertions(+), 32 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 2c23a19eac2..92b914f16fd 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -365,4 +365,18 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/test-documentation.sh
+    - run: ci/lib.sh
+    - run: make check-docs
+    - run: "make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)"
+      shell: bash
+    - run: ci/test-documentation.sh AsciiDoc
+      if: success()
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
+    - run: make clean
+    - run: "make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)"
+      shell: bash
+    - run: ci/test-documentation.sh Asciidoctor
+      if: success()
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh
index 56d04b0db9a..0bc04f32804 100755
--- a/ci/check-unignored-build-artifacts.sh
+++ b/ci/check-unignored-build-artifacts.sh
@@ -5,4 +5,14 @@
 
 . ${0%/*}/lib.sh
 
+check_unignored_build_artifacts ()
+{
+	! git ls-files --other --exclude-standard --error-unmatch \
+		-- ':/*' 2>/dev/null ||
+	{
+		echo "$(tput setaf 1)error: found unignored build artifacts$(tput sgr0)"
+		false
+	}
+}
+
 check_unignored_build_artifacts
diff --git a/ci/lib.sh b/ci/lib.sh
index ffc0e524aaa..35a972fa3f5 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -47,16 +47,6 @@ setenv () {
 	fi
 }
 
-check_unignored_build_artifacts ()
-{
-	! git ls-files --other --exclude-standard --error-unmatch \
-		-- ':/*' 2>/dev/null ||
-	{
-		echo "$(tput setaf 1)error: found unignored build artifacts$(tput sgr0)"
-		false
-	}
-}
-
 # GitHub Action doesn't set TERM, which is required by tput
 setenv TERM ${TERM:-dumb}
 
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 6b5cce03bd7..b8a6a6f664e 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -1,10 +1,12 @@
-#!/usr/bin/env bash
+#!/bin/sh
 #
-# Perform sanity checks on documentation and build it.
+# Perform sanity checks on "make doc" output and built documentation
 #
 
 . ${0%/*}/lib.sh
 
+generator=$1
+
 filter_log () {
 	sed -e '/^GIT_VERSION = /d' \
 	    -e "/constant Gem::ConfigMap is deprecated/d" \
@@ -14,28 +16,15 @@ filter_log () {
 	    "$1"
 }
 
-make check-docs
-
-# Build docs with AsciiDoc
-make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
-cat stderr.raw
-filter_log stderr.raw >stderr.log
-test ! -s stderr.log
-test -s Documentation/git.html
-test -s Documentation/git.xml
-test -s Documentation/git.1
-grep '<meta name="generator" content="AsciiDoc ' Documentation/git.html
-
-rm -f stdout.log stderr.log stderr.raw
-check_unignored_build_artifacts
-
-# Build docs with AsciiDoctor
-make clean
-make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
 cat stderr.raw
 filter_log stderr.raw >stderr.log
 test ! -s stderr.log
 test -s Documentation/git.html
-grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
+if test "$generator" = "Asciidoctor"
+then
+	test -s Documentation/git.xml
+	test -s Documentation/git.1
+fi
+grep "<meta name=\"generator\" content=\"$generator " Documentation/git.html
 
 rm -f stdout.log stderr.log stderr.raw
-- 
2.36.0.rc2.843.g193535c2aa7

Re: [PATCH v3 17/29] CI: split up and reduce "ci/test-documentation.sh"

[Eric Sunshine]

On 4/13/22 3:51 PM, Ævar Arnfjörð Bjarmason wrote:
> Change the "ci/test-documentation.sh" script to run the bash-specific
> parts in as one command in the CI job itself, and to run the two "make
> doc" commands at the top-level.
> 
> It'll now be obvious from the title of the step if if we failed in the
> asciidoc or asciidoctor step.

s/if if/if/

> Since the "check_unignored_build_artifacts()" function is now only
> used in "ci/check-unignored-build-artifacts.sh" move that function
> there.
> 
> The recipe for the job in ".github/workflows/main.yml" is now a bit
> verbose because it's effectively the same job twice, with a "make
> clean" in-between. It would be better for the verbosity to run it via
> a matrix as done in the alternate approach in [1] does, but then we'd
> sacrifice overall CPU time for the brevity. It might still be worth
> doing, but let's go for this simpler approach for now.
> 
> 1. https://lore.kernel.org/git/patch-v2-6.6-7c423c8283d-20211120T030848Z-avarab@gmail.com/
> 
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

[PATCH v3 18/29] CI: combine ci/install{,-docker}-dependencies.sh

[Ævar Arnfjörð Bjarmason]

Combine the "ci/install-dependencies.sh" and
"ci/install-docker-dependencies.sh" into one script. The "case"
statement in the latter only cared about "$jobname", and can be folded
into the same "case" statement in the former.

The reason they split up is historical, and because the
"ci/install-dependencies.sh" used "ci/lib.sh", which requires
"bash". At least one of the docker containers doesn't have "bash". To
make the existing code POSIX-compatible we need to replace pushd/popd
with a sub-shell, but no other changes were needed.

This also fixes issue in "ci/install-docker-dependencies.sh" where
we'd hide errors due to not using "set -e". Now that we include
"ci/lib.sh" we won't have that potential issue.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml        |  2 +-
 ci/install-dependencies.sh        | 27 ++++++++++++++++++++++-----
 ci/install-docker-dependencies.sh | 22 ----------------------
 3 files changed, 23 insertions(+), 28 deletions(-)
 delete mode 100755 ci/install-docker-dependencies.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 92b914f16fd..3fd3e4f873c 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -311,7 +311,7 @@ jobs:
     container: ${{matrix.vector.image}}
     steps:
     - uses: actions/checkout@v1
-    - run: ci/install-docker-dependencies.sh
+    - run: ci/install-dependencies.sh
     - run: ci/lib.sh
     - run: make
     - run: make test
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 01006379fd0..9388289d0ed 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -1,4 +1,4 @@
-#!/usr/bin/env bash
+#!/bin/sh
 #
 # Install dependencies required to build and test Git on Linux and macOS
 #
@@ -34,18 +34,20 @@ ubuntu-latest)
 	sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \
 		$UBUNTU_COMMON_PKGS $CC_PACKAGE
 	mkdir --parents "$P4_PATH"
-	pushd "$P4_PATH"
+	(
+		cd "$P4_PATH"
 		wget --quiet "$P4WHENCE/bin.linux26x86_64/p4d"
 		wget --quiet "$P4WHENCE/bin.linux26x86_64/p4"
 		chmod u+x p4d
 		chmod u+x p4
-	popd
+	)
 	mkdir --parents "$GIT_LFS_PATH"
-	pushd "$GIT_LFS_PATH"
+	(
+		cd "$GIT_LFS_PATH"
 		wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
 		tar --extract --gunzip --file "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
 		cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs .
-	popd
+	)
 	;;
 macos-latest)
 	export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
@@ -90,6 +92,21 @@ linux-gcc-default)
 	sudo apt-get -q update
 	sudo apt-get -q -y install $UBUNTU_COMMON_PKGS
 	;;
+linux32)
+	linux32 --32bit i386 sh -c '
+		apt update >/dev/null &&
+		apt install -y build-essential libcurl4-openssl-dev \
+			libssl-dev libexpat-dev gettext python >/dev/null
+	'
+	;;
+linux-musl)
+	apk add --update build-base curl-dev openssl-dev expat-dev gettext \
+		pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null
+	;;
+pedantic)
+	dnf -yq update >/dev/null &&
+	dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null
+	;;
 esac
 
 if type p4d >/dev/null && type p4 >/dev/null
diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh
deleted file mode 100755
index 78b7e326da6..00000000000
--- a/ci/install-docker-dependencies.sh
+++ /dev/null
@@ -1,22 +0,0 @@
-#!/bin/sh
-#
-# Install dependencies required to build and test Git inside container
-#
-
-case "$jobname" in
-linux32)
-	linux32 --32bit i386 sh -c '
-		apt update >/dev/null &&
-		apt install -y build-essential libcurl4-openssl-dev \
-			libssl-dev libexpat-dev gettext python >/dev/null
-	'
-	;;
-linux-musl)
-	apk add --update build-base curl-dev openssl-dev expat-dev gettext \
-		pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null
-	;;
-pedantic)
-	dnf -yq update >/dev/null &&
-	dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null
-	;;
-esac
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 19/29] CI: move "env" definitions into ci/lib.sh

[Ævar Arnfjörð Bjarmason]

Have the GitHub CI jobs use the "ci/lib".sh as a source of truth for
environment variables that affect their builds and tests. This brings
them in line with the rest of the jobs, and make it easier to see the
entire CI configuration per jobname at a glance.

To do this we need to add a ci/lib.sh "step" to the remaining jobs
that didn't have it added in preceding commits.

The Makefile parameters are luckily all accepted via the environment,
so we can export these instead of adding them to MAKEFLAGS.

Let's also use the documented GIT_CONIFG_* mechanism for setting
config instead of the internal GIT_CONFIG_PARAMETERS variable. This
adjusts code added in 889cacb6897 (ci: configure GitHub Actions for
CI/PR, 2020-04-11), we could probably use the GIT_AUTHOR_NAME etc, but
let's keep setting this via config, just with the documented
mechanism.

By setting "ARTIFACTS_DIRECTORY=artifacts" once we don't need to
repeat it in various places, let's get it from the environment
instead.

Finally, the "DEVELOPER: 1" was only needed by the "sparse" job, which
wasn't using "ci/lib.sh" (and which sets DEVELOPER=1). Let's instead
have the "sparse" job use the "ci/lib.sh" and remove DEVELOPER=1 from
".github/workflows/main.yml".

This substantially speeds up the "sparse" job, since it'll now pick up
the "--jobs" setting in MAKEFLAGS that we use everywhere else. Before
it ran in around 4m30s, now in around 2m30s.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 29 +++++++++++------------------
 ci/lib.sh                  | 21 +++++++++++++++++++++
 2 files changed, 32 insertions(+), 18 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 3fd3e4f873c..fe37e0f1b36 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -2,9 +2,6 @@ name: CI
 
 on: [push, pull_request]
 
-env:
-  DEVELOPER: 1
-
 jobs:
   ci-config:
     name: config
@@ -93,8 +90,7 @@ jobs:
       shell: bash
       env:
         HOME: ${{runner.workspace}}
-        NO_PERL: 1
-      run: . /etc/profile && make artifacts-tar ARTIFACTS_DIRECTORY=artifacts
+      run: . /etc/profile && make artifacts-tar
       if: success()
     - run: ci/check-unignored-build-artifacts.sh
       if: success()
@@ -105,7 +101,7 @@ jobs:
       uses: actions/upload-artifact@v2
       with:
         name: windows-artifacts
-        path: artifacts
+        path: ${{env.ARTIFACTS_DIRECTORY}}
   windows-test:
     name: win test
     env:
@@ -126,6 +122,8 @@ jobs:
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
+    - run: ci/lib.sh
+      shell: bash
     - name: test
       shell: bash
       run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
@@ -145,9 +143,6 @@ jobs:
       jobname: vs-build
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
-    env:
-      NO_PERL: 1
-      GIT_CONFIG_PARAMETERS: "'user.name=CI' 'user.email=ci@git'"
     runs-on: windows-latest
     steps:
     - uses: actions/checkout@v2
@@ -171,6 +166,8 @@ jobs:
     - name: copy dlls to root
       shell: cmd
       run: compat\vcbuild\vcpkg_copy_dlls.bat release
+    - run: ci/lib.sh
+      shell: bash
     - name: generate Visual Studio solution
       shell: bash
       run: |
@@ -181,21 +178,16 @@ jobs:
     - name: bundle artifact tar
       shell: bash
       env:
-        MSVC: 1
         VCPKG_ROOT: ${{github.workspace}}\compat\vcbuild\vcpkg
       run: |
-        mkdir -p artifacts &&
-        eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts NO_GETTEXT=YesPlease 2>&1 | grep ^tar)"
-    - run: ci/check-unignored-build-artifacts.sh
-      if: success()
-      shell: bash
+        eval "$(make -n artifacts-tar 2>&1 | grep -e ^mkdir -e ^tar)"
     - name: zip up tracked files
       run: git archive -o artifacts/tracked.tar.gz HEAD
     - name: upload tracked files and build artifacts
       uses: actions/upload-artifact@v2
       with:
         name: vs-artifacts
-        path: artifacts
+        path: ${{env.ARTIFACTS_DIRECTORY}}
   vs-test:
     name: win+VS test
     env:
@@ -216,10 +208,10 @@ jobs:
     - name: extract tracked files and build artifacts
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
+    - run: ci/lib.sh
+      shell: bash
     - name: test
       shell: bash
-      env:
-        NO_SVN_TESTS: 1
       run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
     - name: ci/print-test-failures.sh
       if: failure()
@@ -354,6 +346,7 @@ jobs:
     - uses: actions/checkout@v2
     - name: Install other dependencies
       run: ci/install-dependencies.sh
+    - run: ci/lib.sh
     - run: make sparse
   documentation:
     name: documentation
diff --git a/ci/lib.sh b/ci/lib.sh
index 35a972fa3f5..8c574b21a67 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -104,6 +104,27 @@ macos-latest)
 esac
 
 case "$jobname" in
+windows-build)
+	setenv --build NO_PERL NoThanks
+	setenv --build ARTIFACTS_DIRECTORY artifacts
+	;;
+vs-build)
+	setenv --build NO_PERL NoThanks
+	setenv --build NO_GETTEXT NoThanks
+	setenv --build ARTIFACTS_DIRECTORY artifacts
+	setenv --build INCLUDE_DLLS_IN_ARTIFACTS YesPlease
+	setenv --build MSVC YesPlease
+
+	setenv --build GIT_CONFIG_COUNT 2
+	setenv --build GIT_CONFIG_KEY_0 user.name
+	setenv --build GIT_CONFIG_VALUE_0 CI
+	setenv --build GIT_CONFIG_KEY_1 user.emailname
+	setenv --build GIT_CONFIG_VALUE_1 ci@git
+	setenv --build GIT_CONFIG_VALUE_1 ci@git
+	;;
+vs-test)
+	setenv --test NO_SVN_TESTS YesPlease
+	;;
 linux-gcc)
 	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
 	;;
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 20/29] ci/run-test-slice.sh: replace shelling out with "echo"

[Ævar Arnfjörð Bjarmason]

Amend this code added in b819f1d2cec (ci: parallelize testing on
Windows, 2019-01-29) to stop shelling out to "tr", all we're trying to
do here is to turn '\n'-delimited output into a ' '-delimited list. To
do that we can just use "echo".

None of these files will contain the odd characters using "echo" might
choke on in the general case (i.e. '\'-escapes and the like).

This has the added benefit of not adding a redundant ' ' to the end of
the T-parameter, which didn't matter in practice, but as we'll see in
a subsequent commit makes for slightly nicer output, as we'll show
this variable using a GitHub CI-native feature.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/run-test-slice.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index 70326961454..1de6a18ca47 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -5,6 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-make --quiet -C t T="$(cd t &&
-	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
-	tr '\n' ' ')"
+tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \
+	t[0-9]*.sh))
+make --quiet -C t T="$tests"
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 21/29] CI: pre-select test slice in Windows & VS tests

[Ævar Arnfjörð Bjarmason]

In preceding commits the tests have been changed to do their setup via
$GITHUB_ENV in one step, and to then have subsequent steps that re-use
that environment.

Let's change the "test slice" tests added in b819f1d2cec (ci:
parallelize testing on Windows, 2019-01-29) to do the same. These
tests select 10% of the tests to run in 10 "test slices". Now we'll
select those in a step that immediately precedes the testing step, and
then simply invoke "make -C t -e".

This has the advantage that the tests to be run are now listed in the
standard "Run" drop-down at the start of the "test" step.

Since the "T" variable in "t/Makefile" doesn't normally accept
overrides from the environment we need to invoke "make" with the "-e"
option (a.k.a. "--environment-overrides"). We could also make $(T) in
t/Makefile be a "?=" assigned variable, but this way works, and is
arguably clearer as it's more obvious that we're injecting a special
list of tests that override the normal behavior of that Makefile.

Note that we cannot run the top-level "make test" here, because of how
the Windows CI builds git, i.e. either via CMake or some option that
would cause "make test" to recompile git itself. Instead we run "make
-C t [...]".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml                     | 10 ++++++++--
 ci/{run-test-slice.sh => select-test-slice.sh} |  4 ++--
 2 files changed, 10 insertions(+), 4 deletions(-)
 rename ci/{run-test-slice.sh => select-test-slice.sh} (57%)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index fe37e0f1b36..1e13718b989 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -124,9 +124,12 @@ jobs:
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
     - run: ci/lib.sh
       shell: bash
+    - name: select tests
+      run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
+      shell: bash
     - name: test
       shell: bash
-      run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
+      run: . /etc/profile && make -C t -e
     - name: ci/print-test-failures.sh
       if: failure()
       shell: bash
@@ -210,9 +213,12 @@ jobs:
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
     - run: ci/lib.sh
       shell: bash
+    - name: select tests
+      run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
+      shell: bash
     - name: test
       shell: bash
-      run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
+      run: . /etc/profile && make -C t -e
     - name: ci/print-test-failures.sh
       if: failure()
       shell: bash
diff --git a/ci/run-test-slice.sh b/ci/select-test-slice.sh
similarity index 57%
rename from ci/run-test-slice.sh
rename to ci/select-test-slice.sh
index 1de6a18ca47..ec602f8a053 100755
--- a/ci/run-test-slice.sh
+++ b/ci/select-test-slice.sh
@@ -1,10 +1,10 @@
 #!/bin/sh
 #
-# Test Git in parallel
+# Select a portion of the tests for testing Git in parallel
 #
 
 . ${0%/*}/lib.sh
 
 tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \
 	t[0-9]*.sh))
-make --quiet -C t T="$tests"
+echo T="$tests" >>$GITHUB_ENV
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 22/29] CI: only invoke ci/lib.sh as "steps" in main.yml

[Ævar Arnfjörð Bjarmason]

Change the scripts in ci/ to stop using ci/lib.sh as a library, now
that the only thing it did for them was to "set -ex" and possibly set
TERM=dumb.

Let's create a ci/lib-tput.sh for those that need to use "tput"
instead, and have these scripts invoke "set -ex" themselves.

This makes their invocation a lot less verbose, since they'll be
relying on an earlier step in the CI job to have set the variables in
$GITHUB_ENV, and won't be spewing their own trace output to set those
variables again.

Let's also create a ci/lib-ci-type.sh, and have ci/lib.sh and
ci/print-test-failures.sh share the logic to discover the CI type. We
could have set the CI_TYPE in the environment with "setenv", but let's
avoid that verbosity for this purely internal variable.

The "ci/lib.sh" is now no longer a "Library of functions shared by all
CI scripts", so let's remove that commentary, and the misleading
comment about "set -ex" being for "installing dependencies", we're now
no longer using it in "ci/install-dependencies.sh" (but it does its
own "set -ex").

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/check-unignored-build-artifacts.sh |  4 +++-
 ci/install-dependencies.sh            |  2 +-
 ci/lib-ci-type.sh                     |  9 +++++++++
 ci/lib-tput.sh                        |  5 +++++
 ci/lib.sh                             | 27 ++++++++++-----------------
 ci/print-test-failures.sh             |  6 +++---
 ci/select-test-slice.sh               |  2 +-
 ci/test-documentation.sh              |  2 +-
 8 files changed, 33 insertions(+), 24 deletions(-)
 create mode 100644 ci/lib-ci-type.sh
 create mode 100644 ci/lib-tput.sh

diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh
index 0bc04f32804..c27d6a97f45 100755
--- a/ci/check-unignored-build-artifacts.sh
+++ b/ci/check-unignored-build-artifacts.sh
@@ -3,7 +3,9 @@
 # Check whether the build created anything not in our .gitignore
 #
 
-. ${0%/*}/lib.sh
+set -ex
+
+. ${0%/*}/lib-tput.sh
 
 check_unignored_build_artifacts ()
 {
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 9388289d0ed..ee9af62fc92 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -3,7 +3,7 @@
 # Install dependencies required to build and test Git on Linux and macOS
 #
 
-. ${0%/*}/lib.sh
+set -ex
 
 UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
  tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
diff --git a/ci/lib-ci-type.sh b/ci/lib-ci-type.sh
new file mode 100644
index 00000000000..6f01fd9e5d9
--- /dev/null
+++ b/ci/lib-ci-type.sh
@@ -0,0 +1,9 @@
+#!/bin/sh
+
+if test "$GITHUB_ACTIONS" = "true"
+then
+	CI_TYPE=github-actions
+else
+	echo "Could not identify CI type" >&2
+	exit 1
+fi
diff --git a/ci/lib-tput.sh b/ci/lib-tput.sh
new file mode 100644
index 00000000000..b62ef13c44c
--- /dev/null
+++ b/ci/lib-tput.sh
@@ -0,0 +1,5 @@
+#!/bin/sh
+
+# GitHub Action doesn't set TERM, which is required by tput
+TERM=${TERM:-dumb}
+export TERM
diff --git a/ci/lib.sh b/ci/lib.sh
index 8c574b21a67..8f7e178f3ff 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,13 +1,9 @@
 #!/bin/sh
-
-# Library of functions shared by all CI scripts
-
-# Set 'exit on error' for all CI scripts to let the caller know that
-# something went wrong.
-# Set tracing executed commands, primarily setting environment variables
-# and installing dependencies.
 set -ex
 
+# Helper libraries
+. ${0%/*}/lib-ci-type.sh
+
 # Starting assertions
 if test -z "$jobname"
 then
@@ -47,18 +43,14 @@ setenv () {
 	fi
 }
 
-# GitHub Action doesn't set TERM, which is required by tput
-setenv TERM ${TERM:-dumb}
-
 # How many jobs to run in parallel?
 NPROC=10
 
 # Clear MAKEFLAGS that may come from the outside world.
 MAKEFLAGS=--jobs=$NPROC
 
-if test "$GITHUB_ACTIONS" = "true"
-then
-	CI_TYPE=github-actions
+case "$CI_TYPE" in
+github-actions)
 	CC="${CC:-gcc}"
 
 	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
@@ -66,11 +58,12 @@ then
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
-else
-	echo "Could not identify CI type" >&2
-	env >&2
+	;;
+*)
+	echo "Unhandled CI type: $CI_TYPE" >&2
 	exit 1
-fi
+	;;
+esac
 
 setenv --build DEVELOPER 1
 setenv --test DEFAULT_TEST_TARGET prove
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index a878cca67e4..2405f65650a 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -3,10 +3,10 @@
 # Print output of failing tests
 #
 
-. ${0%/*}/lib.sh
+set -e
 
-# Tracing executed commands would produce too much noise in the loop below.
-set +x
+. ${0%/*}/lib-ci-type.sh
+. ${0%/*}/lib-tput.sh
 
 cd t/
 
diff --git a/ci/select-test-slice.sh b/ci/select-test-slice.sh
index ec602f8a053..a0332095bb3 100755
--- a/ci/select-test-slice.sh
+++ b/ci/select-test-slice.sh
@@ -3,7 +3,7 @@
 # Select a portion of the tests for testing Git in parallel
 #
 
-. ${0%/*}/lib.sh
+set -ex
 
 tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \
 	t[0-9]*.sh))
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index b8a6a6f664e..64ff212cdaa 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -3,7 +3,7 @@
 # Perform sanity checks on "make doc" output and built documentation
 #
 
-. ${0%/*}/lib.sh
+set -ex
 
 generator=$1
 
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 23/29] CI: narrow down variable definitions in --build and --test

[Ævar Arnfjörð Bjarmason]

In a preceding step the "setvar" function was made to take a
"--build", "--test" or "--all" argument to indicate where the
variables it sets were used.

Let's make use of that by having the relevant parts of
".github/workflows/main.yml" invoke "ci/lib.sh" with those options.

By doing this the set of variables shown in build-only steps will be
fewer, which makes diagnosing anything going on there easier, as we
won't have to look at a deluge of e.g. GIT_TEST_* variables.

Since we'll now always run "--build" for "make" and "--test" for "make
test" we can stop setting the "fat" MAKEFLAGS entirely on the "test"
steps, i.e. the one with PYTHON_PATH, CC etc. These will all be
carried over from the earlier --build step.

That won't be true in the case of the "windows-test" and "vs-test"
jobs, since they run separately from the corresponding "build"
step. So we'll need to make sure that we have the --jobs=N argument
for those.

This doesn't matter that much, as we'll still have --jobs=N in
GIT_PROVE_OPTS. So the only thing we'll use it for is parallelism in
the t/Makefile before we get to running "prove". Still, it's good to
be consistent for good measure, and to run the t/Makefile itself in
parallel.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 20 +++++++++++---------
 ci/lib.sh                  | 30 +++++++++++++++++++++++-------
 2 files changed, 34 insertions(+), 16 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 1e13718b989..0787cadc76b 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -84,7 +84,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
       shell: bash
     - name: build
       shell: bash
@@ -122,7 +122,7 @@ jobs:
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
-    - run: ci/lib.sh
+    - run: ci/lib.sh --test
       shell: bash
     - name: select tests
       run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
@@ -169,7 +169,7 @@ jobs:
     - name: copy dlls to root
       shell: cmd
       run: compat\vcbuild\vcpkg_copy_dlls.bat release
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
       shell: bash
     - name: generate Visual Studio solution
       shell: bash
@@ -211,7 +211,7 @@ jobs:
     - name: extract tracked files and build artifacts
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
-    - run: ci/lib.sh
+    - run: ci/lib.sh --test
       shell: bash
     - name: select tests
       run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
@@ -275,8 +275,9 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make
+    - run: ci/lib.sh --test
     - run: make test
       if: success()
     - run: ci/print-test-failures.sh
@@ -310,8 +311,9 @@ jobs:
     steps:
     - uses: actions/checkout@v1
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make
+    - run: ci/lib.sh --test
     - run: make test
       if: success() && matrix.vector.skip-tests != 'no'
     - run: ci/print-test-failures.sh
@@ -331,7 +333,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make ci-static-analysis
   sparse:
     needs: ci-config
@@ -352,7 +354,7 @@ jobs:
     - uses: actions/checkout@v2
     - name: Install other dependencies
       run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make sparse
   documentation:
     name: documentation
@@ -364,7 +366,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make check-docs
     - run: "make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)"
       shell: bash
diff --git a/ci/lib.sh b/ci/lib.sh
index 8f7e178f3ff..50bc766254b 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -5,6 +5,13 @@ set -ex
 . ${0%/*}/lib-ci-type.sh
 
 # Starting assertions
+mode=$1
+if test -z "$mode"
+then
+	echo "need a $0 mode, e.g. --build or --test" >&2
+	exit 1
+fi
+
 if test -z "$jobname"
 then
 	echo "must set a CI jobname in the environment" >&2
@@ -16,11 +23,12 @@ setenv () {
 	while test $# != 0
 	do
 		case "$1" in
-		--build)
-			;;
-		--test)
-			;;
-		--all)
+		--build | --test)
+			if test "$1" != "$mode"
+			then
+				return 0
+			fi
+
 			;;
 		-*)
 			echo "BUG: bad setenv() option '$1'" >&2
@@ -46,8 +54,12 @@ setenv () {
 # How many jobs to run in parallel?
 NPROC=10
 
+# For "--test" we carry the MAKEFLAGS over from earlier steps, except
+# in stand-alone jobs which will use $COMMON_MAKEFLAGS.
+COMMON_MAKEFLAGS=--jobs=$NPROC
+
 # Clear MAKEFLAGS that may come from the outside world.
-MAKEFLAGS=--jobs=$NPROC
+MAKEFLAGS=$COMMON_MAKEFLAGS
 
 case "$CI_TYPE" in
 github-actions)
@@ -101,6 +113,9 @@ windows-build)
 	setenv --build NO_PERL NoThanks
 	setenv --build ARTIFACTS_DIRECTORY artifacts
 	;;
+windows-test)
+	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
+	;;
 vs-build)
 	setenv --build NO_PERL NoThanks
 	setenv --build NO_GETTEXT NoThanks
@@ -117,6 +132,7 @@ vs-build)
 	;;
 vs-test)
 	setenv --test NO_SVN_TESTS YesPlease
+	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
 	;;
 linux-gcc)
 	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
@@ -162,4 +178,4 @@ linux-leaks)
 	;;
 esac
 
-setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
+setenv --build MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 24/29] CI: add more variables to MAKEFLAGS, except under vs-build

[Ævar Arnfjörð Bjarmason]

It's clearer that "DEVELOPER" is a flag that affects the Makefile
itself in particular if it's put into "MAKEFLAGS than" if it
generically sits in the environment. Let's move both it and
"SKIP_DASHED_BUILT_INS" to "MAKEFLAGS".

We can't do this under vs-build, since that invokes cmake. Let's have
only that job set these in the environment.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 50bc766254b..8fb0bfd43e1 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -61,6 +61,13 @@ COMMON_MAKEFLAGS=--jobs=$NPROC
 # Clear MAKEFLAGS that may come from the outside world.
 MAKEFLAGS=$COMMON_MAKEFLAGS
 
+# Use common options for "make" (cmake in "vs-build" below uses the
+# intermediate variables directly)
+DEVELOPER=1
+MAKEFLAGS="$MAKEFLAGS DEVELOPER=$DEVELOPER"
+SKIP_DASHED_BUILT_INS=YesPlease
+MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
+
 case "$CI_TYPE" in
 github-actions)
 	CC="${CC:-gcc}"
@@ -77,10 +84,8 @@ github-actions)
 	;;
 esac
 
-setenv --build DEVELOPER 1
 setenv --test DEFAULT_TEST_TARGET prove
 setenv --test GIT_TEST_CLONE_2GB true
-setenv --build SKIP_DASHED_BUILT_INS YesPlease
 
 case "$runs_on_pool" in
 ubuntu-latest)
@@ -117,6 +122,9 @@ windows-test)
 	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
 	;;
 vs-build)
+	setenv --build DEVELOPER $DEVELOPER
+	setenv --build SKIP_DASHED_BUILT_INS $SKIP_DASHED_BUILT_INS
+
 	setenv --build NO_PERL NoThanks
 	setenv --build NO_GETTEXT NoThanks
 	setenv --build ARTIFACTS_DIRECTORY artifacts
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 25/29] CI: set CC in MAKEFLAGS directly, don't add it to the environment

[Ævar Arnfjörð Bjarmason]

Rather than pass a "$CC" and "$CC_PACKAGE" in the environment to be
picked up in ci/lib.sh let's instead have ci/lib.sh itself add it
directly to MAKEFLAGS.

Setting CC=gcc by default made for confusing trace output, and since a
preceding change to carry it and others over across "steps" in the
GitHub CI it's been even more misleading.  E.g. the "win+VS build" job
confusingly has CC=gcc set, even though it builds with MSVC.

Let's instead reply on the Makefile default of CC=cc, and only
override it for those jobs where it's needed. This does mean that
we'll need to set it for the "pedantic" job, which previously relied
on the default CC=gcc in case "clang" become the default on that
platform.

This partially reverts my 707d2f2fe86 (CI: use "$runs_on_pool", not
"$jobname" to select packages & config, 2021-11-23), i.e. we're now
aiming to only set those variables specific jobs need.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 13 -------------
 ci/lib.sh                  | 27 ++++++++++++++++++++++++---
 2 files changed, 24 insertions(+), 16 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 0787cadc76b..6d25ec4ae3b 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -238,37 +238,24 @@ jobs:
       matrix:
         vector:
           - jobname: linux-clang
-            cc: clang
             pool: ubuntu-latest
           - jobname: linux-sha256
-            cc: clang
             os: ubuntu
             pool: ubuntu-latest
           - jobname: linux-gcc
-            cc: gcc
-            cc_package: gcc-8
             pool: ubuntu-latest
           - jobname: linux-TEST-vars
-            cc: gcc
             os: ubuntu
-            cc_package: gcc-8
             pool: ubuntu-latest
           - jobname: osx-clang
-            cc: clang
             pool: macos-latest
           - jobname: osx-gcc
-            cc: gcc
-            cc_package: gcc-9
             pool: macos-latest
           - jobname: linux-gcc-default
-            cc: gcc
             pool: ubuntu-latest
           - jobname: linux-leaks
-            cc: gcc
             pool: ubuntu-latest
     env:
-      CC: ${{matrix.vector.cc}}
-      CC_PACKAGE: ${{matrix.vector.cc_package}}
       jobname: ${{matrix.vector.jobname}}
       runs_on_pool: ${{matrix.vector.pool}}
     runs-on: ${{matrix.vector.pool}}
diff --git a/ci/lib.sh b/ci/lib.sh
index 8fb0bfd43e1..c73b107d9c7 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -51,6 +51,10 @@ setenv () {
 	fi
 }
 
+# Clear variables that may come from the outside world.
+CC=
+CC_PACKAGE=
+
 # How many jobs to run in parallel?
 NPROC=10
 
@@ -70,8 +74,6 @@ MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
 
 case "$CI_TYPE" in
 github-actions)
-	CC="${CC:-gcc}"
-
 	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
 	GIT_TEST_OPTS="--verbose-log -x"
 	test Windows != "$RUNNER_OS" ||
@@ -143,9 +145,16 @@ vs-test)
 	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
 	;;
 linux-gcc)
+	CC=gcc
+	CC_PACKAGE=gcc-8
 	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
 	;;
+linux-gcc-default)
+	CC=gcc
+	;;
 linux-TEST-vars)
+	CC=gcc
+	CC_PACKAGE=gcc-8
 	setenv --test GIT_TEST_SPLIT_INDEX yes
 	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
 	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
@@ -160,13 +169,23 @@ linux-TEST-vars)
 	setenv --test GIT_TEST_WRITE_REV_INDEX 1
 	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
 	;;
+osx-gcc)
+	CC=gcc
+	CC_PACKAGE=gcc-9
+	;;
+osx-clang)
+	CC=clang
+	;;
 linux-clang)
+	CC=clang
 	setenv --test GIT_TEST_DEFAULT_HASH sha1
 	;;
 linux-sha256)
+	CC=clang
 	setenv --test GIT_TEST_DEFAULT_HASH sha256
 	;;
 pedantic)
+	CC=gcc
 	# Don't run the tests; we only care about whether Git can be
 	# built.
 	setenv --build DEVOPTS pedantic
@@ -181,9 +200,11 @@ linux-musl)
 	MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8"
 	;;
 linux-leaks)
+	CC=gcc
 	setenv --build SANITIZE leak
 	setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true
 	;;
 esac
 
-setenv --build MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
+MAKEFLAGS="$MAKEFLAGS${CC:+ CC=$CC}"
+setenv --build MAKEFLAGS "$MAKEFLAGS"
-- 
2.36.0.rc2.843.g193535c2aa7

Re: [PATCH v3 25/29] CI: set CC in MAKEFLAGS directly, don't add it to the environment

[Eric Sunshine]

On 4/13/22 3:51 PM, Ævar Arnfjörð Bjarmason wrote:
> Rather than pass a "$CC" and "$CC_PACKAGE" in the environment to be
> picked up in ci/lib.sh let's instead have ci/lib.sh itself add it
> directly to MAKEFLAGS.
> 
> Setting CC=gcc by default made for confusing trace output, and since a
> preceding change to carry it and others over across "steps" in the
> GitHub CI it's been even more misleading.  E.g. the "win+VS build" job
> confusingly has CC=gcc set, even though it builds with MSVC.
> 
> Let's instead reply on the Makefile default of CC=cc, and only
> override it for those jobs where it's needed. This does mean that
> we'll need to set it for the "pedantic" job, which previously relied
> on the default CC=gcc in case "clang" become the default on that
> platform.

s/reply/rely/

> This partially reverts my 707d2f2fe86 (CI: use "$runs_on_pool", not
> "$jobname" to select packages & config, 2021-11-23), i.e. we're now
> aiming to only set those variables specific jobs need.
> 
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

Re: [PATCH v3 25/29] CI: set CC in MAKEFLAGS directly, don't add it to the environment

[Carlo Arenas]

On Wed, Apr 13, 2022 at 12:52 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> @@ -160,13 +169,23 @@ linux-TEST-vars)
>         setenv --test GIT_TEST_WRITE_REV_INDEX 1
>         setenv --test GIT_TEST_CHECKOUT_WORKERS 2
>         ;;
> +osx-gcc)
> +       CC=gcc
> +       CC_PACKAGE=gcc-9

not sure when this was broken since there were too many refactorings
around this code, but this is definitely wrong.

macos' gcc is really clang, so if we really want to build with gcc
instead (and there are 9, 10 and 11 versions installed) need to use
instead (for version 9, which was what was used originally and what
CC_PACKAGE was installing if needed)

CC=gcc-9

Right now both macos jobs are using clang, regardless of what the
nicely named label says.

Carlo

Re: [PATCH v3 25/29] CI: set CC in MAKEFLAGS directly, don't add it to the environment

[Ævar Arnfjörð Bjarmason]

On Thu, Apr 14 2022, Carlo Arenas wrote:

> On Wed, Apr 13, 2022 at 12:52 PM Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>> @@ -160,13 +169,23 @@ linux-TEST-vars)
>>         setenv --test GIT_TEST_WRITE_REV_INDEX 1
>>         setenv --test GIT_TEST_CHECKOUT_WORKERS 2
>>         ;;
>> +osx-gcc)
>> +       CC=gcc
>> +       CC_PACKAGE=gcc-9
>
> not sure when this was broken since there were too many refactorings
> around this code, but this is definitely wrong.
>
> macos' gcc is really clang, so if we really want to build with gcc
> instead (and there are 9, 10 and 11 versions installed) need to use
> instead (for version 9, which was what was used originally and what
> CC_PACKAGE was installing if needed)
>
> CC=gcc-9
>
> Right now both macos jobs are using clang, regardless of what the
> nicely named label says.

I didn't know gcc on OSX was clang, that does seem broken.

But unless I'm missing something that's already been the case on
"master" for a while, i.e. this is the master run showing that we'll
invoke "gcc":
https://github.com/git/git/runs/6031562726?check_suite_focus=true#step:3:6

And "seen", with this change, which shows that we'll do the same:
https://github.com/git/git/runs/6031564900?check_suite_focus=true#step:5:7

If I understand you correctly both are effectively a NOOP and we don't
use that "gcc-9", but we should.

It looks like it was broken in my 707d2f2fe86 (CI: use "$runs_on_pool",
not "$jobname" to select packages & config, 2021-11-23).

I'll fix that as along with any small follow-ups after this series,
sound good?

Re: [PATCH v3 25/29] CI: set CC in MAKEFLAGS directly, don't add it to the environment

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> I'll fix that as along with any small follow-ups after this series,
> sound good?

Without any small follow-ups would be much much more preferrrable,
and leave the follow-up to a separate series.

A series doing too many things at once can mix bad apples slip in by
overwhelming the reviewer bandwidth, and I am afraid that it may be
what happened in the original "bunch of Makefile updates" that the
bug originated from.  I am guessing that Carlo meant the same when
he said "since there were too many refactorings around this code".

Thanks.

Re: [PATCH v3 25/29] CI: set CC in MAKEFLAGS directly, don't add it to the environment

[Carlo Arenas]

On Fri, Apr 15, 2022 at 6:52 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> But unless I'm missing something that's already been the case on
> "master" for a while, i.e. this is the master run showing that we'll
> invoke "gcc":
> https://github.com/git/git/runs/6031562726?check_suite_focus=true#step:3:6

That would be unfortunate, considering the code in next (and all other
not yet affected) branches still has the definition added in
176441bfb58 (ci: build Git with GCC 9 in the 'osx-gcc' build job,
2019-11-27) and which survived your travis-ci removal series.

Note that the value of CC is only relevant when the code is built, and
GitHub actions provides its own for all tasks and at least this "mock"
run (which adds `$CC --version` before make) with next seems to
indicate the right compiler is still being used there and hopefully
also in all other unaffected runs from all stable branches:

  https://github.com/carenas/git/runs/6041742809

Eitherway, making sure that we really use gcc in the jobs that are
tagged as such will make these extra runs (and their additional CPU
time) less wasteful and should be corrected ASAP.

Carlo

Re: [PATCH v3 25/29] CI: set CC in MAKEFLAGS directly, don't add it to the environment

[Carlo Arenas]

On Fri, Apr 15, 2022 at 12:03 PM Carlo Arenas <carenas@gmail.com> wrote:
>
> On Fri, Apr 15, 2022 at 6:52 AM Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
> >
> > But unless I'm missing something that's already been the case on
> > "master" for a while, i.e. this is the master run showing that we'll
> > invoke "gcc":
> > https://github.com/git/git/runs/6031562726?check_suite_focus=true#step:3:6
>
> That would be unfortunate, considering the code in next (and all other
> not yet affected) branches still have the definition added in
> 176441bfb58 (ci: build Git with GCC 9 in the 'osx-gcc' build job,
> 2019-11-27) and which survived your travis-ci removal series.

Nevermind, my test was flawed, this[1] updated run does show this code
was indeed refactored away and the bug also introduced to next and
beyond.

Carlo

[1] https://github.com/carenas/git/runs/6042506530

[PATCH v3 26/29] CI: set SANITIZE=leak in MAKEFLAGS directly

[Ævar Arnfjörð Bjarmason]

As in the preceding commit's change to do likewise for CC, let's not
set SANITIZE in the environment. When I added the "linux-leaks" job in
in 956d2e4639b (tests: add a test mode for SANITIZE=leak, run it in
CI, 2021-09-23) I missed that I could do that.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index c73b107d9c7..b439c1a4f82 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -201,7 +201,7 @@ linux-musl)
 	;;
 linux-leaks)
 	CC=gcc
-	setenv --build SANITIZE leak
+	MAKEFLAGS="$MAKEFLAGS SANITIZE=leak"
 	setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true
 	;;
 esac
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 27/29] CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case

[Ævar Arnfjörð Bjarmason]

Move the setting of "PYTHON_PATH=[...]" to the "$jobname" case
statement. This partially backs out of my 707d2f2fe86 (CI: use
"$runs_on_pool", not "$jobname" to select packages & config,
2021-11-23), now that we have a "osx-{clang,gcc}" anyway for setting
"$CC" we might as well do away with this part of the "$runs_on_pool"
case.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index b439c1a4f82..d62cd757ef4 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -105,14 +105,6 @@ ubuntu-latest)
 
 	setenv --test GIT_TEST_HTTPD true
 	;;
-macos-latest)
-	if test "$jobname" = osx-gcc
-	then
-		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
-	else
-		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
-	fi
-	;;
 esac
 
 case "$jobname" in
@@ -170,10 +162,12 @@ linux-TEST-vars)
 	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
 	;;
 osx-gcc)
+	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
 	CC=gcc
 	CC_PACKAGE=gcc-9
 	;;
 osx-clang)
+	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
 	CC=clang
 	;;
 linux-clang)
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 28/29] CI: don't use "set -x" in "ci/lib.sh" output

[Ævar Arnfjörð Bjarmason]

Remove the "set -x" from the "ci/lib.sh" output. Before preceding
commits the logic in that file was much more complex, and likely to
fail in some scenarios.

Now we only task "ci/lib.sh" with setting various variables for
subsequent steps in our jobs, so we can start emitting more tailored
debugging output, which makes what it's doing easier to read.

This change also changes the output of the "ci/print-test-failures.sh"
script, since it's the only other user of "ci/lib-ci-type.sh". In that
case it's also helpful to know what "$CI_TYPE" we're using, as that
script doesn't "set -x" and will act differently depending on the
$CI_TYPE.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 23 +++++++++++++++++++++--
 1 file changed, 21 insertions(+), 2 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index d62cd757ef4..94f83069884 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,5 +1,5 @@
 #!/bin/sh
-set -ex
+set -e
 
 # Helper libraries
 . ${0%/*}/lib-ci-type.sh
@@ -18,15 +18,26 @@ then
 	exit 1
 fi
 
+# Show our configuration
+echo "CONFIG: CI_TYPE=$CI_TYPE" >&2
+echo "CONFIG: jobname=$jobname" >&2
+echo "CONFIG: runs_on_pool=$runs_on_pool" >&2
+echo "CONFIG: GITHUB_ENV=$GITHUB_ENV" >&2
+if test -n "$GITHUB_ENV"
+then
+	echo "CONFIG: GITHUB_ENV=$GITHUB_ENV" >&2
+fi
+
 # Helper functions
 setenv () {
+	local skip=
 	while test $# != 0
 	do
 		case "$1" in
 		--build | --test)
 			if test "$1" != "$mode"
 			then
-				return 0
+				skip=t
 			fi
 
 			;;
@@ -45,10 +56,18 @@ setenv () {
 	val=$2
 	shift 2
 
+	if test -n "$skip"
+	then
+		echo "SKIP '$key=$val'" >&2
+		return 0
+	fi
+
 	if test -n "$GITHUB_ENV"
 	then
 		echo "$key=$val" >>"$GITHUB_ENV"
 	fi
+
+	echo "SET: '$key=$val'" >&2
 }
 
 # Clear variables that may come from the outside world.
-- 
2.36.0.rc2.843.g193535c2aa7

[PATCH v3 29/29] CI: make it easy to use ci/*.sh outside of CI

[Ævar Arnfjörð Bjarmason]

In preceding commits the ci/.sh scripts have lost most of their
CI-specific assumptions. Let's go even further and explicitly support
running ci/lib.sh outside of CI.

This was possible before by faking up enough CI-specific variables,
but as shown in the new "help" output being added here using the
ci/lib.sh to provide "CI-like" has now become trivial.

The ci/print-test-failures.sh scripts can now be used outside of CI as
well, the only GitHub CI-specific part is now guarded by a check that
we'll pass if outside of GitHub CI.

There's also a special-case here to not clobber $MAKEFLAGS in the
environment if we're outside of CI, in case the user has e.g. "jN" or
other flags to "make" that they'd prefer configured already.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib-ci-type.sh         |  3 --
 ci/lib.sh                 | 61 ++++++++++++++++++++++++++++++++++++---
 ci/print-test-failures.sh | 11 ++-----
 3 files changed, 60 insertions(+), 15 deletions(-)

diff --git a/ci/lib-ci-type.sh b/ci/lib-ci-type.sh
index 6f01fd9e5d9..09acab7aaec 100644
--- a/ci/lib-ci-type.sh
+++ b/ci/lib-ci-type.sh
@@ -3,7 +3,4 @@
 if test "$GITHUB_ACTIONS" = "true"
 then
 	CI_TYPE=github-actions
-else
-	echo "Could not identify CI type" >&2
-	exit 1
 fi
diff --git a/ci/lib.sh b/ci/lib.sh
index 94f83069884..3fd5291bc80 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,6 +1,30 @@
 #!/bin/sh
 set -e
 
+#  Usage
+CI_TYPE_HELP_COMMANDS='
+	# run "make all test" like the "linux-leaks" job
+	(eval $(jobname=linux-leaks ci/lib.sh --all) && make test)
+
+	# run "make all test" like the "linux-musl" job
+	(eval $(jobname=linux-musl ci/lib.sh --all) && make test)
+
+	# run "make test" like the "linux-TEST-vars" job (uses various GIT_TEST_* modes)
+	make && (eval $(jobname=linux-TEST-vars ci/lib.sh --test) && make test)
+
+	# run "make test" like the "linux-sha256" job
+	make && (eval $(jobname=linux-sha256 ci/lib.sh --test) && make test)
+'
+
+CI_TYPE_HELP="
+running $0 outside of CI? You can use ci/lib.sh to set up your
+environment like a given CI job. E.g.:
+$CI_TYPE_HELP_COMMANDS
+
+note that some of these (e.g. the linux-musl one) may not work as
+expected due to the CI job configuring a platform that may not match
+yours."
+
 # Helper libraries
 . ${0%/*}/lib-ci-type.sh
 
@@ -9,6 +33,10 @@ mode=$1
 if test -z "$mode"
 then
 	echo "need a $0 mode, e.g. --build or --test" >&2
+	if test -z "$CI_TYPE"
+	then
+		echo "$CI_TYPE_HELP" >&2
+	fi
 	exit 1
 fi
 
@@ -35,7 +63,7 @@ setenv () {
 	do
 		case "$1" in
 		--build | --test)
-			if test "$1" != "$mode"
+			if test "$1" != "$mode" && test "$mode" != "--all"
 			then
 				skip=t
 			fi
@@ -65,6 +93,10 @@ setenv () {
 	if test -n "$GITHUB_ENV"
 	then
 		echo "$key=$val" >>"$GITHUB_ENV"
+	elif test -z "$CI_TYPE"
+	then
+		echo "$key=\"$val\""
+		echo "export $key"
 	fi
 
 	echo "SET: '$key=$val'" >&2
@@ -76,10 +108,29 @@ CC_PACKAGE=
 
 # How many jobs to run in parallel?
 NPROC=10
+case "$CI_TYPE" in
+'')
+	if command -v nproc >/dev/null
+	then
+		NPROC=$(nproc)
+	else
+		NPROC=1
+	fi
+
+	if test -n "$MAKEFLAGS"
+	then
+		COMMON_MAKEFLAGS="$MAKEFLAGS"
+	else
+		COMMON_MAKEFLAGS=--jobs=$NPROC
+	fi
+	;;
+*)
+	# For "--test" we carry the MAKEFLAGS over from earlier steps, except
+	# in stand-alone jobs which will use $COMMON_MAKEFLAGS.
+	COMMON_MAKEFLAGS=--jobs=$NPROC
+	;;
+esac
 
-# For "--test" we carry the MAKEFLAGS over from earlier steps, except
-# in stand-alone jobs which will use $COMMON_MAKEFLAGS.
-COMMON_MAKEFLAGS=--jobs=$NPROC
 
 # Clear MAKEFLAGS that may come from the outside world.
 MAKEFLAGS=$COMMON_MAKEFLAGS
@@ -99,6 +150,8 @@ github-actions)
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
 	;;
+'')
+	;;
 *)
 	echo "Unhandled CI type: $CI_TYPE" >&2
 	exit 1
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 2405f65650a..ba8428ad484 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -38,19 +38,14 @@ do
 		test_name="${TEST_EXIT%.exit}"
 		test_name="${test_name##*/}"
 		trash_dir="trash directory.$test_name"
-		case "$CI_TYPE" in
-		github-actions)
+		if test "$CI_TYPE" = "github-actions"
+		then
 			mkdir -p failed-test-artifacts
 			echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
 			cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
 			tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
 			continue
-			;;
-		*)
-			echo "Unhandled CI type: $CI_TYPE" >&2
-			exit 1
-			;;
-		esac
+		fi
 		trash_tgz_b64="trash.$test_name.base64"
 		if [ -d "$trash_dir" ]
 		then
-- 
2.36.0.rc2.843.g193535c2aa7

Re: [PATCH v3 29/29] CI: make it easy to use ci/*.sh outside of CI

[Eric Sunshine]

On 4/13/22 3:51 PM, Ævar Arnfjörð Bjarmason wrote:
 > In preceding commits the ci/.sh scripts have lost most of their
 > CI-specific assumptions. Let's go even further and explicitly support
 > running ci/lib.sh outside of CI.
 >
 > This was possible before by faking up enough CI-specific variables,
 > but as shown in the new "help" output being added here using the
 > ci/lib.sh to provide "CI-like" has now become trivial.
 >
 > The ci/print-test-failures.sh scripts can now be used outside of CI as
 > well, the only GitHub CI-specific part is now guarded by a check that
 > we'll pass if outside of GitHub CI.
 >
 > There's also a special-case here to not clobber $MAKEFLAGS in the
 > environment if we're outside of CI, in case the user has e.g. "jN" or
 > other flags to "make" that they'd prefer configured already.
 >
 > Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
 > ---
 > diff --git a/ci/lib.sh b/ci/lib.sh
 > @@ -1,6 +1,30 @@
 > +#  Usage
 > +CI_TYPE_HELP_COMMANDS='
 > +	# run "make all test" like the "linux-leaks" job
 > +	(eval $(jobname=linux-leaks ci/lib.sh --all) && make test)
 > +
 > +	# run "make all test" like the "linux-musl" job
 > +	(eval $(jobname=linux-musl ci/lib.sh --all) && make test)
 > +
 > +	# run "make test" like the "linux-TEST-vars" job (uses various 
GIT_TEST_* modes)
 > +	make && (eval $(jobname=linux-TEST-vars ci/lib.sh --test) && make test)
 > +
 > +	# run "make test" like the "linux-sha256" job
 > +	make && (eval $(jobname=linux-sha256 ci/lib.sh --test) && make test)
 > +'
 > +
 > +CI_TYPE_HELP="
 > +running $0 outside of CI? You can use ci/lib.sh to set up your
 > +environment like a given CI job. E.g.:
 > +$CI_TYPE_HELP_COMMANDS
 > +
 > +note that some of these (e.g. the linux-musl one) may not work as
 > +expected due to the CI job configuring a platform that may not match
 > +yours."
 > +
 > @@ -9,6 +33,10 @@ mode=$1
 >   if test -z "$mode"
 >   then
 >   	echo "need a $0 mode, e.g. --build or --test" >&2
 > +	if test -z "$CI_TYPE"
 > +	then
 > +		echo "$CI_TYPE_HELP" >&2
 > +	fi
 >   	exit 1
 >   fi

It would never occur to me to try running a script named ci/lib.sh in
the first place, and I'm not sure I would even think to poke around in
the `ci` directory with the idea of being able to run CI tasks
locally. Would it make sense to aid discovery by mentioning this new
feature in the "GitHub CI" section of SubmittingPatches as a follow up
to this series? (If so, perhaps the "GitHub CI" section should be
renamed to "CI".)

 > @@ -76,10 +108,29 @@ CC_PACKAGE=
 >   # How many jobs to run in parallel?
 >   NPROC=10
 > +case "$CI_TYPE" in
 > +'')
 > +	if command -v nproc >/dev/null

You need to redirect stderr too in order to avoid a scary error
message on platforms lacking `nproc`:

     if command -v nproc >/dev/null 2>&1

 > +	then
 > +		NPROC=$(nproc)
 > +	else
 > +		NPROC=1
 > +	fi

Neither macOS nor BSD has an `nproc` command, however, they do have
`sysctl`:

     elif command -v sysctl >/dev/null 2>&1
     then # macOS & BSD
         NPROC=$(sysctl -n hw.ncpu 2>/dev/null)

and Windows provides a suitable environment variable:

     elif test -n "$NUMBER_OF_PROCESSORS"
     then # Windows
         NPROC="$NUMBER_OF_PROCESSORS"

 > +	if test -n "$MAKEFLAGS"
 > +	then
 > +		COMMON_MAKEFLAGS="$MAKEFLAGS"
 > +	else
 > +		COMMON_MAKEFLAGS=--jobs=$NPROC
 > +	fi
 > +	;;

Re: [PATCH v3 29/29] CI: make it easy to use ci/*.sh outside of CI

[Ævar Arnfjörð Bjarmason]

On Thu, Apr 14 2022, Eric Sunshine wrote:

> On 4/13/22 3:51 PM, Ævar Arnfjörð Bjarmason wrote:
>> In preceding commits the ci/.sh scripts have lost most of their
>> CI-specific assumptions. Let's go even further and explicitly support
>> running ci/lib.sh outside of CI.
>>
>> This was possible before by faking up enough CI-specific variables,
>> but as shown in the new "help" output being added here using the
>> ci/lib.sh to provide "CI-like" has now become trivial.
>>
>> The ci/print-test-failures.sh scripts can now be used outside of CI as
>> well, the only GitHub CI-specific part is now guarded by a check that
>> we'll pass if outside of GitHub CI.
>>
>> There's also a special-case here to not clobber $MAKEFLAGS in the
>> environment if we're outside of CI, in case the user has e.g. "jN" or
>> other flags to "make" that they'd prefer configured already.
>>
>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>> ---
>> diff --git a/ci/lib.sh b/ci/lib.sh
>> @@ -1,6 +1,30 @@
>> +#  Usage
>> +CI_TYPE_HELP_COMMANDS='
>> +	# run "make all test" like the "linux-leaks" job
>> +	(eval $(jobname=linux-leaks ci/lib.sh --all) && make test)
>> +
>> +	# run "make all test" like the "linux-musl" job
>> +	(eval $(jobname=linux-musl ci/lib.sh --all) && make test)
>> +
>> +	# run "make test" like the "linux-TEST-vars" job (uses various
>   GIT_TEST_* modes)
>> +	make && (eval $(jobname=linux-TEST-vars ci/lib.sh --test) && make test)
>> +
>> +	# run "make test" like the "linux-sha256" job
>> +	make && (eval $(jobname=linux-sha256 ci/lib.sh --test) && make test)
>> +'
>> +
>> +CI_TYPE_HELP="
>> +running $0 outside of CI? You can use ci/lib.sh to set up your
>> +environment like a given CI job. E.g.:
>> +$CI_TYPE_HELP_COMMANDS
>> +
>> +note that some of these (e.g. the linux-musl one) may not work as
>> +expected due to the CI job configuring a platform that may not match
>> +yours."
>> +
>> @@ -9,6 +33,10 @@ mode=$1
>>   if test -z "$mode"
>>   then
>>   	echo "need a $0 mode, e.g. --build or --test" >&2
>> +	if test -z "$CI_TYPE"
>> +	then
>> +		echo "$CI_TYPE_HELP" >&2
>> +	fi
>>   	exit 1
>>   fi
>
> It would never occur to me to try running a script named ci/lib.sh in
> the first place, and I'm not sure I would even think to poke around in
> the `ci` directory with the idea of being able to run CI tasks
> locally. Would it make sense to aid discovery by mentioning this new
> feature in the "GitHub CI" section of SubmittingPatches as a follow up
> to this series? (If so, perhaps the "GitHub CI" section should be
> renamed to "CI".)

Yes, I'd like to do that, but as a follow-up, specifically to have
failing tests in CI suggest a way to run them locally.

It would be better as ci/config.sh or something, but it was ci/lib.sh
already, and this way minimized the diff/conflicts.

>> @@ -76,10 +108,29 @@ CC_PACKAGE=
>>   # How many jobs to run in parallel?
>>   NPROC=10
>> +case "$CI_TYPE" in
>> +'')
>> +	if command -v nproc >/dev/null
>
> You need to redirect stderr too in order to avoid a scary error
> message on platforms lacking `nproc`:
>
>     if command -v nproc >/dev/null 2>&1

I figured the scary error would be useful, it's only if you run it
ad-hoc, so it would be a nice hint as to why we fallback on the default
nproc.

>> +	then
>> +		NPROC=$(nproc)
>> +	else
>> +		NPROC=1
>> +	fi
>
> Neither macOS nor BSD has an `nproc` command, however, they do have
> `sysctl`:
>
>     elif command -v sysctl >/dev/null 2>&1
>     then # macOS & BSD
>         NPROC=$(sysctl -n hw.ncpu 2>/dev/null)
>
> and Windows provides a suitable environment variable:
>
>     elif test -n "$NUMBER_OF_PROCESSORS"
>     then # Windows
>         NPROC="$NUMBER_OF_PROCESSORS"

*nod*

I'd rather not finish this part up though and instead just have this
call some moral equivalent of t/helper/test-online-cpus.c.

Most of our CI is linux, so just nproc for now felt like a good initial
addition which we could expand on later.

>> +	if test -n "$MAKEFLAGS"
>> +	then
>> +		COMMON_MAKEFLAGS="$MAKEFLAGS"
>> +	else
>> +		COMMON_MAKEFLAGS=--jobs=$NPROC
>> +	fi
>> +	;;

Re: [PATCH v3 00/29] CI: run "make" in CI "steps", improve UX

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

>  * The rather minor in code but rather big in UX terms is that I've
>    added 29/29, which explicitly documents and shows examples of using
>    the CI tooling in a way where you can locally run a CI target as if
>    though it were a given $jobname, e.g.:

Yay.

>    Likewise the ci/print-test-failures.sh script can now be run
>    locally without any extra mock-ups, e.g.:

Double yay.

>     @@ ci/lib.sh
>      +# Starting assertions
>      +if test -z "$jobname"
>      +then
>     -+	echo "must set a CI jobname" >&2
>     ++	echo "must set a CI jobname in the environment" >&2
>      +	exit 1
>      +fi

OK.  In the truely-automated-machine-initiated run, this should not
matter, but once we start allowing end-user to interact with the
guts of the CI machinery, being as helpful as possible matters.

[PATCH v4 00/31] CI: run "make" in CI "steps", improve UX

[Ævar Arnfjörð Bjarmason]

This series changes CI "step" targets that are shellscripts that do N
things to instead be single command invocations at the "step" level,
driven by the CI recipe itself.

To do that we need to pass state that we previously re-setup for every
"step" via $GITHUB_ENV, whose state we then helpfully show (this is just
a standard GitHub CI feature) in a drop-down at the start of every
"step".

I.e. at the tip of this series you can reliably look at that
$GITHUB_ENV view to see what the full and relevant environment was for
that "make", "make test" or whatever.

Changes since v3:

 * Typo/grammar fixes pointed out by Eric Sunshine

 * The ci/print-test-failures.sh scripts still had some dead Travis
   code (noticed while re-rolling). Removed it, and squashed into the
   travis removal commit.

 * Split up the part of "CI: combine
   ci/install{,-docker}-dependencies.sh" that was replacing bash
   constructs with POSIX ones into a preceding commit doing only that.

 * There was a regression in "CI: set CC in MAKEFLAGS directly, don't
   add it to the environment" where we didn't properly install the
   right CI package. The whole CC_PACKAGE is now contained within
   "ci/install-dependencies.sh".

 * In addition to the regression Carlo noted already on "master" where
   the osx-gcc job is using clang. That's now fixed in a new "CI: have
   osx-gcc use gcc, not clang".

 * Extend the last commit to have an "nproc" on BSD/OSX/Windows too,
   this is only for the case of ad-hoc running ci/lib.sh locally.

Ævar Arnfjörð Bjarmason (31):
  CI: run "set -ex" early in ci/lib.sh
  CI: make "$jobname" explicit, remove fallback
  CI: remove more dead Travis CI support
  CI: remove dead "tree skipping" code
  CI: remove unused Azure ci/* code
  CI/lib.sh: stop adding leading whitespace to $MAKEFLAGS
  CI: don't have "git grep" invoke a pager in tree content check
  CI: have "static-analysis" run a "make ci-static-analysis" target
  CI: have "static-analysis" run "check-builtins", not "documentation"
  CI: move p4 and git-lfs variables to ci/install-dependencies.sh
  CI: consistently use "export" in ci/lib.sh
  CI: export variables via a wrapper
  CI: remove "run-build-and-tests.sh", run "make [test]" directly
  ci/lib.sh: use "test" instead of "["
  CI: check ignored unignored build artifacts in "win[+VS] build" too
  CI: invoke "make artifacts-tar" directly in windows-build
  CI: split up and reduce "ci/test-documentation.sh"
  CI: make ci/install-dependencies.sh POSIX-compatible
  CI: combine ci/install{,-docker}-dependencies.sh
  CI: move "env" definitions into ci/lib.sh
  ci/run-test-slice.sh: replace shelling out with "echo"
  CI: pre-select test slice in Windows & VS tests
  CI: only invoke ci/lib.sh as "steps" in main.yml
  CI: narrow down variable definitions in --build and --test
  CI: add more variables to MAKEFLAGS, except under vs-build
  CI: set CC in MAKEFLAGS directly, don't add it to the environment
  CI: set SANITIZE=leak in MAKEFLAGS directly
  CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case
  CI: don't use "set -x" in "ci/lib.sh" output
  CI: have osx-gcc use gcc, not clang
  CI: make it easy to use ci/*.sh outside of CI

 .github/workflows/main.yml            | 100 ++++---
 Makefile                              |  31 ++-
 ci/check-directional-formatting.bash  |   2 +-
 ci/check-unignored-build-artifacts.sh |  20 ++
 ci/install-dependencies.sh            |  74 ++++-
 ci/install-docker-dependencies.sh     |  22 --
 ci/lib-ci-type.sh                     |   6 +
 ci/lib-online_cpus.sh                 |  26 ++
 ci/lib-tput.sh                        |   5 +
 ci/lib.sh                             | 379 +++++++++++++++-----------
 ci/make-test-artifacts.sh             |  12 -
 ci/mount-fileshare.sh                 |  25 --
 ci/print-test-failures.sh             |  53 +---
 ci/run-build-and-tests.sh             |  54 ----
 ci/run-docker-build.sh                |  66 -----
 ci/run-docker.sh                      |  47 ----
 ci/run-static-analysis.sh             |  32 ---
 ci/run-test-slice.sh                  |  17 --
 ci/select-test-slice.sh               |  10 +
 ci/test-documentation.sh              |  37 +--
 ci/util/extract-trash-dirs.sh         |  50 ----
 shared.mak                            |   1 +
 22 files changed, 453 insertions(+), 616 deletions(-)
 create mode 100755 ci/check-unignored-build-artifacts.sh
 delete mode 100755 ci/install-docker-dependencies.sh
 create mode 100644 ci/lib-ci-type.sh
 create mode 100644 ci/lib-online_cpus.sh
 create mode 100644 ci/lib-tput.sh
 delete mode 100755 ci/make-test-artifacts.sh
 delete mode 100755 ci/mount-fileshare.sh
 delete mode 100755 ci/run-build-and-tests.sh
 delete mode 100755 ci/run-docker-build.sh
 delete mode 100755 ci/run-docker.sh
 delete mode 100755 ci/run-static-analysis.sh
 delete mode 100755 ci/run-test-slice.sh
 create mode 100755 ci/select-test-slice.sh
 delete mode 100755 ci/util/extract-trash-dirs.sh

Range-diff against v3:
 1:  3cb4749c5ea =  1:  ed27ca0b7e7 CI: run "set -ex" early in ci/lib.sh
 2:  7de95ff437e =  2:  737fe1e2c4c CI: make "$jobname" explicit, remove fallback
 3:  820c4e551d9 !  3:  d875069fc28 CI: remove more dead Travis CI support
    @@ Commit message
         would do the wrong thing, because the different test slices would each
         try to clobber the same "t/.prove" file.
     
    -    If a subsequent run then used the -"-state=failed,slow,save" it would
    +    If a subsequent run then used the "--state=failed,slow,save" it would
         defeat the purpose of "ci/run-test-slice.sh", since all slices would
         then run all tests. I.e. behavior of prove's "--state" options is to
         select tests to run from the provided "--state" file, in addition to
    @@ Commit message
         ".github/workflows/main.yml" calls the same entry points as the main
         "regular" job.
     
    +    For "ci/print-test-failures.sh" the creation of the tarball added in
    +    aea8879a6ac (travis-ci: include the trash directories of failed tests
    +    in the trace log, 2018-08-01) has been dead since my 4a6e4b96026,
    +    which removed the fall-through case from the "case/esac" statement.
    +
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## ci/lib.sh ##
    @@ ci/lib.sh: then
      	echo "Could not identify CI type" >&2
     
      ## ci/print-test-failures.sh ##
    -@@ ci/print-test-failures.sh: do
    - 			fi
    - 			combined_trash_size=$new_combined_trash_size
    +@@ ci/print-test-failures.sh: then
    + 	exit
    + fi
      
    +-case "$jobname" in
    +-osx-clang|osx-gcc)
    +-	# base64 in OSX doesn't wrap its output at 76 columns by
    +-	# default, but prints a single, very long line.
    +-	base64_opts="-b 76"
    +-	;;
    +-esac
    +-
    +-combined_trash_size=0
    + for TEST_EXIT in test-results/*.exit
    + do
    + 	if [ "$(cat "$TEST_EXIT")" != "0" ]
    +@@ ci/print-test-failures.sh: do
    + 		azure-pipelines)
    + 			mkdir -p failed-test-artifacts
    + 			mv "$trash_dir" failed-test-artifacts
    +-			continue
    + 			;;
    + 		github-actions)
    + 			mkdir -p failed-test-artifacts
    + 			echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
    + 			cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
    + 			tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
    +-			continue
    + 			;;
    + 		*)
    + 			echo "Unhandled CI type: $CI_TYPE" >&2
    + 			exit 1
    + 			;;
    + 		esac
    +-		trash_tgz_b64="trash.$test_name.base64"
    +-		if [ -d "$trash_dir" ]
    +-		then
    +-			tar czp "$trash_dir" |base64 $base64_opts >"$trash_tgz_b64"
    +-
    +-			trash_size=$(wc -c <"$trash_tgz_b64")
    +-			if [ $trash_size -gt 1048576 ]
    +-			then
    +-				# larger than 1MB
    +-				echo "$(tput setaf 1)Didn't include the trash directory of '$test_name' in the trace log, it's too big$(tput sgr0)"
    +-				continue
    +-			fi
    +-
    +-			new_combined_trash_size=$(($combined_trash_size + $trash_size))
    +-			if [ $new_combined_trash_size -gt 1048576 ]
    +-			then
    +-				echo "$(tput setaf 1)Didn't include the trash directory of '$test_name' in the trace log, there is plenty of trash in there already.$(tput sgr0)"
    +-				continue
    +-			fi
    +-			combined_trash_size=$new_combined_trash_size
    +-
     -			# DO NOT modify these two 'echo'-ed strings below
     -			# without updating 'ci/util/extract-trash-dirs.sh'
     -			# as well.
    - 			echo "$(tput setaf 1)Start of trash directory of '$test_name':$(tput sgr0)"
    - 			cat "$trash_tgz_b64"
    - 			echo "$(tput setaf 1)End of trash directory of '$test_name'$(tput sgr0)"
    +-			echo "$(tput setaf 1)Start of trash directory of '$test_name':$(tput sgr0)"
    +-			cat "$trash_tgz_b64"
    +-			echo "$(tput setaf 1)End of trash directory of '$test_name'$(tput sgr0)"
    +-		fi
    + 	fi
    + done
     
      ## ci/run-build-and-tests.sh ##
     @@
 4:  914e81118b8 =  4:  2590459db33 CI: remove dead "tree skipping" code
 5:  b2e456f9a9b !  5:  e311ea8b9b3 CI: remove unused Azure ci/* code
    @@ ci/print-test-failures.sh: do
     -		azure-pipelines)
     -			mkdir -p failed-test-artifacts
     -			mv "$trash_dir" failed-test-artifacts
    --			continue
     -			;;
      		github-actions)
      			mkdir -p failed-test-artifacts
 6:  30968e36bdd =  6:  151f28e3818 CI/lib.sh: stop adding leading whitespace to $MAKEFLAGS
 7:  cc81b9fe37e =  7:  6c2a8ee9c4e CI: don't have "git grep" invoke a pager in tree content check
 8:  2387b0c5842 =  8:  64a0a1fc8ce CI: have "static-analysis" run a "make ci-static-analysis" target
 9:  54a4d79bf8d =  9:  0f780bf9e3c CI: have "static-analysis" run "check-builtins", not "documentation"
10:  9a31b7d5011 = 10:  3cdeae9c141 CI: move p4 and git-lfs variables to ci/install-dependencies.sh
11:  8ab4e81e1ca = 11:  1d3d357dcef CI: consistently use "export" in ci/lib.sh
12:  9fe00566767 = 12:  5ef0756d095 CI: export variables via a wrapper
13:  13feda050c0 ! 13:  e0ce614eb6d CI: remove "run-build-and-tests.sh", run "make [test]" directly
    @@ Commit message
     
            Now that verbosity is in the earlier "ci/lib.sh" step, and not in
            any subsequent one. The "make" targets then start out with the
    -       relevant output non-trace output right away.
    +       relevant non-trace output right away.
     
          * If we do want to use the grouping syntax within a "step" it'll now
            be easier to do so. It doesn't support nesting, so we'd have to
14:  d3909b8e896 = 14:  a8e70124929 ci/lib.sh: use "test" instead of "["
15:  d63a689eae0 = 15:  bb335d1c0ca CI: check ignored unignored build artifacts in "win[+VS] build" too
16:  ac1ffe9c642 = 16:  5458e6dab0b CI: invoke "make artifacts-tar" directly in windows-build
17:  8837bfa5433 ! 17:  5e5a5b71700 CI: split up and reduce "ci/test-documentation.sh"
    @@ Commit message
         parts in as one command in the CI job itself, and to run the two "make
         doc" commands at the top-level.
     
    -    It'll now be obvious from the title of the step if if we failed in the
    +    It'll now be obvious from the title of the step if we failed in the
         asciidoc or asciidoctor step.
     
         Since the "check_unignored_build_artifacts()" function is now only
 -:  ----------- > 18:  eec7a11376e CI: make ci/install-dependencies.sh POSIX-compatible
18:  aa491990c1a ! 19:  a9c98582de1 CI: combine ci/install{,-docker}-dependencies.sh
    @@ Commit message
         statement in the latter only cared about "$jobname", and can be folded
         into the same "case" statement in the former.
     
    -    The reason they split up is historical, and because the
    -    "ci/install-dependencies.sh" used "ci/lib.sh", which requires
    -    "bash". At least one of the docker containers doesn't have "bash". To
    -    make the existing code POSIX-compatible we need to replace pushd/popd
    -    with a sub-shell, but no other changes were needed.
    +    The reason they split up is historical, until a preceding commit
    +    "ci/lib.sh" required "bash", which might not have been available in
    +    "docker".
     
         This also fixes issue in "ci/install-docker-dependencies.sh" where
         we'd hide errors due to not using "set -e". Now that we include
    @@ .github/workflows/main.yml: jobs:
          - run: make test
     
      ## ci/install-dependencies.sh ##
    -@@
    --#!/usr/bin/env bash
    -+#!/bin/sh
    - #
    - # Install dependencies required to build and test Git on Linux and macOS
    - #
    -@@ ci/install-dependencies.sh: ubuntu-latest)
    - 	sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \
    - 		$UBUNTU_COMMON_PKGS $CC_PACKAGE
    - 	mkdir --parents "$P4_PATH"
    --	pushd "$P4_PATH"
    -+	(
    -+		cd "$P4_PATH"
    - 		wget --quiet "$P4WHENCE/bin.linux26x86_64/p4d"
    - 		wget --quiet "$P4WHENCE/bin.linux26x86_64/p4"
    - 		chmod u+x p4d
    - 		chmod u+x p4
    --	popd
    -+	)
    - 	mkdir --parents "$GIT_LFS_PATH"
    --	pushd "$GIT_LFS_PATH"
    -+	(
    -+		cd "$GIT_LFS_PATH"
    - 		wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
    - 		tar --extract --gunzip --file "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
    - 		cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs .
    --	popd
    -+	)
    - 	;;
    - macos-latest)
    - 	export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
     @@ ci/install-dependencies.sh: linux-gcc-default)
      	sudo apt-get -q update
      	sudo apt-get -q -y install $UBUNTU_COMMON_PKGS
19:  39b13298425 = 20:  96e7eb68906 CI: move "env" definitions into ci/lib.sh
20:  0a92c478d28 = 21:  fdbe8554f5d ci/run-test-slice.sh: replace shelling out with "echo"
21:  16ac7f5c210 = 22:  f257ab59971 CI: pre-select test slice in Windows & VS tests
22:  939ef573b75 = 23:  1d95088da29 CI: only invoke ci/lib.sh as "steps" in main.yml
23:  8e79c3e39e0 = 24:  a7434b215db CI: narrow down variable definitions in --build and --test
24:  d1d5c1e2f65 = 25:  78f4609a9c1 CI: add more variables to MAKEFLAGS, except under vs-build
25:  1268792233f ! 26:  26a34f1d4b9 CI: set CC in MAKEFLAGS directly, don't add it to the environment
    @@ Metadata
      ## Commit message ##
         CI: set CC in MAKEFLAGS directly, don't add it to the environment
     
    -    Rather than pass a "$CC" and "$CC_PACKAGE" in the environment to be
    -    picked up in ci/lib.sh let's instead have ci/lib.sh itself add it
    -    directly to MAKEFLAGS.
    +    Rather than pass a "$CC" in the environment to be picked up in
    +    ci/lib.sh let's instead have ci/lib.sh itself add it directly to
    +    MAKEFLAGS. For "$CC_PACKAGE" its setting and use can stay within
    +    ci/install-dependencies.sh.
     
         Setting CC=gcc by default made for confusing trace output, and since a
         preceding change to carry it and others over across "steps" in the
         GitHub CI it's been even more misleading.  E.g. the "win+VS build" job
         confusingly has CC=gcc set, even though it builds with MSVC.
     
    -    Let's instead reply on the Makefile default of CC=cc, and only
    -    override it for those jobs where it's needed. This does mean that
    -    we'll need to set it for the "pedantic" job, which previously relied
    -    on the default CC=gcc in case "clang" become the default on that
    -    platform.
    +    Let's instead rely on the Makefile default of CC=cc, and only override
    +    it for those jobs where it's needed. This does mean that we'll need to
    +    set it for the "pedantic" job, which previously relied on the default
    +    CC=gcc in case "clang" become the default on that platform.
     
         This partially reverts my 707d2f2fe86 (CI: use "$runs_on_pool", not
         "$jobname" to select packages & config, 2021-11-23), i.e. we're now
    @@ .github/workflows/main.yml: jobs:
            runs_on_pool: ${{matrix.vector.pool}}
          runs-on: ${{matrix.vector.pool}}
     
    + ## ci/install-dependencies.sh ##
    +@@ ci/install-dependencies.sh: UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
    +  tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
    +  libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl"
    + 
    ++CC_PACKAGE=
    ++case "$jobname" in
    ++linux-gcc | linux-TEST-vars)
    ++	CC_PACKAGE=gcc-8
    ++	;;
    ++osx-gcc)
    ++	CC_PACKAGE=gcc-9
    ++	;;
    ++esac
    ++
    + case "$runs_on_pool" in
    + ubuntu-latest)
    + 	# The Linux build installs the defined dependency versions below.
    +
      ## ci/lib.sh ##
     @@ ci/lib.sh: setenv () {
      	fi
    @@ ci/lib.sh: setenv () {
      
     +# Clear variables that may come from the outside world.
     +CC=
    -+CC_PACKAGE=
     +
      # How many jobs to run in parallel?
      NPROC=10
    @@ ci/lib.sh: vs-test)
      	;;
      linux-gcc)
     +	CC=gcc
    -+	CC_PACKAGE=gcc-8
      	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
      	;;
     +linux-gcc-default)
    @@ ci/lib.sh: vs-test)
     +	;;
      linux-TEST-vars)
     +	CC=gcc
    -+	CC_PACKAGE=gcc-8
      	setenv --test GIT_TEST_SPLIT_INDEX yes
      	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
      	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
    @@ ci/lib.sh: linux-TEST-vars)
      	;;
     +osx-gcc)
     +	CC=gcc
    -+	CC_PACKAGE=gcc-9
     +	;;
     +osx-clang)
     +	CC=clang
26:  83138dacd3e = 27:  514de8d16b0 CI: set SANITIZE=leak in MAKEFLAGS directly
27:  0508777c68e ! 28:  6501059c594 CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case
    @@ ci/lib.sh: linux-TEST-vars)
      osx-gcc)
     +	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
      	CC=gcc
    - 	CC_PACKAGE=gcc-9
      	;;
      osx-clang)
     +	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
28:  c0ce0fd3a80 = 29:  1bdcf1399d9 CI: don't use "set -x" in "ci/lib.sh" output
 -:  ----------- > 30:  94abb826627 CI: have osx-gcc use gcc, not clang
29:  2e3c02fa0df ! 31:  4fc67e668da CI: make it easy to use ci/*.sh outside of CI
    @@ Commit message
         environment if we're outside of CI, in case the user has e.g. "jN" or
         other flags to "make" that they'd prefer configured already.
     
    +    Using "ci/lib.sh" as a stand-alone script is much more useful if it
    +    doesn't hardcode NPROC=10, let's provide a poor shellscript
    +    replacement for the online_cpus() we have in thread-utils.c to cover
    +    the most common OS's. It was suggested to use "2>&1" to invoke
    +    "command -v", but per my reading of [2] and my own testing that
    +    doesn't seem to be needed. Perhaps it's only needed for "which(1)"?
    +
    +    1. https://lore.kernel.org/git/214f8670-91d5-f4b6-efa1-76966c3ab1ee@sunshineco.com/
    +    2. https://pubs.opengroup.org/onlinepubs/009604499/utilities/command.html
    +
    +    Helped-by: Eric Sunshine <sunshine@sunshineco.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## ci/lib-ci-type.sh ##
    @@ ci/lib-ci-type.sh
     -	exit 1
      fi
     
    + ## ci/lib-online_cpus.sh (new) ##
    +@@
    ++#!/bin/sh
    ++
    ++# TODO: Ideally we'd compile t/helper/test-online-cpus.c early, but
    ++# that presents a chicken & egg problem. But if we move it to a
    ++# stand-oline command...
    ++online_cpus() {
    ++	NPROC=
    ++
    ++	if command -v nproc >/dev/null
    ++	then
    ++		# GNU coreutils
    ++		NPROC=$(nproc)
    ++	elif command -v sysctl >/dev/null
    ++	then
    ++		# BSD & Mac OS X
    ++		NPROC=$(sysctl -n hw.ncpu)
    ++	elif test -n "$NUMBER_OF_PROCESSORS"
    ++	then
    ++		# Windows
    ++		NPROC="$NUMBER_OF_PROCESSORS"
    ++	else
    ++		NPROC=1
    ++	fi
    ++
    ++	echo $NPROC
    ++}
    +
      ## ci/lib.sh ##
     @@
      #!/bin/sh
    @@ ci/lib.sh: setenv () {
      	fi
      
      	echo "SET: '$key=$val'" >&2
    -@@ ci/lib.sh: CC_PACKAGE=
    +@@ ci/lib.sh: CC=
      
      # How many jobs to run in parallel?
      NPROC=10
     +case "$CI_TYPE" in
     +'')
    -+	if command -v nproc >/dev/null
    -+	then
    -+		NPROC=$(nproc)
    -+	else
    -+		NPROC=1
    -+	fi
    ++	. ${0%/*}/lib-online_cpus.sh
    ++	NPROC=$(online_cpus)
     +
     +	if test -n "$MAKEFLAGS"
     +	then
    @@ ci/lib.sh: github-actions)
     
      ## ci/print-test-failures.sh ##
     @@ ci/print-test-failures.sh: do
    - 		test_name="${TEST_EXIT%.exit}"
    - 		test_name="${test_name##*/}"
    - 		trash_dir="trash directory.$test_name"
    --		case "$CI_TYPE" in
    --		github-actions)
    -+		if test "$CI_TYPE" = "github-actions"
    -+		then
    - 			mkdir -p failed-test-artifacts
    - 			echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
      			cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
      			tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
    - 			continue
    --			;;
    + 			;;
     -		*)
     -			echo "Unhandled CI type: $CI_TYPE" >&2
     -			exit 1
     -			;;
    --		esac
    -+		fi
    - 		trash_tgz_b64="trash.$test_name.base64"
    - 		if [ -d "$trash_dir" ]
    - 		then
    + 		esac
    + 	fi
    + done
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 01/31] CI: run "set -ex" early in ci/lib.sh

[Ævar Arnfjörð Bjarmason]

Change ci/lib.sh to run "set -ex" before it does anything else, before
this we were omitting a few "export" commands in our trace output that
happened before we enabled tracing.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index cbc2f8f1caa..57141d38a85 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,5 +1,11 @@
 # Library of functions shared by all CI scripts
 
+# Set 'exit on error' for all CI scripts to let the caller know that
+# something went wrong.
+# Set tracing executed commands, primarily setting environment variables
+# and installing dependencies.
+set -ex
+
 skip_branch_tip_with_tag () {
 	# Sometimes, a branch is pushed at the same time the tag that points
 	# at the same commit as the tip of the branch is pushed, and building
@@ -85,12 +91,6 @@ export TERM=${TERM:-dumb}
 # Clear MAKEFLAGS that may come from the outside world.
 export MAKEFLAGS=
 
-# Set 'exit on error' for all CI scripts to let the caller know that
-# something went wrong.
-# Set tracing executed commands, primarily setting environment variables
-# and installing dependencies.
-set -ex
-
 if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
 then
 	CI_TYPE=azure-pipelines
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 02/31] CI: make "$jobname" explicit, remove fallback

[Ævar Arnfjörð Bjarmason]

Remove the $jobname fallback last adjusted in b011fabd6e0 (ci/lib.sh:
encapsulate Travis-specific things, 2019-01-27).

This makes all of our CI jobs have an explicit $jobname, and removes
the special-case of the Windows jobs implicitly setting one (that they
don't currently use). In subsequent commits we'll use this new
$jobname explicitly.

This is a follow-up to my 25715419bf4 (CI: don't run "make test" twice
in one job, 2021-11-23), since then all the jobs except the Windows
jobs have had an explicit $jobname.

In the case of the Windows jobs we were using a jobname of windows-gcc
for both, due to picking "gcc" as a default fallback for "$CC" in
ci/lib.sh.

Since we'll need this to run ci/lib.sh it makes sense to move the
check to the top of the file. Subsequent commits will add more such
assertions to it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml |  8 ++++++++
 ci/lib.sh                  | 12 +++++++-----
 2 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index c35200defb9..309b9141249 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -79,6 +79,8 @@ jobs:
 
   windows-build:
     name: win build
+    env:
+      jobname: windows-build
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
     runs-on: windows-latest
@@ -100,6 +102,8 @@ jobs:
         path: artifacts
   windows-test:
     name: win test
+    env:
+      jobname: windows-test
     runs-on: windows-latest
     needs: [windows-build]
     strategy:
@@ -131,6 +135,8 @@ jobs:
         path: ${{env.FAILED_TEST_ARTIFACTS}}
   vs-build:
     name: win+VS build
+    env:
+      jobname: vs-build
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
     env:
@@ -183,6 +189,8 @@ jobs:
         path: artifacts
   vs-test:
     name: win+VS test
+    env:
+      jobname: vs-test
     runs-on: windows-latest
     needs: vs-build
     strategy:
diff --git a/ci/lib.sh b/ci/lib.sh
index 57141d38a85..5fda166ad47 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -6,6 +6,13 @@
 # and installing dependencies.
 set -ex
 
+# Starting assertions
+if test -z "$jobname"
+then
+	echo "must set a CI jobname in the environment" >&2
+	exit 1
+fi
+
 skip_branch_tip_with_tag () {
 	# Sometimes, a branch is pushed at the same time the tag that points
 	# at the same commit as the tip of the branch is pushed, and building
@@ -146,11 +153,6 @@ test -n "${DONT_SKIP_TAGS-}" ||
 skip_branch_tip_with_tag
 skip_good_tree
 
-if test -z "$jobname"
-then
-	jobname="$CI_OS_NAME-$CC"
-fi
-
 export DEVELOPER=1
 export DEFAULT_TEST_TARGET=prove
 export GIT_TEST_CLONE_2GB=true
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 03/31] CI: remove more dead Travis CI support

[Ævar Arnfjörð Bjarmason]

Remove code that I missed in 4a6e4b96026 (CI: remove Travis CI
support, 2021-11-23). This code was only called from or used by the
now-removed .travis.yml, or needed by the Travis CI environment.

For the symlinking in ci/run-build-and-tests.sh: Back when
3c93b829205 (travis-ci: build Git during the 'script' phase,
2018-01-08) and 4b060a4d973 (ci: use a junction on Windows instead of
a symlink, 2019-01-27) adjusted this "ln" command, the Windows build
would use ci/run-build-and-tests.sh.

As seen in 889cacb6897 (ci: configure GitHub Actions for CI/PR,
2020-04-11) the current windows build uses a different entry point
under the GitHub CI, which doesn't use this .prove caching. Namely
"ci/run-test-slice.sh".

We can be certain that it's never used in "ci/run-test-slice.sh"
because to have a ".prove" file we'd need to use "--state=save", which
we only do in the dead Azure codepath in ci/lib.sh. If it were used it
would do the wrong thing, because the different test slices would each
try to clobber the same "t/.prove" file.

If a subsequent run then used the "--state=failed,slow,save" it would
defeat the purpose of "ci/run-test-slice.sh", since all slices would
then run all tests. I.e. behavior of prove's "--state" options is to
select tests to run from the provided "--state" file, in addition to
those specified on the command-line.

For ci/run-docker{,-build}.sh: It was likewise last referenced in the
.travis.yml removed in my 4a6e4b96026. The current "dockerized" run in
".github/workflows/main.yml" calls the same entry points as the main
"regular" job.

For "ci/print-test-failures.sh" the creation of the tarball added in
aea8879a6ac (travis-ci: include the trash directories of failed tests
in the trace log, 2018-08-01) has been dead since my 4a6e4b96026,
which removed the fall-through case from the "case/esac" statement.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                     | 10 ++----
 ci/print-test-failures.sh     | 39 ---------------------
 ci/run-build-and-tests.sh     |  5 ---
 ci/run-docker-build.sh        | 66 -----------------------------------
 ci/run-docker.sh              | 47 -------------------------
 ci/run-test-slice.sh          |  5 ---
 ci/util/extract-trash-dirs.sh | 50 --------------------------
 7 files changed, 2 insertions(+), 220 deletions(-)
 delete mode 100755 ci/run-docker-build.sh
 delete mode 100755 ci/run-docker.sh
 delete mode 100755 ci/util/extract-trash-dirs.sh

diff --git a/ci/lib.sh b/ci/lib.sh
index 5fda166ad47..510b20726ab 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -106,9 +106,6 @@ then
 	CI_COMMIT="$BUILD_SOURCEVERSION"
 	CI_JOB_ID="$BUILD_BUILDID"
 	CI_JOB_NUMBER="$BUILD_BUILDNUMBER"
-	CI_OS_NAME="$(echo "$AGENT_OS" | tr A-Z a-z)"
-	test darwin != "$CI_OS_NAME" || CI_OS_NAME=osx
-	CI_REPO_SLUG="$(expr "$BUILD_REPOSITORY_URI" : '.*/\([^/]*/[^/]*\)$')"
 	CC="${CC:-gcc}"
 
 	# use a subdirectory of the cache dir (because the file share is shared
@@ -118,16 +115,13 @@ then
 	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
-	test windows_nt != "$CI_OS_NAME" ||
+	test Windows_NT != "$AGENT_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 elif test true = "$GITHUB_ACTIONS"
 then
 	CI_TYPE=github-actions
 	CI_BRANCH="$GITHUB_REF"
 	CI_COMMIT="$GITHUB_SHA"
-	CI_OS_NAME="$(echo "$RUNNER_OS" | tr A-Z a-z)"
-	test macos != "$CI_OS_NAME" || CI_OS_NAME=osx
-	CI_REPO_SLUG="$GITHUB_REPOSITORY"
 	CI_JOB_ID="$GITHUB_RUN_ID"
 	CC="${CC:-gcc}"
 	DONT_SKIP_TAGS=t
@@ -137,7 +131,7 @@ then
 	export GIT_PROVE_OPTS="--timer --jobs 10"
 	export GIT_TEST_OPTS="--verbose-log -x"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
-	test windows != "$CI_OS_NAME" ||
+	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 else
 	echo "Could not identify CI type" >&2
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 57277eefcd0..740ebb7c529 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -16,15 +16,6 @@ then
 	exit
 fi
 
-case "$jobname" in
-osx-clang|osx-gcc)
-	# base64 in OSX doesn't wrap its output at 76 columns by
-	# default, but prints a single, very long line.
-	base64_opts="-b 76"
-	;;
-esac
-
-combined_trash_size=0
 for TEST_EXIT in test-results/*.exit
 do
 	if [ "$(cat "$TEST_EXIT")" != "0" ]
@@ -42,47 +33,17 @@ do
 		azure-pipelines)
 			mkdir -p failed-test-artifacts
 			mv "$trash_dir" failed-test-artifacts
-			continue
 			;;
 		github-actions)
 			mkdir -p failed-test-artifacts
 			echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
 			cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
 			tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
-			continue
 			;;
 		*)
 			echo "Unhandled CI type: $CI_TYPE" >&2
 			exit 1
 			;;
 		esac
-		trash_tgz_b64="trash.$test_name.base64"
-		if [ -d "$trash_dir" ]
-		then
-			tar czp "$trash_dir" |base64 $base64_opts >"$trash_tgz_b64"
-
-			trash_size=$(wc -c <"$trash_tgz_b64")
-			if [ $trash_size -gt 1048576 ]
-			then
-				# larger than 1MB
-				echo "$(tput setaf 1)Didn't include the trash directory of '$test_name' in the trace log, it's too big$(tput sgr0)"
-				continue
-			fi
-
-			new_combined_trash_size=$(($combined_trash_size + $trash_size))
-			if [ $new_combined_trash_size -gt 1048576 ]
-			then
-				echo "$(tput setaf 1)Didn't include the trash directory of '$test_name' in the trace log, there is plenty of trash in there already.$(tput sgr0)"
-				continue
-			fi
-			combined_trash_size=$new_combined_trash_size
-
-			# DO NOT modify these two 'echo'-ed strings below
-			# without updating 'ci/util/extract-trash-dirs.sh'
-			# as well.
-			echo "$(tput setaf 1)Start of trash directory of '$test_name':$(tput sgr0)"
-			cat "$trash_tgz_b64"
-			echo "$(tput setaf 1)End of trash directory of '$test_name'$(tput sgr0)"
-		fi
 	fi
 done
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 280dda7d285..0a1ec8c2bae 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -5,11 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-case "$CI_OS_NAME" in
-windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
-*) ln -s "$cache_dir/.prove" t/.prove;;
-esac
-
 export MAKE_TARGETS="all test"
 
 case "$jobname" in
diff --git a/ci/run-docker-build.sh b/ci/run-docker-build.sh
deleted file mode 100755
index 6cd832efb9c..00000000000
--- a/ci/run-docker-build.sh
+++ /dev/null
@@ -1,66 +0,0 @@
-#!/bin/sh
-#
-# Build and test Git inside container
-#
-# Usage:
-#   run-docker-build.sh <host-user-id>
-#
-
-set -ex
-
-if test $# -ne 1 || test -z "$1"
-then
-	echo >&2 "usage: run-docker-build.sh <host-user-id>"
-	exit 1
-fi
-
-case "$jobname" in
-linux32)
-	switch_cmd="linux32 --32bit i386"
-	;;
-linux-musl)
-	switch_cmd=
-	useradd () { adduser -D "$@"; }
-	;;
-*)
-	exit 1
-	;;
-esac
-
-"${0%/*}/install-docker-dependencies.sh"
-
-# If this script runs inside a docker container, then all commands are
-# usually executed as root. Consequently, the host user might not be
-# able to access the test output files.
-# If a non 0 host user id is given, then create a user "ci" with that
-# user id to make everything accessible to the host user.
-HOST_UID=$1
-if test $HOST_UID -eq 0
-then
-	# Just in case someone does want to run the test suite as root.
-	CI_USER=root
-else
-	CI_USER=ci
-	if test "$(id -u $CI_USER 2>/dev/null)" = $HOST_UID
-	then
-		echo "user '$CI_USER' already exists with the requested ID $HOST_UID"
-	else
-		useradd -u $HOST_UID $CI_USER
-	fi
-fi
-
-# Build and test
-command $switch_cmd su -m -l $CI_USER -c "
-	set -ex
-	export DEVELOPER='$DEVELOPER'
-	export DEFAULT_TEST_TARGET='$DEFAULT_TEST_TARGET'
-	export GIT_PROVE_OPTS='$GIT_PROVE_OPTS'
-	export GIT_TEST_OPTS='$GIT_TEST_OPTS'
-	export GIT_TEST_CLONE_2GB='$GIT_TEST_CLONE_2GB'
-	export MAKEFLAGS='$MAKEFLAGS'
-	export cache_dir='$cache_dir'
-	cd /usr/src/git
-	test -n '$cache_dir' && ln -s '$cache_dir/.prove' t/.prove
-	make
-	make test
-"
diff --git a/ci/run-docker.sh b/ci/run-docker.sh
deleted file mode 100755
index af89d1624a4..00000000000
--- a/ci/run-docker.sh
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/bin/sh
-#
-# Download and run Docker image to build and test Git
-#
-
-. ${0%/*}/lib.sh
-
-case "$jobname" in
-linux32)
-	CI_CONTAINER="daald/ubuntu32:xenial"
-	;;
-linux-musl)
-	CI_CONTAINER=alpine
-	;;
-*)
-	exit 1
-	;;
-esac
-
-docker pull "$CI_CONTAINER"
-
-# Use the following command to debug the docker build locally:
-# <host-user-id> must be 0 if podman is used as drop-in replacement for docker
-# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/sh "$CI_CONTAINER"
-# root@container:/# export jobname=<jobname>
-# root@container:/# /usr/src/git/ci/run-docker-build.sh <host-user-id>
-
-container_cache_dir=/tmp/container-cache
-
-docker run \
-	--interactive \
-	--env DEVELOPER \
-	--env DEFAULT_TEST_TARGET \
-	--env GIT_PROVE_OPTS \
-	--env GIT_TEST_OPTS \
-	--env GIT_TEST_CLONE_2GB \
-	--env MAKEFLAGS \
-	--env jobname \
-	--env cache_dir="$container_cache_dir" \
-	--volume "${PWD}:/usr/src/git" \
-	--volume "$cache_dir:$container_cache_dir" \
-	"$CI_CONTAINER" \
-	/usr/src/git/ci/run-docker-build.sh $(id -u $USER)
-
-check_unignored_build_artifacts
-
-save_good_tree
diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index f8c2c3106a2..b9a682b4bcd 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -5,11 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-case "$CI_OS_NAME" in
-windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
-*) ln -s "$cache_dir/.prove" t/.prove;;
-esac
-
 make --quiet -C t T="$(cd t &&
 	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
 	tr '\n' ' ')"
diff --git a/ci/util/extract-trash-dirs.sh b/ci/util/extract-trash-dirs.sh
deleted file mode 100755
index 8e67bec21a2..00000000000
--- a/ci/util/extract-trash-dirs.sh
+++ /dev/null
@@ -1,50 +0,0 @@
-#!/bin/sh
-
-error () {
-	echo >&2 "error: $@"
-	exit 1
-}
-
-find_embedded_trash () {
-	while read -r line
-	do
-		case "$line" in
-		*Start\ of\ trash\ directory\ of\ \'t[0-9][0-9][0-9][0-9]-*\':*)
-			test_name="${line#*\'}"
-			test_name="${test_name%\'*}"
-
-			return 0
-		esac
-	done
-
-	return 1
-}
-
-extract_embedded_trash () {
-	while read -r line
-	do
-		case "$line" in
-		*End\ of\ trash\ directory\ of\ \'$test_name\'*)
-			return
-			;;
-		*)
-			printf '%s\n' "$line"
-			;;
-		esac
-	done
-
-	error "unexpected end of input"
-}
-
-# Raw logs from Linux build jobs have CRLF line endings, while OSX
-# build jobs mostly have CRCRLF, except an odd line every now and
-# then that has CRCRCRLF.  'base64 -d' from 'coreutils' doesn't like
-# CRs and complains about "invalid input", so remove all CRs at the
-# end of lines.
-sed -e 's/\r*$//' | \
-while find_embedded_trash
-do
-	echo "Extracting trash directory of '$test_name'"
-
-	extract_embedded_trash |base64 -d |tar xzp
-done
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 04/31] CI: remove dead "tree skipping" code

[Ævar Arnfjörð Bjarmason]

Remove the code related to skipping CI runs if the tree was
successfully tested before. With the GitHub CI this is done with the
"ci-config" step, see e76eec35540 (ci: allow per-branch config for
GitHub Actions, 2020-05-07) and 7d78d5fc1a9 (ci: skip GitHub workflow
runs for already-tested commits/trees, 2020-10-08).

This code hasn't been used since 4a6e4b96026 (CI: remove Travis CI
support, 2021-11-23), and before that for the retired Azure pipeline
support removed in 6081d3898fe (ci: retire the Azure Pipelines
definition, 2020-04-11).

This change is needed because this and subsequent commits are turning
"ci/lib.sh" into a dumber library that'll only be tasked with setting
variables for CI jobs, or for specific steps within those jobs. Now we
don't need to worry about it potentially skipping the run on its own.

This change also removes a subtle potential logic error introduced in
0e7696c64db (ci: disallow directional formatting, 2021-11-04). The
"ci/check-directional-formatting.bash" script would have been made to
run after the "save_good_tree" in invoked in
"ci/run-static-analysis.sh". So if e.g. the "make coccicheck" failed
we'd still mark the tree as good.

I.e. the addition of "ci/check-directional-formatting.bash" didn't
take into account that the various users of the "save_good_tree"
function in ci/*.sh made a hard assumption that they're the only
ci/*.sh script that's being run (and they use "set -e").

This is entirely academic since we weren't actually running this code,
but is something to be careful of if anyone ever needs to resurrect
parts of this.

Finally, the cache_dir="$HOME/none" added in a3f2eec862b (ci/lib:
allow running in GitHub Actions, 2020-04-08) for the GitHub CI can be
removed. It appears to have been added to appease the subsequent
'mkdir -p "$cache_dir"', which wasn't needed in GitHub CI.

1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                 | 91 ---------------------------------------
 ci/run-build-and-tests.sh |  2 -
 ci/run-static-analysis.sh |  2 -
 ci/test-documentation.sh  |  2 -
 4 files changed, 97 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 510b20726ab..d12eaf338f9 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -13,75 +13,6 @@ then
 	exit 1
 fi
 
-skip_branch_tip_with_tag () {
-	# Sometimes, a branch is pushed at the same time the tag that points
-	# at the same commit as the tip of the branch is pushed, and building
-	# both at the same time is a waste.
-	#
-	# When the build is triggered by a push to a tag, $CI_BRANCH will
-	# have that tagname, e.g. v2.14.0.  Let's see if $CI_BRANCH is
-	# exactly at a tag, and if so, if it is different from $CI_BRANCH.
-	# That way, we can tell if we are building the tip of a branch that
-	# is tagged and we can skip the build because we won't be skipping a
-	# build of a tag.
-
-	if TAG=$(git describe --exact-match "$CI_BRANCH" 2>/dev/null) &&
-		test "$TAG" != "$CI_BRANCH"
-	then
-		echo "$(tput setaf 2)Tip of $CI_BRANCH is exactly at $TAG$(tput sgr0)"
-		exit 0
-	fi
-}
-
-# Save some info about the current commit's tree, so we can skip the build
-# job if we encounter the same tree again and can provide a useful info
-# message.
-save_good_tree () {
-	echo "$(git rev-parse $CI_COMMIT^{tree}) $CI_COMMIT $CI_JOB_NUMBER $CI_JOB_ID" >>"$good_trees_file"
-	# limit the file size
-	tail -1000 "$good_trees_file" >"$good_trees_file".tmp
-	mv "$good_trees_file".tmp "$good_trees_file"
-}
-
-# Skip the build job if the same tree has already been built and tested
-# successfully before (e.g. because the branch got rebased, changing only
-# the commit messages).
-skip_good_tree () {
-	if test true = "$GITHUB_ACTIONS"
-	then
-		return
-	fi
-
-	if ! good_tree_info="$(grep "^$(git rev-parse $CI_COMMIT^{tree}) " "$good_trees_file")"
-	then
-		# Haven't seen this tree yet, or no cached good trees file yet.
-		# Continue the build job.
-		return
-	fi
-
-	echo "$good_tree_info" | {
-		read tree prev_good_commit prev_good_job_number prev_good_job_id
-
-		if test "$CI_JOB_ID" = "$prev_good_job_id"
-		then
-			cat <<-EOF
-			$(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0)
-			This commit has already been built and tested successfully by this build job.
-			To force a re-build delete the branch's cache and then hit 'Restart job'.
-			EOF
-		else
-			cat <<-EOF
-			$(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0)
-			This commit's tree has already been built and tested successfully in build job $prev_good_job_number for commit $prev_good_commit.
-			The log of that build job is available at $SYSTEM_TASKDEFINITIONSURI$SYSTEM_TEAMPROJECT/_build/results?buildId=$prev_good_job_id
-			To force a re-build delete the branch's cache and then hit 'Restart job'.
-			EOF
-		fi
-	}
-
-	exit 0
-}
-
 check_unignored_build_artifacts ()
 {
 	! git ls-files --other --exclude-standard --error-unmatch \
@@ -102,16 +33,8 @@ if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
 then
 	CI_TYPE=azure-pipelines
 	# We are running in Azure Pipelines
-	CI_BRANCH="$BUILD_SOURCEBRANCH"
-	CI_COMMIT="$BUILD_SOURCEVERSION"
-	CI_JOB_ID="$BUILD_BUILDID"
-	CI_JOB_NUMBER="$BUILD_BUILDNUMBER"
 	CC="${CC:-gcc}"
 
-	# use a subdirectory of the cache dir (because the file share is shared
-	# among *all* phases)
-	cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME"
-
 	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
@@ -120,13 +43,7 @@ then
 elif test true = "$GITHUB_ACTIONS"
 then
 	CI_TYPE=github-actions
-	CI_BRANCH="$GITHUB_REF"
-	CI_COMMIT="$GITHUB_SHA"
-	CI_JOB_ID="$GITHUB_RUN_ID"
 	CC="${CC:-gcc}"
-	DONT_SKIP_TAGS=t
-
-	cache_dir="$HOME/none"
 
 	export GIT_PROVE_OPTS="--timer --jobs 10"
 	export GIT_TEST_OPTS="--verbose-log -x"
@@ -139,14 +56,6 @@ else
 	exit 1
 fi
 
-good_trees_file="$cache_dir/good-trees"
-
-mkdir -p "$cache_dir"
-
-test -n "${DONT_SKIP_TAGS-}" ||
-skip_branch_tip_with_tag
-skip_good_tree
-
 export DEVELOPER=1
 export DEFAULT_TEST_TARGET=prove
 export GIT_TEST_CLONE_2GB=true
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 0a1ec8c2bae..e5a21985b44 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -45,5 +45,3 @@ esac
 # start running tests.
 make $MAKE_TARGETS
 check_unignored_build_artifacts
-
-save_good_tree
diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh
index 65bcebda41a..5a87b1cac96 100755
--- a/ci/run-static-analysis.sh
+++ b/ci/run-static-analysis.sh
@@ -28,5 +28,3 @@ fi
 
 make hdr-check ||
 exit 1
-
-save_good_tree
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index de41888430a..9e0652c30dd 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -41,5 +41,3 @@ grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
 
 rm -f stdout.log stderr.log stderr.raw
 check_unignored_build_artifacts
-
-save_good_tree
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 05/31] CI: remove unused Azure ci/* code

[Ævar Arnfjörð Bjarmason]

Remove Azure-specific code that's been unused since 6081d3898fe (ci:
retire the Azure Pipelines definition, 2020-04-11). As noted in a
larger removal of all of the Azure-supporting code in [1] (although
that missed some of this) there is more of it in-tree, but let's focus
on only the ci/* code for now.

This is needed because in subsequent commits this unused code would
either need to be changed to accommodate changes in ci/*, or be
removed.

As we'll see in those subsequent commits the end-state we're heading
towards will actually make it easier to add new CI types in the
future, even though the only one we're left with now is the GitHub
CI. I.e. the "ci/*" framework will be made to do less, not more. We'll
be offloading more of what it does to our generic build and test
system.

While I'm at it (since the line needs to be touched anyway) change an
odd 'if test true == "$GITHUB_ACTIONS"' to the more usual style used
in other places of 'if test "$GITHUB_ACTIONS" = "true"'.

1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/install-dependencies.sh |  3 ---
 ci/lib.sh                  | 13 +------------
 ci/mount-fileshare.sh      | 25 -------------------------
 ci/print-test-failures.sh  |  4 ----
 4 files changed, 1 insertion(+), 44 deletions(-)
 delete mode 100755 ci/mount-fileshare.sh

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index dbcebad2fb2..e7ea8799411 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -34,8 +34,6 @@ macos-latest)
 	export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
 	# Uncomment this if you want to run perf tests:
 	# brew install gnu-time
-	test -z "$BREW_INSTALL_PACKAGES" ||
-	brew install $BREW_INSTALL_PACKAGES
 	brew link --force gettext
 	brew install --cask --no-quarantine perforce || {
 		# Update the definitions and try again
@@ -69,7 +67,6 @@ Documentation)
 	sudo apt-get -q update
 	sudo apt-get -q -y install asciidoc xmlto docbook-xsl-ns make
 
-	test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
 	sudo gem install --version 1.5.8 asciidoctor
 	;;
 linux-gcc-default)
diff --git a/ci/lib.sh b/ci/lib.sh
index d12eaf338f9..c4b829c1eed 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -29,18 +29,7 @@ export TERM=${TERM:-dumb}
 # Clear MAKEFLAGS that may come from the outside world.
 export MAKEFLAGS=
 
-if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
-then
-	CI_TYPE=azure-pipelines
-	# We are running in Azure Pipelines
-	CC="${CC:-gcc}"
-
-	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
-	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
-	MAKEFLAGS="$MAKEFLAGS --jobs=10"
-	test Windows_NT != "$AGENT_OS" ||
-	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
-elif test true = "$GITHUB_ACTIONS"
+if test "$GITHUB_ACTIONS" = "true"
 then
 	CI_TYPE=github-actions
 	CC="${CC:-gcc}"
diff --git a/ci/mount-fileshare.sh b/ci/mount-fileshare.sh
deleted file mode 100755
index 26b58a80960..00000000000
--- a/ci/mount-fileshare.sh
+++ /dev/null
@@ -1,25 +0,0 @@
-#!/bin/sh
-
-die () {
-	echo "$*" >&2
-	exit 1
-}
-
-test $# = 4 ||
-die "Usage: $0 <share> <username> <password> <mountpoint>"
-
-mkdir -p "$4" || die "Could not create $4"
-
-case "$(uname -s)" in
-Linux)
-	sudo mount -t cifs -o vers=3.0,username="$2",password="$3",dir_mode=0777,file_mode=0777,serverino "$1" "$4"
-	;;
-Darwin)
-	pass="$(echo "$3" | sed -e 's/\//%2F/g' -e 's/+/%2B/g')" &&
-	mount -t smbfs,soft "smb://$2:$pass@${1#//}" "$4"
-	;;
-*)
-	die "No support for $(uname -s)"
-	;;
-esac ||
-die "Could not mount $4"
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 740ebb7c529..47e2ea1ad10 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -30,10 +30,6 @@ do
 		test_name="${test_name##*/}"
 		trash_dir="trash directory.$test_name"
 		case "$CI_TYPE" in
-		azure-pipelines)
-			mkdir -p failed-test-artifacts
-			mv "$trash_dir" failed-test-artifacts
-			;;
 		github-actions)
 			mkdir -p failed-test-artifacts
 			echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 06/31] CI/lib.sh: stop adding leading whitespace to $MAKEFLAGS

[Ævar Arnfjörð Bjarmason]

In a8c51f77d16 (ci: clear and mark MAKEFLAGS exported just once,
2019-02-07) the assignment of MAKEFLAGS was done up-front to
future-proof this code for the bug fixed in 406f93ae488 (ci: make sure
we build Git parallel, 2019-02-07).

But since 4a6e4b96026 (CI: remove Travis CI support, 2021-11-23) the
--jobs=10 has been an invariant here, so let's assign --jobs=$NPROC to
the MAKEFLAGS right away, this gets rid of the leading whitespace from
the $MAKEFLAGS variable, as it would previously be empty on the first
assignment.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index c4b829c1eed..21c009757f4 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -26,17 +26,19 @@ check_unignored_build_artifacts ()
 # GitHub Action doesn't set TERM, which is required by tput
 export TERM=${TERM:-dumb}
 
+# How many jobs to run in parallel?
+NPROC=10
+
 # Clear MAKEFLAGS that may come from the outside world.
-export MAKEFLAGS=
+export MAKEFLAGS=--jobs=$NPROC
 
 if test "$GITHUB_ACTIONS" = "true"
 then
 	CI_TYPE=github-actions
 	CC="${CC:-gcc}"
 
-	export GIT_PROVE_OPTS="--timer --jobs 10"
+	export GIT_PROVE_OPTS="--timer --jobs $NPROC"
 	export GIT_TEST_OPTS="--verbose-log -x"
-	MAKEFLAGS="$MAKEFLAGS --jobs=10"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 else
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 07/31] CI: don't have "git grep" invoke a pager in tree content check

[Ævar Arnfjörð Bjarmason]

Fix the omission of not adding "-P" (a.k.a. "--no-pager") to the "git"
invocation that spaws "git grep", added in 0e7696c64db (ci: disallow
directional formatting, 2021-11-04).

This didn't matter for the GitHub CI, but does when running the check
locally. A subsequent commit will start invoking it via a new check
from the top-level Makefile, and without this we'd visibly spawn a
pager when doing so.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/check-directional-formatting.bash | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ci/check-directional-formatting.bash b/ci/check-directional-formatting.bash
index e6211b141aa..de8e6e486f0 100755
--- a/ci/check-directional-formatting.bash
+++ b/ci/check-directional-formatting.bash
@@ -23,5 +23,5 @@
 # U+2066..U+2069: LRI, RLI, FSI and PDI
 regex='(\u202a|\u202b|\u202c|\u202d|\u202e|\u2066|\u2067|\u2068|\u2069)'
 
-! LC_CTYPE=C git grep -El "$(LC_CTYPE=C.UTF-8 printf "$regex")" \
+! LC_CTYPE=C git -P grep -El "$(LC_CTYPE=C.UTF-8 printf "$regex")" \
 	-- ':(exclude,attr:binary)' ':(exclude)*.po'
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 08/31] CI: have "static-analysis" run a "make ci-static-analysis" target

[Ævar Arnfjörð Bjarmason]

Make it easier to run the CI logic locally by creating
"ci-static-analysis" target.

Now the "check-coccicheck" logic added in 0860a7641ba (travis-ci: fail
if Coccinelle static analysis found something to transform,
2018-07-23) can be easily tested outside of CI, and likewise the
sanity check added in 0e7696c64db (ci: disallow directional
formatting, 2021-11-04).

These targets could be improved, the "hdr-check" target will re-do all
of its work every time it's run, and could with a change similar to my
c234e8a0ecf (Makefile: make the "sparse" target non-.PHONY,
2021-09-23). The same goes for the new
"ci-check-directional-formatting" target.

But without those improvements being able to easily run a 1=1
equivalent of the checks we do in CI during a local build is already
an improvement. Subsequent commits will expand on this ability.

This change will also make the CI check faster, since we can take
advantage of parallelism. The "make coccicheck" takes a very long
time, and at the tail end we might only have one job pegging 100% of
one CPU core for a long time. Now any extra cores will be free to run
the rest of the targets under "ci-static-analysis".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml |  3 +--
 Makefile                   | 28 ++++++++++++++++++++++++++++
 ci/run-static-analysis.sh  | 30 ------------------------------
 shared.mak                 |  1 +
 4 files changed, 30 insertions(+), 32 deletions(-)
 delete mode 100755 ci/run-static-analysis.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 309b9141249..ad752010102 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -317,8 +317,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/run-static-analysis.sh
-    - run: ci/check-directional-formatting.bash
+    - run: make ci-static-analysis
   sparse:
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
diff --git a/Makefile b/Makefile
index f8bccfab5e9..8717ea7e745 100644
--- a/Makefile
+++ b/Makefile
@@ -3031,6 +3031,20 @@ coccicheck: $(addsuffix .patch,$(filter-out %.pending.cocci,$(wildcard contrib/c
 # See contrib/coccinelle/README
 coccicheck-pending: $(addsuffix .patch,$(wildcard contrib/coccinelle/*.pending.cocci))
 
+.PHONY: check-coccicheck
+check-coccicheck: coccicheck
+	$(QUIET_CHECK)for cocci_patch in contrib/coccinelle/*.patch; do \
+		if test -s "$$cocci_patch"; then \
+			echo "Coccinelle suggests the following changes in '$$cocci_patch':"; \
+			cat "$$cocci_patch"; \
+			fail=UnfortunatelyYes; \
+		fi \
+	done; \
+	if test -n "$$fail"; then \
+		echo "error: Coccinelle suggested some changes"; \
+		exit 1; \
+	fi
+
 .PHONY: coccicheck coccicheck-pending
 
 ### Installation rules
@@ -3466,3 +3480,17 @@ $(FUZZ_PROGRAMS): all
 		$(XDIFF_OBJS) $(EXTLIBS) git.o $@.o $(LIB_FUZZING_ENGINE) -o $@
 
 fuzz-all: $(FUZZ_PROGRAMS)
+
+### CI "check" targets
+#
+# These targets are run from the CI, see .github/workflows/main.yml,
+# but can also be run manually to run the same assertions locally.
+
+.PHONY: ci-check-directional-formatting
+ci-check-directional-formatting:
+	$(QUIET_CHECK)ci/check-directional-formatting.bash
+
+.PHONY: ci-static-analysis
+ci-static-analysis: ci-check-directional-formatting
+ci-static-analysis: check-coccicheck
+ci-static-analysis: hdr-check
diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh
deleted file mode 100755
index 5a87b1cac96..00000000000
--- a/ci/run-static-analysis.sh
+++ /dev/null
@@ -1,30 +0,0 @@
-#!/bin/sh
-#
-# Perform various static code analysis checks
-#
-
-. ${0%/*}/lib.sh
-
-make coccicheck
-
-set +x
-
-fail=
-for cocci_patch in contrib/coccinelle/*.patch
-do
-	if test -s "$cocci_patch"
-	then
-		echo "$(tput setaf 1)Coccinelle suggests the following changes in '$cocci_patch':$(tput sgr0)"
-		cat "$cocci_patch"
-		fail=UnfortunatelyYes
-	fi
-done
-
-if test -n "$fail"
-then
-	echo "$(tput setaf 1)error: Coccinelle suggested some changes$(tput sgr0)"
-	exit 1
-fi
-
-make hdr-check ||
-exit 1
diff --git a/shared.mak b/shared.mak
index 50d4596f0d6..83e0aa16e7d 100644
--- a/shared.mak
+++ b/shared.mak
@@ -68,6 +68,7 @@ ifndef V
 	QUIET_HDR      = @echo '   ' HDR $(<:hcc=h);
 	QUIET_RC       = @echo '   ' RC $@;
 	QUIET_SPATCH   = @echo '   ' SPATCH $<;
+	QUIET_CHECK    = @echo '   ' CHECK $@;
 
 ## Used in "Documentation/Makefile"
 	QUIET_ASCIIDOC	= @echo '   ' ASCIIDOC $@;
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 09/31] CI: have "static-analysis" run "check-builtins", not "documentation"

[Ævar Arnfjörð Bjarmason]

Move the "make check-builtins" check from the "documentation" job to
the "static-analysis" job.

The "check-builtins" target added in c74390e4a1d (cherry is built-in,
do not ship git-cherry.sh, 2006-11-05) is unrelated to the
documentation, so it's odd that b98712b9aa9 (travis-ci: build
documentation, 2016-05-04) added it to the "documentation" job.

Let's just move it to the "ci-static-analysis" target, and while we're
at it improve the output with $(QUIET_CHECK).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile                 | 3 ++-
 ci/test-documentation.sh | 1 -
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 8717ea7e745..2e1ae6c6067 100644
--- a/Makefile
+++ b/Makefile
@@ -3402,7 +3402,7 @@ check-docs::
 ### Make sure built-ins do not have dups and listed in git.c
 #
 check-builtins::
-	./check-builtins.sh
+	$(QUIET_CHECK)./check-builtins.sh
 
 ### Test suite coverage testing
 #
@@ -3492,5 +3492,6 @@ ci-check-directional-formatting:
 
 .PHONY: ci-static-analysis
 ci-static-analysis: ci-check-directional-formatting
+ci-static-analysis: check-builtins
 ci-static-analysis: check-coccicheck
 ci-static-analysis: hdr-check
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 9e0652c30dd..41e2b126311 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -14,7 +14,6 @@ filter_log () {
 	    "$1"
 }
 
-make check-builtins
 make check-docs
 
 # Build docs with AsciiDoc
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 10/31] CI: move p4 and git-lfs variables to ci/install-dependencies.sh

[Ævar Arnfjörð Bjarmason]

Move the declaration of variables that are only used by the
"ubuntu-latest" block in "ci/install-dependencies.sh" there from
"ci/lib.sh".

This makes the code easier to follow, and changes "ci/lib.sh" to a
library that only exports CI variables for general use, we didn't need
to export these $P4_PATH and $GIT_LFS_PATH variables.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/install-dependencies.sh | 17 +++++++++++++++--
 ci/lib.sh                  | 12 ------------
 2 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index e7ea8799411..34b4400f38b 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -5,14 +5,27 @@
 
 . ${0%/*}/lib.sh
 
-P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
-LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
 UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
  tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
  libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl"
 
 case "$runs_on_pool" in
 ubuntu-latest)
+	# The Linux build installs the defined dependency versions below.
+	# The OS X build installs much more recent versions, whichever
+	# were recorded in the Homebrew database upon creating the OS X
+	# image.
+	# Keep that in mind when you encounter a broken OS X build!
+	LINUX_P4_VERSION="16.2"
+	LINUX_GIT_LFS_VERSION="1.5.2"
+
+	P4_PATH="$HOME/custom/p4"
+	GIT_LFS_PATH="$HOME/custom/git-lfs"
+	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
+
+	P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
+	LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
+
 	sudo apt-get -q update
 	sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \
 		$UBUNTU_COMMON_PKGS $CC_PACKAGE
diff --git a/ci/lib.sh b/ci/lib.sh
index 21c009757f4..5736a4d2e04 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -67,18 +67,6 @@ ubuntu-latest)
 	fi
 
 	export GIT_TEST_HTTPD=true
-
-	# The Linux build installs the defined dependency versions below.
-	# The OS X build installs much more recent versions, whichever
-	# were recorded in the Homebrew database upon creating the OS X
-	# image.
-	# Keep that in mind when you encounter a broken OS X build!
-	export LINUX_P4_VERSION="16.2"
-	export LINUX_GIT_LFS_VERSION="1.5.2"
-
-	P4_PATH="$HOME/custom/p4"
-	GIT_LFS_PATH="$HOME/custom/git-lfs"
-	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
 	;;
 macos-latest)
 	if [ "$jobname" = osx-gcc ]
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 11/31] CI: consistently use "export" in ci/lib.sh

[Ævar Arnfjörð Bjarmason]

Change the "ci/lib.sh" script co consistently use "export", for
e.g. MAKEFLAGS we were exporting it, and then assigning to it, let's
do it the other way around.

Right now this doesn't matter, since we in
e.g. "ci/install-dependencies.sh" source this file, and don't use
something like "env(1)" to retrieve these variables. But in a
subsequent commit we'll "export" these variables through a wrapper (to
additionally write them to a GitHub CI-specific $GITHUB_ENV
file). This change makes that subsequent change easier to read, as it
won't need to do any control flow refactoring.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 5736a4d2e04..5783eed1b20 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -30,7 +30,7 @@ export TERM=${TERM:-dumb}
 NPROC=10
 
 # Clear MAKEFLAGS that may come from the outside world.
-export MAKEFLAGS=--jobs=$NPROC
+MAKEFLAGS=--jobs=$NPROC
 
 if test "$GITHUB_ACTIONS" = "true"
 then
@@ -38,9 +38,10 @@ then
 	CC="${CC:-gcc}"
 
 	export GIT_PROVE_OPTS="--timer --jobs $NPROC"
-	export GIT_TEST_OPTS="--verbose-log -x"
+	GIT_TEST_OPTS="--verbose-log -x"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
+	export GIT_TEST_OPTS
 else
 	echo "Could not identify CI type" >&2
 	env >&2
@@ -94,4 +95,4 @@ linux-leaks)
 	;;
 esac
 
-MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
+export MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 12/31] CI: export variables via a wrapper

[Ævar Arnfjörð Bjarmason]

Add and use a "setenv" wrapper to export the variables set in
ci/lib.sh.

For now this merely adds debugging information about what variables
are set, but in a subsequent commit we'll expand this to set variables
in $GITHUB_ENV instead of the environment, and thus carry variable
settings between steps in a job.

We'll also eventually be interested in what type of variable we're
setting, and when it needs to be used, because we'll prominently
expose what variables we set in what CI "step", so having only the
ones we actually use will improve readability.

To that end call the "setenv" function with optional "--build",
"--test" and "--all" parameters to indicate whether they're needed for
building, testing or both.

We currently ignore the "varname" parameter, but will make use of it in a
subsequent commit. Adding it here reduces the needless churn, as we
won't need to change all these lines again for that subsequent change.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                 | 51 ++++++++++++++++++++++++++++++---------
 ci/run-build-and-tests.sh | 34 +++++++++++++-------------
 2 files changed, 57 insertions(+), 28 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 5783eed1b20..3727e6ec2f9 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -13,6 +13,35 @@ then
 	exit 1
 fi
 
+# Helper functions
+setenv () {
+	while test $# != 0
+	do
+		case "$1" in
+		--build)
+			;;
+		--test)
+			;;
+		--all)
+			;;
+		-*)
+			echo "BUG: bad setenv() option '$1'" >&2
+			exit 1
+			;;
+		*)
+			break
+			;;
+		esac
+		shift
+	done
+
+	key=$1
+	val=$2
+	shift 2
+
+	eval "export $key=\"$val\""
+}
+
 check_unignored_build_artifacts ()
 {
 	! git ls-files --other --exclude-standard --error-unmatch \
@@ -24,7 +53,7 @@ check_unignored_build_artifacts ()
 }
 
 # GitHub Action doesn't set TERM, which is required by tput
-export TERM=${TERM:-dumb}
+setenv TERM ${TERM:-dumb}
 
 # How many jobs to run in parallel?
 NPROC=10
@@ -37,21 +66,21 @@ then
 	CI_TYPE=github-actions
 	CC="${CC:-gcc}"
 
-	export GIT_PROVE_OPTS="--timer --jobs $NPROC"
+	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
 	GIT_TEST_OPTS="--verbose-log -x"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
-	export GIT_TEST_OPTS
+	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
 else
 	echo "Could not identify CI type" >&2
 	env >&2
 	exit 1
 fi
 
-export DEVELOPER=1
-export DEFAULT_TEST_TARGET=prove
-export GIT_TEST_CLONE_2GB=true
-export SKIP_DASHED_BUILT_INS=YesPlease
+setenv --build DEVELOPER 1
+setenv --test DEFAULT_TEST_TARGET prove
+setenv --test GIT_TEST_CLONE_2GB true
+setenv --build SKIP_DASHED_BUILT_INS YesPlease
 
 case "$runs_on_pool" in
 ubuntu-latest)
@@ -67,7 +96,7 @@ ubuntu-latest)
 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python2"
 	fi
 
-	export GIT_TEST_HTTPD=true
+	setenv --test GIT_TEST_HTTPD true
 	;;
 macos-latest)
 	if [ "$jobname" = osx-gcc ]
@@ -90,9 +119,9 @@ linux-musl)
 	MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8"
 	;;
 linux-leaks)
-	export SANITIZE=leak
-	export GIT_TEST_PASSING_SANITIZE_LEAK=true
+	setenv --build SANITIZE leak
+	setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true
 	;;
 esac
 
-export MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
+setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index e5a21985b44..35d45a9373e 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -9,33 +9,33 @@ export MAKE_TARGETS="all test"
 
 case "$jobname" in
 linux-gcc)
-	export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
 	;;
 linux-TEST-vars)
-	export GIT_TEST_SPLIT_INDEX=yes
-	export GIT_TEST_MERGE_ALGORITHM=recursive
-	export GIT_TEST_FULL_IN_PACK_ARRAY=true
-	export GIT_TEST_OE_SIZE=10
-	export GIT_TEST_OE_DELTA_SIZE=5
-	export GIT_TEST_COMMIT_GRAPH=1
-	export GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS=1
-	export GIT_TEST_MULTI_PACK_INDEX=1
-	export GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP=1
-	export GIT_TEST_ADD_I_USE_BUILTIN=1
-	export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master
-	export GIT_TEST_WRITE_REV_INDEX=1
-	export GIT_TEST_CHECKOUT_WORKERS=2
+	setenv --test GIT_TEST_SPLIT_INDEX yes
+	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
+	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
+	setenv --test GIT_TEST_OE_SIZE 10
+	setenv --test GIT_TEST_OE_DELTA_SIZE 5
+	setenv --test GIT_TEST_COMMIT_GRAPH 1
+	setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
+	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
+	setenv --test GIT_TEST_WRITE_REV_INDEX 1
+	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
 	;;
 linux-clang)
-	export GIT_TEST_DEFAULT_HASH=sha1
+	setenv --test GIT_TEST_DEFAULT_HASH sha1
 	;;
 linux-sha256)
-	export GIT_TEST_DEFAULT_HASH=sha256
+	setenv --test GIT_TEST_DEFAULT_HASH sha256
 	;;
 pedantic)
 	# Don't run the tests; we only care about whether Git can be
 	# built.
-	export DEVOPTS=pedantic
+	setenv --build DEVOPTS pedantic
 	export MAKE_TARGETS=all
 	;;
 esac
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 13/31] CI: remove "run-build-and-tests.sh", run "make [test]" directly

[Ævar Arnfjörð Bjarmason]

Remove the already thin "ci/run-build-and-tests.sh" wrapper and
instead make the CI run "make" or "make test" directly. By doing this
we'll be able to easily see at a glance whether our failure was in the
compilation or testing, whether that's via human eyes or improve
machine readability.

We also need to run our new "ci/check-unignored-build-artifacts.sh" on
success() in the CI now, just like we already had a step conditional
on failure() running ci/print-test-failures.sh.

The reason we used a "ci/run-build-and-tests.sh" wrapper in the first
place had to do with Travis CI-specific constraints that no longer
apply to us, as the Travis CI support has been removed.

Instead we can configure the CI in an earlier step by running
"ci/lib.sh", which under GitHub CI will write the environment
variables we need to the "$GITHUB_ENV" file.

We'll then have access to them in subsequent steps, and crucially
those variables will be prominently visible at the start of each step
via an expandable drop-down in the UI.drop-do.

I.e. this changes the CI run from a top-down flow like (pseudocode):

 - job:
   - step1:
     - use ci/lib.sh to set env vars
     - run a script like ci/run-build-and-tests.sh
   - step2:
     - if: failure()
     - use ci/lib.sh to set env vars
     - run ci/print-test-failures.sh

To:

 - job:
   - step1:
     - set variables in $GITHUB_ENV using ci/lib.sh
   - step2:
     - make
   - step3:
     - make test
   - step4:
     - if: failure()
     - run ci/print-test-failures.sh
   - step5:
     - if: success()
     - run ci/check-unignored-build-artifacts.sh

There is a proposal[2] to get some of the benefits of this approach by
not re-arranging our variable setup in this way, but to instead use
the GitHub CI grouping syntax to focus on the relevant parts of "make"
or "make test" when we have failures.

Doing it this way makes for better looking GitHub CI UI, and lays much
better ground work for our CI going forward. Because:

 * The CI logic will be more portable to a future CI system, since a
   common feature of them is to run various commands in sequence, but
   a future system won't necessarily support the GitHub-specifics
   syntax of "grouping" output within a "step".

   Even if those systems don't support a "$GITHUB_ENV" emulating will
   be much easier than to deal with some CI-specific grouping syntax.

 * At the start of every step the GitHub CI presents an expandable
   list of environment variables from "$GITHUB_ENV". We'll now see
   exactly what variables affected that step (although we currently
   overshoot that a bit, and always define all variables).

 * CI failures will be easier to reproduce locally, as this makes the
   relevant ci/* scripts something that sets up our environment, but
   leaves "make" and "make test" working as they do locally.

   To reproduce a run the user only needs to set the variables
   discussed in the drop-down above, either manually or by running
   "ci/lib.sh".

 * The output will be less verbose. The "ci/lib.sh" script uses "set
   -x", and before this e.g. "ci/static-analysis.sh" would start with
   40 lines of trace output, culminating in using "export" to export
   the relevant environment variables.

   Now that verbosity is in the earlier "ci/lib.sh" step, and not in
   any subsequent one. The "make" targets then start out with the
   relevant non-trace output right away.

 * If we do want to use the grouping syntax within a "step" it'll now
   be easier to do so. It doesn't support nesting, so we'd have to
   make a choice between using it for e.g. "make" v.s. "make test", or
   individual test failures. See "sadly" in [3].

1. https://lore.kernel.org/git/211120.86k0h30zuw.gmgdl@evledraar.gmail.com/
2. https://lore.kernel.org/git/pull.1117.git.1643050574.gitgitgadget@gmail.com/
3. https://lore.kernel.org/git/9333ba781b8240f704e739b00d274f8c3d887e39.1643050574.git.gitgitgadget@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml            | 14 ++++++--
 ci/check-unignored-build-artifacts.sh |  8 +++++
 ci/install-dependencies.sh            |  4 +++
 ci/lib.sh                             | 36 +++++++++++++++++++-
 ci/make-test-artifacts.sh             |  2 --
 ci/run-build-and-tests.sh             | 47 ---------------------------
 ci/run-test-slice.sh                  |  2 --
 ci/test-documentation.sh              |  1 -
 8 files changed, 58 insertions(+), 56 deletions(-)
 create mode 100755 ci/check-unignored-build-artifacts.sh
 delete mode 100755 ci/run-build-and-tests.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index ad752010102..c83f6a21ca3 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -268,7 +268,10 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/run-build-and-tests.sh
+    - run: ci/lib.sh
+    - run: make
+    - run: make test
+      if: success()
     - run: ci/print-test-failures.sh
       if: failure()
     - name: Upload failed tests' directories
@@ -292,6 +295,7 @@ jobs:
           image: daald/ubuntu32:xenial
         - jobname: pedantic
           image: fedora
+          skip-tests: no
     env:
       jobname: ${{matrix.vector.jobname}}
     runs-on: ubuntu-latest
@@ -299,9 +303,12 @@ jobs:
     steps:
     - uses: actions/checkout@v1
     - run: ci/install-docker-dependencies.sh
-    - run: ci/run-build-and-tests.sh
+    - run: ci/lib.sh
+    - run: make
+    - run: make test
+      if: success() && matrix.vector.skip-tests != 'no'
     - run: ci/print-test-failures.sh
-      if: failure()
+      if: failure() && matrix.vector.skip-tests != 'no'
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v1
@@ -317,6 +324,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
+    - run: ci/lib.sh
     - run: make ci-static-analysis
   sparse:
     needs: ci-config
diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh
new file mode 100755
index 00000000000..56d04b0db9a
--- /dev/null
+++ b/ci/check-unignored-build-artifacts.sh
@@ -0,0 +1,8 @@
+#!/bin/sh
+#
+# Check whether the build created anything not in our .gitignore
+#
+
+. ${0%/*}/lib.sh
+
+check_unignored_build_artifacts
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 34b4400f38b..01006379fd0 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -22,6 +22,10 @@ ubuntu-latest)
 	P4_PATH="$HOME/custom/p4"
 	GIT_LFS_PATH="$HOME/custom/git-lfs"
 	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
+	if test -n "$GITHUB_PATH"
+	then
+		echo "$PATH" >>"$GITHUB_PATH"
+	fi
 
 	P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
 	LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
diff --git a/ci/lib.sh b/ci/lib.sh
index 3727e6ec2f9..cc9a368be42 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,3 +1,5 @@
+#!/bin/sh
+
 # Library of functions shared by all CI scripts
 
 # Set 'exit on error' for all CI scripts to let the caller know that
@@ -39,7 +41,10 @@ setenv () {
 	val=$2
 	shift 2
 
-	eval "export $key=\"$val\""
+	if test -n "$GITHUB_ENV"
+	then
+		echo "$key=$val" >>"$GITHUB_ENV"
+	fi
 }
 
 check_unignored_build_artifacts ()
@@ -109,6 +114,35 @@ macos-latest)
 esac
 
 case "$jobname" in
+linux-gcc)
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
+	;;
+linux-TEST-vars)
+	setenv --test GIT_TEST_SPLIT_INDEX yes
+	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
+	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
+	setenv --test GIT_TEST_OE_SIZE 10
+	setenv --test GIT_TEST_OE_DELTA_SIZE 5
+	setenv --test GIT_TEST_COMMIT_GRAPH 1
+	setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
+	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
+	setenv --test GIT_TEST_WRITE_REV_INDEX 1
+	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
+	;;
+linux-clang)
+	setenv --test GIT_TEST_DEFAULT_HASH sha1
+	;;
+linux-sha256)
+	setenv --test GIT_TEST_DEFAULT_HASH sha256
+	;;
+pedantic)
+	# Don't run the tests; we only care about whether Git can be
+	# built.
+	setenv --build DEVOPTS pedantic
+	;;
 linux32)
 	CC=gcc
 	;;
diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh
index 646967481f6..45298562982 100755
--- a/ci/make-test-artifacts.sh
+++ b/ci/make-test-artifacts.sh
@@ -8,5 +8,3 @@ mkdir -p "$1" # in case ci/lib.sh decides to quit early
 . ${0%/*}/lib.sh
 
 make artifacts-tar ARTIFACTS_DIRECTORY="$1"
-
-check_unignored_build_artifacts
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
deleted file mode 100755
index 35d45a9373e..00000000000
--- a/ci/run-build-and-tests.sh
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/bin/sh
-#
-# Build and test Git
-#
-
-. ${0%/*}/lib.sh
-
-export MAKE_TARGETS="all test"
-
-case "$jobname" in
-linux-gcc)
-	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
-	;;
-linux-TEST-vars)
-	setenv --test GIT_TEST_SPLIT_INDEX yes
-	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
-	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
-	setenv --test GIT_TEST_OE_SIZE 10
-	setenv --test GIT_TEST_OE_DELTA_SIZE 5
-	setenv --test GIT_TEST_COMMIT_GRAPH 1
-	setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
-	setenv --test GIT_TEST_MULTI_PACK_INDEX 1
-	setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
-	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
-	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
-	setenv --test GIT_TEST_WRITE_REV_INDEX 1
-	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
-	;;
-linux-clang)
-	setenv --test GIT_TEST_DEFAULT_HASH sha1
-	;;
-linux-sha256)
-	setenv --test GIT_TEST_DEFAULT_HASH sha256
-	;;
-pedantic)
-	# Don't run the tests; we only care about whether Git can be
-	# built.
-	setenv --build DEVOPTS pedantic
-	export MAKE_TARGETS=all
-	;;
-esac
-
-# Any new "test" targets should not go after this "make", but should
-# adjust $MAKE_TARGETS. Otherwise compilation-only targets above will
-# start running tests.
-make $MAKE_TARGETS
-check_unignored_build_artifacts
diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index b9a682b4bcd..70326961454 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -8,5 +8,3 @@
 make --quiet -C t T="$(cd t &&
 	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
 	tr '\n' ' ')"
-
-check_unignored_build_artifacts
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 41e2b126311..6b5cce03bd7 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -39,4 +39,3 @@ test -s Documentation/git.html
 grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
 
 rm -f stdout.log stderr.log stderr.raw
-check_unignored_build_artifacts
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 14/31] ci/lib.sh: use "test" instead of "["

[Ævar Arnfjörð Bjarmason]

Change the ci/lib.sh script to use "if test" rather than "if [". Until
the preceding commit this script relied on the non-POSIX "export X=Y"
syntax, but now that it's POSIX portable let's use our usual style in
it, and not the bash-specific style that was used in it historically.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index cc9a368be42..ffc0e524aaa 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -94,7 +94,7 @@ ubuntu-latest)
 		break
 	fi
 
-	if [ "$jobname" = linux-gcc ]
+	if test "$jobname" = linux-gcc
 	then
 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python3"
 	else
@@ -104,7 +104,7 @@ ubuntu-latest)
 	setenv --test GIT_TEST_HTTPD true
 	;;
 macos-latest)
-	if [ "$jobname" = osx-gcc ]
+	if test "$jobname" = osx-gcc
 	then
 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
 	else
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 15/31] CI: check ignored unignored build artifacts in "win[+VS] build" too

[Ævar Arnfjörð Bjarmason]

Use the newly created "ci/check-unignored-build-artifacts.sh" script
in the "win build" and "win+VS build" steps too. These steps weren't
doing this sanity check of untracked files against .gitignore before,
but as the check passes for them let's keep doing that extra sanity
check.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index c83f6a21ca3..05c5771e7f6 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -93,6 +93,9 @@ jobs:
         HOME: ${{runner.workspace}}
         NO_PERL: 1
       run: . /etc/profile && ci/make-test-artifacts.sh artifacts
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
+      shell: bash
     - name: zip up tracked files
       run: git archive -o artifacts/tracked.tar.gz HEAD
     - name: upload tracked files and build artifacts
@@ -180,6 +183,9 @@ jobs:
       run: |
         mkdir -p artifacts &&
         eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts NO_GETTEXT=YesPlease 2>&1 | grep ^tar)"
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
+      shell: bash
     - name: zip up tracked files
       run: git archive -o artifacts/tracked.tar.gz HEAD
     - name: upload tracked files and build artifacts
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 16/31] CI: invoke "make artifacts-tar" directly in windows-build

[Ævar Arnfjörð Bjarmason]

Change the windows-build job to invoke the "make artifacts-tar" step
directly, instead of calling a "ci/make-test-artifacts.sh" script.

The script was needed because "ci/lib.sh" would set up various
environment variables for us, but now we can instead use the
"ci/lib.sh" in its script mode.

The "mkdir -p" added in b819f1d2cec (ci: parallelize testing on
Windows, 2019-01-29) isn't needed, the same commit added that "mkdir
-p" to the "artifacts-tar" rule itself, so we can have "make" create
the directory for us.

This also has the benefit of making the "build" step less chatty,
since it won't start with the verbose "set -x" output, that's now
contained in the "ci/lib.sh" step.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml |  5 ++++-
 ci/make-test-artifacts.sh  | 10 ----------
 2 files changed, 4 insertions(+), 11 deletions(-)
 delete mode 100755 ci/make-test-artifacts.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 05c5771e7f6..2c23a19eac2 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -87,12 +87,15 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
+    - run: ci/lib.sh
+      shell: bash
     - name: build
       shell: bash
       env:
         HOME: ${{runner.workspace}}
         NO_PERL: 1
-      run: . /etc/profile && ci/make-test-artifacts.sh artifacts
+      run: . /etc/profile && make artifacts-tar ARTIFACTS_DIRECTORY=artifacts
+      if: success()
     - run: ci/check-unignored-build-artifacts.sh
       if: success()
       shell: bash
diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh
deleted file mode 100755
index 45298562982..00000000000
--- a/ci/make-test-artifacts.sh
+++ /dev/null
@@ -1,10 +0,0 @@
-#!/bin/sh
-#
-# Build Git and store artifacts for testing
-#
-
-mkdir -p "$1" # in case ci/lib.sh decides to quit early
-
-. ${0%/*}/lib.sh
-
-make artifacts-tar ARTIFACTS_DIRECTORY="$1"
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 17/31] CI: split up and reduce "ci/test-documentation.sh"

[Ævar Arnfjörð Bjarmason]

Change the "ci/test-documentation.sh" script to run the bash-specific
parts in as one command in the CI job itself, and to run the two "make
doc" commands at the top-level.

It'll now be obvious from the title of the step if we failed in the
asciidoc or asciidoctor step.

Since the "check_unignored_build_artifacts()" function is now only
used in "ci/check-unignored-build-artifacts.sh" move that function
there.

The recipe for the job in ".github/workflows/main.yml" is now a bit
verbose because it's effectively the same job twice, with a "make
clean" in-between. It would be better for the verbosity to run it via
a matrix as done in the alternate approach in [1] does, but then we'd
sacrifice overall CPU time for the brevity. It might still be worth
doing, but let's go for this simpler approach for now.

1. https://lore.kernel.org/git/patch-v2-6.6-7c423c8283d-20211120T030848Z-avarab@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml            | 16 +++++++++++++-
 ci/check-unignored-build-artifacts.sh | 10 +++++++++
 ci/lib.sh                             | 10 ---------
 ci/test-documentation.sh              | 31 +++++++++------------------
 4 files changed, 35 insertions(+), 32 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 2c23a19eac2..92b914f16fd 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -365,4 +365,18 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/test-documentation.sh
+    - run: ci/lib.sh
+    - run: make check-docs
+    - run: "make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)"
+      shell: bash
+    - run: ci/test-documentation.sh AsciiDoc
+      if: success()
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
+    - run: make clean
+    - run: "make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)"
+      shell: bash
+    - run: ci/test-documentation.sh Asciidoctor
+      if: success()
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh
index 56d04b0db9a..0bc04f32804 100755
--- a/ci/check-unignored-build-artifacts.sh
+++ b/ci/check-unignored-build-artifacts.sh
@@ -5,4 +5,14 @@
 
 . ${0%/*}/lib.sh
 
+check_unignored_build_artifacts ()
+{
+	! git ls-files --other --exclude-standard --error-unmatch \
+		-- ':/*' 2>/dev/null ||
+	{
+		echo "$(tput setaf 1)error: found unignored build artifacts$(tput sgr0)"
+		false
+	}
+}
+
 check_unignored_build_artifacts
diff --git a/ci/lib.sh b/ci/lib.sh
index ffc0e524aaa..35a972fa3f5 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -47,16 +47,6 @@ setenv () {
 	fi
 }
 
-check_unignored_build_artifacts ()
-{
-	! git ls-files --other --exclude-standard --error-unmatch \
-		-- ':/*' 2>/dev/null ||
-	{
-		echo "$(tput setaf 1)error: found unignored build artifacts$(tput sgr0)"
-		false
-	}
-}
-
 # GitHub Action doesn't set TERM, which is required by tput
 setenv TERM ${TERM:-dumb}
 
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 6b5cce03bd7..b8a6a6f664e 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -1,10 +1,12 @@
-#!/usr/bin/env bash
+#!/bin/sh
 #
-# Perform sanity checks on documentation and build it.
+# Perform sanity checks on "make doc" output and built documentation
 #
 
 . ${0%/*}/lib.sh
 
+generator=$1
+
 filter_log () {
 	sed -e '/^GIT_VERSION = /d' \
 	    -e "/constant Gem::ConfigMap is deprecated/d" \
@@ -14,28 +16,15 @@ filter_log () {
 	    "$1"
 }
 
-make check-docs
-
-# Build docs with AsciiDoc
-make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
-cat stderr.raw
-filter_log stderr.raw >stderr.log
-test ! -s stderr.log
-test -s Documentation/git.html
-test -s Documentation/git.xml
-test -s Documentation/git.1
-grep '<meta name="generator" content="AsciiDoc ' Documentation/git.html
-
-rm -f stdout.log stderr.log stderr.raw
-check_unignored_build_artifacts
-
-# Build docs with AsciiDoctor
-make clean
-make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
 cat stderr.raw
 filter_log stderr.raw >stderr.log
 test ! -s stderr.log
 test -s Documentation/git.html
-grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
+if test "$generator" = "Asciidoctor"
+then
+	test -s Documentation/git.xml
+	test -s Documentation/git.1
+fi
+grep "<meta name=\"generator\" content=\"$generator " Documentation/git.html
 
 rm -f stdout.log stderr.log stderr.raw
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 18/31] CI: make ci/install-dependencies.sh POSIX-compatible

[Ævar Arnfjörð Bjarmason]

Change the unportable constructs in the "ci/install-dependencies.sh"
scripts to use POSIX replacements, and have it run under "/bin/sh"
instead of "/usr/bin/env bash". We needed to:

* Replace pushd/popd with a sub-shell.
* Change "mkdir --parents" to "mkdir -p".
* Use POSIX "export" syntax.

This is in preparation for combining this script and
"ci/install-docker-dependencies.sh", which doesn't have access to
"bash".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/install-dependencies.sh | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 01006379fd0..db58de0a6fd 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -1,4 +1,4 @@
-#!/usr/bin/env bash
+#!/bin/sh
 #
 # Install dependencies required to build and test Git on Linux and macOS
 #
@@ -21,7 +21,8 @@ ubuntu-latest)
 
 	P4_PATH="$HOME/custom/p4"
 	GIT_LFS_PATH="$HOME/custom/git-lfs"
-	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
+	PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
+	export PATH
 	if test -n "$GITHUB_PATH"
 	then
 		echo "$PATH" >>"$GITHUB_PATH"
@@ -33,22 +34,28 @@ ubuntu-latest)
 	sudo apt-get -q update
 	sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \
 		$UBUNTU_COMMON_PKGS $CC_PACKAGE
-	mkdir --parents "$P4_PATH"
-	pushd "$P4_PATH"
+	mkdir -p "$P4_PATH"
+	(
+		cd "$P4_PATH"
 		wget --quiet "$P4WHENCE/bin.linux26x86_64/p4d"
 		wget --quiet "$P4WHENCE/bin.linux26x86_64/p4"
 		chmod u+x p4d
 		chmod u+x p4
-	popd
-	mkdir --parents "$GIT_LFS_PATH"
-	pushd "$GIT_LFS_PATH"
+	)
+	mkdir -p "$GIT_LFS_PATH"
+	(
+		cd "$GIT_LFS_PATH"
 		wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
 		tar --extract --gunzip --file "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
 		cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs .
-	popd
+	)
 	;;
 macos-latest)
-	export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
+	HOMEBREW_NO_AUTO_UPDATE=1
+	export HOMEBREW_NO_AUTO_UPDATE
+	HOMEBREW_NO_INSTALL_CLEANUP=1
+	export HOMEBREW_NO_INSTALL_CLEANUP
+
 	# Uncomment this if you want to run perf tests:
 	# brew install gnu-time
 	brew link --force gettext
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 19/31] CI: combine ci/install{,-docker}-dependencies.sh

[Ævar Arnfjörð Bjarmason]

Combine the "ci/install-dependencies.sh" and
"ci/install-docker-dependencies.sh" into one script. The "case"
statement in the latter only cared about "$jobname", and can be folded
into the same "case" statement in the former.

The reason they split up is historical, until a preceding commit
"ci/lib.sh" required "bash", which might not have been available in
"docker".

This also fixes issue in "ci/install-docker-dependencies.sh" where
we'd hide errors due to not using "set -e". Now that we include
"ci/lib.sh" we won't have that potential issue.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml        |  2 +-
 ci/install-dependencies.sh        | 15 +++++++++++++++
 ci/install-docker-dependencies.sh | 22 ----------------------
 3 files changed, 16 insertions(+), 23 deletions(-)
 delete mode 100755 ci/install-docker-dependencies.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 92b914f16fd..3fd3e4f873c 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -311,7 +311,7 @@ jobs:
     container: ${{matrix.vector.image}}
     steps:
     - uses: actions/checkout@v1
-    - run: ci/install-docker-dependencies.sh
+    - run: ci/install-dependencies.sh
     - run: ci/lib.sh
     - run: make
     - run: make test
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index db58de0a6fd..8e41747ef6f 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -97,6 +97,21 @@ linux-gcc-default)
 	sudo apt-get -q update
 	sudo apt-get -q -y install $UBUNTU_COMMON_PKGS
 	;;
+linux32)
+	linux32 --32bit i386 sh -c '
+		apt update >/dev/null &&
+		apt install -y build-essential libcurl4-openssl-dev \
+			libssl-dev libexpat-dev gettext python >/dev/null
+	'
+	;;
+linux-musl)
+	apk add --update build-base curl-dev openssl-dev expat-dev gettext \
+		pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null
+	;;
+pedantic)
+	dnf -yq update >/dev/null &&
+	dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null
+	;;
 esac
 
 if type p4d >/dev/null && type p4 >/dev/null
diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh
deleted file mode 100755
index 78b7e326da6..00000000000
--- a/ci/install-docker-dependencies.sh
+++ /dev/null
@@ -1,22 +0,0 @@
-#!/bin/sh
-#
-# Install dependencies required to build and test Git inside container
-#
-
-case "$jobname" in
-linux32)
-	linux32 --32bit i386 sh -c '
-		apt update >/dev/null &&
-		apt install -y build-essential libcurl4-openssl-dev \
-			libssl-dev libexpat-dev gettext python >/dev/null
-	'
-	;;
-linux-musl)
-	apk add --update build-base curl-dev openssl-dev expat-dev gettext \
-		pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null
-	;;
-pedantic)
-	dnf -yq update >/dev/null &&
-	dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null
-	;;
-esac
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 20/31] CI: move "env" definitions into ci/lib.sh

[Ævar Arnfjörð Bjarmason]

Have the GitHub CI jobs use the "ci/lib".sh as a source of truth for
environment variables that affect their builds and tests. This brings
them in line with the rest of the jobs, and make it easier to see the
entire CI configuration per jobname at a glance.

To do this we need to add a ci/lib.sh "step" to the remaining jobs
that didn't have it added in preceding commits.

The Makefile parameters are luckily all accepted via the environment,
so we can export these instead of adding them to MAKEFLAGS.

Let's also use the documented GIT_CONIFG_* mechanism for setting
config instead of the internal GIT_CONFIG_PARAMETERS variable. This
adjusts code added in 889cacb6897 (ci: configure GitHub Actions for
CI/PR, 2020-04-11), we could probably use the GIT_AUTHOR_NAME etc, but
let's keep setting this via config, just with the documented
mechanism.

By setting "ARTIFACTS_DIRECTORY=artifacts" once we don't need to
repeat it in various places, let's get it from the environment
instead.

Finally, the "DEVELOPER: 1" was only needed by the "sparse" job, which
wasn't using "ci/lib.sh" (and which sets DEVELOPER=1). Let's instead
have the "sparse" job use the "ci/lib.sh" and remove DEVELOPER=1 from
".github/workflows/main.yml".

This substantially speeds up the "sparse" job, since it'll now pick up
the "--jobs" setting in MAKEFLAGS that we use everywhere else. Before
it ran in around 4m30s, now in around 2m30s.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 29 +++++++++++------------------
 ci/lib.sh                  | 21 +++++++++++++++++++++
 2 files changed, 32 insertions(+), 18 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 3fd3e4f873c..fe37e0f1b36 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -2,9 +2,6 @@ name: CI
 
 on: [push, pull_request]
 
-env:
-  DEVELOPER: 1
-
 jobs:
   ci-config:
     name: config
@@ -93,8 +90,7 @@ jobs:
       shell: bash
       env:
         HOME: ${{runner.workspace}}
-        NO_PERL: 1
-      run: . /etc/profile && make artifacts-tar ARTIFACTS_DIRECTORY=artifacts
+      run: . /etc/profile && make artifacts-tar
       if: success()
     - run: ci/check-unignored-build-artifacts.sh
       if: success()
@@ -105,7 +101,7 @@ jobs:
       uses: actions/upload-artifact@v2
       with:
         name: windows-artifacts
-        path: artifacts
+        path: ${{env.ARTIFACTS_DIRECTORY}}
   windows-test:
     name: win test
     env:
@@ -126,6 +122,8 @@ jobs:
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
+    - run: ci/lib.sh
+      shell: bash
     - name: test
       shell: bash
       run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
@@ -145,9 +143,6 @@ jobs:
       jobname: vs-build
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
-    env:
-      NO_PERL: 1
-      GIT_CONFIG_PARAMETERS: "'user.name=CI' 'user.email=ci@git'"
     runs-on: windows-latest
     steps:
     - uses: actions/checkout@v2
@@ -171,6 +166,8 @@ jobs:
     - name: copy dlls to root
       shell: cmd
       run: compat\vcbuild\vcpkg_copy_dlls.bat release
+    - run: ci/lib.sh
+      shell: bash
     - name: generate Visual Studio solution
       shell: bash
       run: |
@@ -181,21 +178,16 @@ jobs:
     - name: bundle artifact tar
       shell: bash
       env:
-        MSVC: 1
         VCPKG_ROOT: ${{github.workspace}}\compat\vcbuild\vcpkg
       run: |
-        mkdir -p artifacts &&
-        eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts NO_GETTEXT=YesPlease 2>&1 | grep ^tar)"
-    - run: ci/check-unignored-build-artifacts.sh
-      if: success()
-      shell: bash
+        eval "$(make -n artifacts-tar 2>&1 | grep -e ^mkdir -e ^tar)"
     - name: zip up tracked files
       run: git archive -o artifacts/tracked.tar.gz HEAD
     - name: upload tracked files and build artifacts
       uses: actions/upload-artifact@v2
       with:
         name: vs-artifacts
-        path: artifacts
+        path: ${{env.ARTIFACTS_DIRECTORY}}
   vs-test:
     name: win+VS test
     env:
@@ -216,10 +208,10 @@ jobs:
     - name: extract tracked files and build artifacts
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
+    - run: ci/lib.sh
+      shell: bash
     - name: test
       shell: bash
-      env:
-        NO_SVN_TESTS: 1
       run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
     - name: ci/print-test-failures.sh
       if: failure()
@@ -354,6 +346,7 @@ jobs:
     - uses: actions/checkout@v2
     - name: Install other dependencies
       run: ci/install-dependencies.sh
+    - run: ci/lib.sh
     - run: make sparse
   documentation:
     name: documentation
diff --git a/ci/lib.sh b/ci/lib.sh
index 35a972fa3f5..8c574b21a67 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -104,6 +104,27 @@ macos-latest)
 esac
 
 case "$jobname" in
+windows-build)
+	setenv --build NO_PERL NoThanks
+	setenv --build ARTIFACTS_DIRECTORY artifacts
+	;;
+vs-build)
+	setenv --build NO_PERL NoThanks
+	setenv --build NO_GETTEXT NoThanks
+	setenv --build ARTIFACTS_DIRECTORY artifacts
+	setenv --build INCLUDE_DLLS_IN_ARTIFACTS YesPlease
+	setenv --build MSVC YesPlease
+
+	setenv --build GIT_CONFIG_COUNT 2
+	setenv --build GIT_CONFIG_KEY_0 user.name
+	setenv --build GIT_CONFIG_VALUE_0 CI
+	setenv --build GIT_CONFIG_KEY_1 user.emailname
+	setenv --build GIT_CONFIG_VALUE_1 ci@git
+	setenv --build GIT_CONFIG_VALUE_1 ci@git
+	;;
+vs-test)
+	setenv --test NO_SVN_TESTS YesPlease
+	;;
 linux-gcc)
 	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
 	;;
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 21/31] ci/run-test-slice.sh: replace shelling out with "echo"

[Ævar Arnfjörð Bjarmason]

Amend this code added in b819f1d2cec (ci: parallelize testing on
Windows, 2019-01-29) to stop shelling out to "tr", all we're trying to
do here is to turn '\n'-delimited output into a ' '-delimited list. To
do that we can just use "echo".

None of these files will contain the odd characters using "echo" might
choke on in the general case (i.e. '\'-escapes and the like).

This has the added benefit of not adding a redundant ' ' to the end of
the T-parameter, which didn't matter in practice, but as we'll see in
a subsequent commit makes for slightly nicer output, as we'll show
this variable using a GitHub CI-native feature.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/run-test-slice.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index 70326961454..1de6a18ca47 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -5,6 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-make --quiet -C t T="$(cd t &&
-	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
-	tr '\n' ' ')"
+tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \
+	t[0-9]*.sh))
+make --quiet -C t T="$tests"
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 22/31] CI: pre-select test slice in Windows & VS tests

[Ævar Arnfjörð Bjarmason]

In preceding commits the tests have been changed to do their setup via
$GITHUB_ENV in one step, and to then have subsequent steps that re-use
that environment.

Let's change the "test slice" tests added in b819f1d2cec (ci:
parallelize testing on Windows, 2019-01-29) to do the same. These
tests select 10% of the tests to run in 10 "test slices". Now we'll
select those in a step that immediately precedes the testing step, and
then simply invoke "make -C t -e".

This has the advantage that the tests to be run are now listed in the
standard "Run" drop-down at the start of the "test" step.

Since the "T" variable in "t/Makefile" doesn't normally accept
overrides from the environment we need to invoke "make" with the "-e"
option (a.k.a. "--environment-overrides"). We could also make $(T) in
t/Makefile be a "?=" assigned variable, but this way works, and is
arguably clearer as it's more obvious that we're injecting a special
list of tests that override the normal behavior of that Makefile.

Note that we cannot run the top-level "make test" here, because of how
the Windows CI builds git, i.e. either via CMake or some option that
would cause "make test" to recompile git itself. Instead we run "make
-C t [...]".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml                     | 10 ++++++++--
 ci/{run-test-slice.sh => select-test-slice.sh} |  4 ++--
 2 files changed, 10 insertions(+), 4 deletions(-)
 rename ci/{run-test-slice.sh => select-test-slice.sh} (57%)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index fe37e0f1b36..1e13718b989 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -124,9 +124,12 @@ jobs:
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
     - run: ci/lib.sh
       shell: bash
+    - name: select tests
+      run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
+      shell: bash
     - name: test
       shell: bash
-      run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
+      run: . /etc/profile && make -C t -e
     - name: ci/print-test-failures.sh
       if: failure()
       shell: bash
@@ -210,9 +213,12 @@ jobs:
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
     - run: ci/lib.sh
       shell: bash
+    - name: select tests
+      run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
+      shell: bash
     - name: test
       shell: bash
-      run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
+      run: . /etc/profile && make -C t -e
     - name: ci/print-test-failures.sh
       if: failure()
       shell: bash
diff --git a/ci/run-test-slice.sh b/ci/select-test-slice.sh
similarity index 57%
rename from ci/run-test-slice.sh
rename to ci/select-test-slice.sh
index 1de6a18ca47..ec602f8a053 100755
--- a/ci/run-test-slice.sh
+++ b/ci/select-test-slice.sh
@@ -1,10 +1,10 @@
 #!/bin/sh
 #
-# Test Git in parallel
+# Select a portion of the tests for testing Git in parallel
 #
 
 . ${0%/*}/lib.sh
 
 tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \
 	t[0-9]*.sh))
-make --quiet -C t T="$tests"
+echo T="$tests" >>$GITHUB_ENV
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 23/31] CI: only invoke ci/lib.sh as "steps" in main.yml

[Ævar Arnfjörð Bjarmason]

Change the scripts in ci/ to stop using ci/lib.sh as a library, now
that the only thing it did for them was to "set -ex" and possibly set
TERM=dumb.

Let's create a ci/lib-tput.sh for those that need to use "tput"
instead, and have these scripts invoke "set -ex" themselves.

This makes their invocation a lot less verbose, since they'll be
relying on an earlier step in the CI job to have set the variables in
$GITHUB_ENV, and won't be spewing their own trace output to set those
variables again.

Let's also create a ci/lib-ci-type.sh, and have ci/lib.sh and
ci/print-test-failures.sh share the logic to discover the CI type. We
could have set the CI_TYPE in the environment with "setenv", but let's
avoid that verbosity for this purely internal variable.

The "ci/lib.sh" is now no longer a "Library of functions shared by all
CI scripts", so let's remove that commentary, and the misleading
comment about "set -ex" being for "installing dependencies", we're now
no longer using it in "ci/install-dependencies.sh" (but it does its
own "set -ex").

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/check-unignored-build-artifacts.sh |  4 +++-
 ci/install-dependencies.sh            |  2 +-
 ci/lib-ci-type.sh                     |  9 +++++++++
 ci/lib-tput.sh                        |  5 +++++
 ci/lib.sh                             | 27 ++++++++++-----------------
 ci/print-test-failures.sh             |  6 +++---
 ci/select-test-slice.sh               |  2 +-
 ci/test-documentation.sh              |  2 +-
 8 files changed, 33 insertions(+), 24 deletions(-)
 create mode 100644 ci/lib-ci-type.sh
 create mode 100644 ci/lib-tput.sh

diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh
index 0bc04f32804..c27d6a97f45 100755
--- a/ci/check-unignored-build-artifacts.sh
+++ b/ci/check-unignored-build-artifacts.sh
@@ -3,7 +3,9 @@
 # Check whether the build created anything not in our .gitignore
 #
 
-. ${0%/*}/lib.sh
+set -ex
+
+. ${0%/*}/lib-tput.sh
 
 check_unignored_build_artifacts ()
 {
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 8e41747ef6f..52f2180d5b4 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -3,7 +3,7 @@
 # Install dependencies required to build and test Git on Linux and macOS
 #
 
-. ${0%/*}/lib.sh
+set -ex
 
 UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
  tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
diff --git a/ci/lib-ci-type.sh b/ci/lib-ci-type.sh
new file mode 100644
index 00000000000..6f01fd9e5d9
--- /dev/null
+++ b/ci/lib-ci-type.sh
@@ -0,0 +1,9 @@
+#!/bin/sh
+
+if test "$GITHUB_ACTIONS" = "true"
+then
+	CI_TYPE=github-actions
+else
+	echo "Could not identify CI type" >&2
+	exit 1
+fi
diff --git a/ci/lib-tput.sh b/ci/lib-tput.sh
new file mode 100644
index 00000000000..b62ef13c44c
--- /dev/null
+++ b/ci/lib-tput.sh
@@ -0,0 +1,5 @@
+#!/bin/sh
+
+# GitHub Action doesn't set TERM, which is required by tput
+TERM=${TERM:-dumb}
+export TERM
diff --git a/ci/lib.sh b/ci/lib.sh
index 8c574b21a67..8f7e178f3ff 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,13 +1,9 @@
 #!/bin/sh
-
-# Library of functions shared by all CI scripts
-
-# Set 'exit on error' for all CI scripts to let the caller know that
-# something went wrong.
-# Set tracing executed commands, primarily setting environment variables
-# and installing dependencies.
 set -ex
 
+# Helper libraries
+. ${0%/*}/lib-ci-type.sh
+
 # Starting assertions
 if test -z "$jobname"
 then
@@ -47,18 +43,14 @@ setenv () {
 	fi
 }
 
-# GitHub Action doesn't set TERM, which is required by tput
-setenv TERM ${TERM:-dumb}
-
 # How many jobs to run in parallel?
 NPROC=10
 
 # Clear MAKEFLAGS that may come from the outside world.
 MAKEFLAGS=--jobs=$NPROC
 
-if test "$GITHUB_ACTIONS" = "true"
-then
-	CI_TYPE=github-actions
+case "$CI_TYPE" in
+github-actions)
 	CC="${CC:-gcc}"
 
 	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
@@ -66,11 +58,12 @@ then
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
-else
-	echo "Could not identify CI type" >&2
-	env >&2
+	;;
+*)
+	echo "Unhandled CI type: $CI_TYPE" >&2
 	exit 1
-fi
+	;;
+esac
 
 setenv --build DEVELOPER 1
 setenv --test DEFAULT_TEST_TARGET prove
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 47e2ea1ad10..29f8c332eca 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -3,10 +3,10 @@
 # Print output of failing tests
 #
 
-. ${0%/*}/lib.sh
+set -e
 
-# Tracing executed commands would produce too much noise in the loop below.
-set +x
+. ${0%/*}/lib-ci-type.sh
+. ${0%/*}/lib-tput.sh
 
 cd t/
 
diff --git a/ci/select-test-slice.sh b/ci/select-test-slice.sh
index ec602f8a053..a0332095bb3 100755
--- a/ci/select-test-slice.sh
+++ b/ci/select-test-slice.sh
@@ -3,7 +3,7 @@
 # Select a portion of the tests for testing Git in parallel
 #
 
-. ${0%/*}/lib.sh
+set -ex
 
 tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \
 	t[0-9]*.sh))
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index b8a6a6f664e..64ff212cdaa 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -3,7 +3,7 @@
 # Perform sanity checks on "make doc" output and built documentation
 #
 
-. ${0%/*}/lib.sh
+set -ex
 
 generator=$1
 
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 24/31] CI: narrow down variable definitions in --build and --test

[Ævar Arnfjörð Bjarmason]

In a preceding step the "setvar" function was made to take a
"--build", "--test" or "--all" argument to indicate where the
variables it sets were used.

Let's make use of that by having the relevant parts of
".github/workflows/main.yml" invoke "ci/lib.sh" with those options.

By doing this the set of variables shown in build-only steps will be
fewer, which makes diagnosing anything going on there easier, as we
won't have to look at a deluge of e.g. GIT_TEST_* variables.

Since we'll now always run "--build" for "make" and "--test" for "make
test" we can stop setting the "fat" MAKEFLAGS entirely on the "test"
steps, i.e. the one with PYTHON_PATH, CC etc. These will all be
carried over from the earlier --build step.

That won't be true in the case of the "windows-test" and "vs-test"
jobs, since they run separately from the corresponding "build"
step. So we'll need to make sure that we have the --jobs=N argument
for those.

This doesn't matter that much, as we'll still have --jobs=N in
GIT_PROVE_OPTS. So the only thing we'll use it for is parallelism in
the t/Makefile before we get to running "prove". Still, it's good to
be consistent for good measure, and to run the t/Makefile itself in
parallel.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 20 +++++++++++---------
 ci/lib.sh                  | 30 +++++++++++++++++++++++-------
 2 files changed, 34 insertions(+), 16 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 1e13718b989..0787cadc76b 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -84,7 +84,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
       shell: bash
     - name: build
       shell: bash
@@ -122,7 +122,7 @@ jobs:
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
-    - run: ci/lib.sh
+    - run: ci/lib.sh --test
       shell: bash
     - name: select tests
       run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
@@ -169,7 +169,7 @@ jobs:
     - name: copy dlls to root
       shell: cmd
       run: compat\vcbuild\vcpkg_copy_dlls.bat release
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
       shell: bash
     - name: generate Visual Studio solution
       shell: bash
@@ -211,7 +211,7 @@ jobs:
     - name: extract tracked files and build artifacts
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
-    - run: ci/lib.sh
+    - run: ci/lib.sh --test
       shell: bash
     - name: select tests
       run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
@@ -275,8 +275,9 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make
+    - run: ci/lib.sh --test
     - run: make test
       if: success()
     - run: ci/print-test-failures.sh
@@ -310,8 +311,9 @@ jobs:
     steps:
     - uses: actions/checkout@v1
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make
+    - run: ci/lib.sh --test
     - run: make test
       if: success() && matrix.vector.skip-tests != 'no'
     - run: ci/print-test-failures.sh
@@ -331,7 +333,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make ci-static-analysis
   sparse:
     needs: ci-config
@@ -352,7 +354,7 @@ jobs:
     - uses: actions/checkout@v2
     - name: Install other dependencies
       run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make sparse
   documentation:
     name: documentation
@@ -364,7 +366,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make check-docs
     - run: "make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)"
       shell: bash
diff --git a/ci/lib.sh b/ci/lib.sh
index 8f7e178f3ff..50bc766254b 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -5,6 +5,13 @@ set -ex
 . ${0%/*}/lib-ci-type.sh
 
 # Starting assertions
+mode=$1
+if test -z "$mode"
+then
+	echo "need a $0 mode, e.g. --build or --test" >&2
+	exit 1
+fi
+
 if test -z "$jobname"
 then
 	echo "must set a CI jobname in the environment" >&2
@@ -16,11 +23,12 @@ setenv () {
 	while test $# != 0
 	do
 		case "$1" in
-		--build)
-			;;
-		--test)
-			;;
-		--all)
+		--build | --test)
+			if test "$1" != "$mode"
+			then
+				return 0
+			fi
+
 			;;
 		-*)
 			echo "BUG: bad setenv() option '$1'" >&2
@@ -46,8 +54,12 @@ setenv () {
 # How many jobs to run in parallel?
 NPROC=10
 
+# For "--test" we carry the MAKEFLAGS over from earlier steps, except
+# in stand-alone jobs which will use $COMMON_MAKEFLAGS.
+COMMON_MAKEFLAGS=--jobs=$NPROC
+
 # Clear MAKEFLAGS that may come from the outside world.
-MAKEFLAGS=--jobs=$NPROC
+MAKEFLAGS=$COMMON_MAKEFLAGS
 
 case "$CI_TYPE" in
 github-actions)
@@ -101,6 +113,9 @@ windows-build)
 	setenv --build NO_PERL NoThanks
 	setenv --build ARTIFACTS_DIRECTORY artifacts
 	;;
+windows-test)
+	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
+	;;
 vs-build)
 	setenv --build NO_PERL NoThanks
 	setenv --build NO_GETTEXT NoThanks
@@ -117,6 +132,7 @@ vs-build)
 	;;
 vs-test)
 	setenv --test NO_SVN_TESTS YesPlease
+	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
 	;;
 linux-gcc)
 	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
@@ -162,4 +178,4 @@ linux-leaks)
 	;;
 esac
 
-setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
+setenv --build MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 25/31] CI: add more variables to MAKEFLAGS, except under vs-build

[Ævar Arnfjörð Bjarmason]

It's clearer that "DEVELOPER" is a flag that affects the Makefile
itself in particular if it's put into "MAKEFLAGS than" if it
generically sits in the environment. Let's move both it and
"SKIP_DASHED_BUILT_INS" to "MAKEFLAGS".

We can't do this under vs-build, since that invokes cmake. Let's have
only that job set these in the environment.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 50bc766254b..8fb0bfd43e1 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -61,6 +61,13 @@ COMMON_MAKEFLAGS=--jobs=$NPROC
 # Clear MAKEFLAGS that may come from the outside world.
 MAKEFLAGS=$COMMON_MAKEFLAGS
 
+# Use common options for "make" (cmake in "vs-build" below uses the
+# intermediate variables directly)
+DEVELOPER=1
+MAKEFLAGS="$MAKEFLAGS DEVELOPER=$DEVELOPER"
+SKIP_DASHED_BUILT_INS=YesPlease
+MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
+
 case "$CI_TYPE" in
 github-actions)
 	CC="${CC:-gcc}"
@@ -77,10 +84,8 @@ github-actions)
 	;;
 esac
 
-setenv --build DEVELOPER 1
 setenv --test DEFAULT_TEST_TARGET prove
 setenv --test GIT_TEST_CLONE_2GB true
-setenv --build SKIP_DASHED_BUILT_INS YesPlease
 
 case "$runs_on_pool" in
 ubuntu-latest)
@@ -117,6 +122,9 @@ windows-test)
 	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
 	;;
 vs-build)
+	setenv --build DEVELOPER $DEVELOPER
+	setenv --build SKIP_DASHED_BUILT_INS $SKIP_DASHED_BUILT_INS
+
 	setenv --build NO_PERL NoThanks
 	setenv --build NO_GETTEXT NoThanks
 	setenv --build ARTIFACTS_DIRECTORY artifacts
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 26/31] CI: set CC in MAKEFLAGS directly, don't add it to the environment

[Ævar Arnfjörð Bjarmason]

Rather than pass a "$CC" in the environment to be picked up in
ci/lib.sh let's instead have ci/lib.sh itself add it directly to
MAKEFLAGS. For "$CC_PACKAGE" its setting and use can stay within
ci/install-dependencies.sh.

Setting CC=gcc by default made for confusing trace output, and since a
preceding change to carry it and others over across "steps" in the
GitHub CI it's been even more misleading.  E.g. the "win+VS build" job
confusingly has CC=gcc set, even though it builds with MSVC.

Let's instead rely on the Makefile default of CC=cc, and only override
it for those jobs where it's needed. This does mean that we'll need to
set it for the "pedantic" job, which previously relied on the default
CC=gcc in case "clang" become the default on that platform.

This partially reverts my 707d2f2fe86 (CI: use "$runs_on_pool", not
"$jobname" to select packages & config, 2021-11-23), i.e. we're now
aiming to only set those variables specific jobs need.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 13 -------------
 ci/install-dependencies.sh | 10 ++++++++++
 ci/lib.sh                  | 23 ++++++++++++++++++++---
 3 files changed, 30 insertions(+), 16 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 0787cadc76b..6d25ec4ae3b 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -238,37 +238,24 @@ jobs:
       matrix:
         vector:
           - jobname: linux-clang
-            cc: clang
             pool: ubuntu-latest
           - jobname: linux-sha256
-            cc: clang
             os: ubuntu
             pool: ubuntu-latest
           - jobname: linux-gcc
-            cc: gcc
-            cc_package: gcc-8
             pool: ubuntu-latest
           - jobname: linux-TEST-vars
-            cc: gcc
             os: ubuntu
-            cc_package: gcc-8
             pool: ubuntu-latest
           - jobname: osx-clang
-            cc: clang
             pool: macos-latest
           - jobname: osx-gcc
-            cc: gcc
-            cc_package: gcc-9
             pool: macos-latest
           - jobname: linux-gcc-default
-            cc: gcc
             pool: ubuntu-latest
           - jobname: linux-leaks
-            cc: gcc
             pool: ubuntu-latest
     env:
-      CC: ${{matrix.vector.cc}}
-      CC_PACKAGE: ${{matrix.vector.cc_package}}
       jobname: ${{matrix.vector.jobname}}
       runs_on_pool: ${{matrix.vector.pool}}
     runs-on: ${{matrix.vector.pool}}
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 52f2180d5b4..1d05f43d19e 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -9,6 +9,16 @@ UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
  tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
  libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl"
 
+CC_PACKAGE=
+case "$jobname" in
+linux-gcc | linux-TEST-vars)
+	CC_PACKAGE=gcc-8
+	;;
+osx-gcc)
+	CC_PACKAGE=gcc-9
+	;;
+esac
+
 case "$runs_on_pool" in
 ubuntu-latest)
 	# The Linux build installs the defined dependency versions below.
diff --git a/ci/lib.sh b/ci/lib.sh
index 8fb0bfd43e1..977ee6293ef 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -51,6 +51,9 @@ setenv () {
 	fi
 }
 
+# Clear variables that may come from the outside world.
+CC=
+
 # How many jobs to run in parallel?
 NPROC=10
 
@@ -70,8 +73,6 @@ MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
 
 case "$CI_TYPE" in
 github-actions)
-	CC="${CC:-gcc}"
-
 	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
 	GIT_TEST_OPTS="--verbose-log -x"
 	test Windows != "$RUNNER_OS" ||
@@ -143,9 +144,14 @@ vs-test)
 	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
 	;;
 linux-gcc)
+	CC=gcc
 	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
 	;;
+linux-gcc-default)
+	CC=gcc
+	;;
 linux-TEST-vars)
+	CC=gcc
 	setenv --test GIT_TEST_SPLIT_INDEX yes
 	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
 	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
@@ -160,13 +166,22 @@ linux-TEST-vars)
 	setenv --test GIT_TEST_WRITE_REV_INDEX 1
 	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
 	;;
+osx-gcc)
+	CC=gcc
+	;;
+osx-clang)
+	CC=clang
+	;;
 linux-clang)
+	CC=clang
 	setenv --test GIT_TEST_DEFAULT_HASH sha1
 	;;
 linux-sha256)
+	CC=clang
 	setenv --test GIT_TEST_DEFAULT_HASH sha256
 	;;
 pedantic)
+	CC=gcc
 	# Don't run the tests; we only care about whether Git can be
 	# built.
 	setenv --build DEVOPTS pedantic
@@ -181,9 +196,11 @@ linux-musl)
 	MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8"
 	;;
 linux-leaks)
+	CC=gcc
 	setenv --build SANITIZE leak
 	setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true
 	;;
 esac
 
-setenv --build MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
+MAKEFLAGS="$MAKEFLAGS${CC:+ CC=$CC}"
+setenv --build MAKEFLAGS "$MAKEFLAGS"
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 27/31] CI: set SANITIZE=leak in MAKEFLAGS directly

[Ævar Arnfjörð Bjarmason]

As in the preceding commit's change to do likewise for CC, let's not
set SANITIZE in the environment. When I added the "linux-leaks" job in
in 956d2e4639b (tests: add a test mode for SANITIZE=leak, run it in
CI, 2021-09-23) I missed that I could do that.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 977ee6293ef..ac96b7a1c61 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -197,7 +197,7 @@ linux-musl)
 	;;
 linux-leaks)
 	CC=gcc
-	setenv --build SANITIZE leak
+	MAKEFLAGS="$MAKEFLAGS SANITIZE=leak"
 	setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true
 	;;
 esac
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 28/31] CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case

[Ævar Arnfjörð Bjarmason]

Move the setting of "PYTHON_PATH=[...]" to the "$jobname" case
statement. This partially backs out of my 707d2f2fe86 (CI: use
"$runs_on_pool", not "$jobname" to select packages & config,
2021-11-23), now that we have a "osx-{clang,gcc}" anyway for setting
"$CC" we might as well do away with this part of the "$runs_on_pool"
case.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index ac96b7a1c61..1a7200451e4 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -104,14 +104,6 @@ ubuntu-latest)
 
 	setenv --test GIT_TEST_HTTPD true
 	;;
-macos-latest)
-	if test "$jobname" = osx-gcc
-	then
-		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
-	else
-		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
-	fi
-	;;
 esac
 
 case "$jobname" in
@@ -167,9 +159,11 @@ linux-TEST-vars)
 	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
 	;;
 osx-gcc)
+	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
 	CC=gcc
 	;;
 osx-clang)
+	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
 	CC=clang
 	;;
 linux-clang)
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 29/31] CI: don't use "set -x" in "ci/lib.sh" output

[Ævar Arnfjörð Bjarmason]

Remove the "set -x" from the "ci/lib.sh" output. Before preceding
commits the logic in that file was much more complex, and likely to
fail in some scenarios.

Now we only task "ci/lib.sh" with setting various variables for
subsequent steps in our jobs, so we can start emitting more tailored
debugging output, which makes what it's doing easier to read.

This change also changes the output of the "ci/print-test-failures.sh"
script, since it's the only other user of "ci/lib-ci-type.sh". In that
case it's also helpful to know what "$CI_TYPE" we're using, as that
script doesn't "set -x" and will act differently depending on the
$CI_TYPE.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 23 +++++++++++++++++++++--
 1 file changed, 21 insertions(+), 2 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 1a7200451e4..ab7546800e0 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,5 +1,5 @@
 #!/bin/sh
-set -ex
+set -e
 
 # Helper libraries
 . ${0%/*}/lib-ci-type.sh
@@ -18,15 +18,26 @@ then
 	exit 1
 fi
 
+# Show our configuration
+echo "CONFIG: CI_TYPE=$CI_TYPE" >&2
+echo "CONFIG: jobname=$jobname" >&2
+echo "CONFIG: runs_on_pool=$runs_on_pool" >&2
+echo "CONFIG: GITHUB_ENV=$GITHUB_ENV" >&2
+if test -n "$GITHUB_ENV"
+then
+	echo "CONFIG: GITHUB_ENV=$GITHUB_ENV" >&2
+fi
+
 # Helper functions
 setenv () {
+	local skip=
 	while test $# != 0
 	do
 		case "$1" in
 		--build | --test)
 			if test "$1" != "$mode"
 			then
-				return 0
+				skip=t
 			fi
 
 			;;
@@ -45,10 +56,18 @@ setenv () {
 	val=$2
 	shift 2
 
+	if test -n "$skip"
+	then
+		echo "SKIP '$key=$val'" >&2
+		return 0
+	fi
+
 	if test -n "$GITHUB_ENV"
 	then
 		echo "$key=$val" >>"$GITHUB_ENV"
 	fi
+
+	echo "SET: '$key=$val'" >&2
 }
 
 # Clear variables that may come from the outside world.
-- 
2.36.0.rc2.898.gdf6bbac50ff

[PATCH v4 30/31] CI: have osx-gcc use gcc, not clang

[Ævar Arnfjörð Bjarmason]

Fix a regression in 707d2f2fe86 (CI: use "$runs_on_pool", not
"$jobname" to select packages & config, 2021-11-23). In that commit I
changed CC=gcc from CC=gcc-9, but on OSX the "gcc" in $PATH points to
clang, we need to use gcc-9 instead.

Reported-by: Carlo Arenas <carenas@gmail.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index ab7546800e0..8f80be4c3df 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -179,7 +179,7 @@ linux-TEST-vars)
 	;;
 osx-gcc)
 	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
-	CC=gcc
+	CC=gcc-9
 	;;
 osx-clang)
 	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
-- 
2.36.0.rc2.898.gdf6bbac50ff

Re: [PATCH v4 30/31] CI: have osx-gcc use gcc, not clang

[Phillip Wood]

On 18/04/2022 17:29, Ævar Arnfjörð Bjarmason wrote:
> Fix a regression in 707d2f2fe86 (CI: use "$runs_on_pool", not
> "$jobname" to select packages & config, 2021-11-23). In that commit I
> changed CC=gcc from CC=gcc-9, but on OSX the "gcc" in $PATH points to
> clang, we need to use gcc-9 instead.

As this is fixing a bug in master, perhaps it would be better as a 
separate patch that can be merged before this series (this series could 
be rebased on to the fix)

Best Wishes

Phillip

> Reported-by: Carlo Arenas <carenas@gmail.com>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>   ci/lib.sh | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/ci/lib.sh b/ci/lib.sh
> index ab7546800e0..8f80be4c3df 100755
> --- a/ci/lib.sh
> +++ b/ci/lib.sh
> @@ -179,7 +179,7 @@ linux-TEST-vars)
>   	;;
>   osx-gcc)
>   	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
> -	CC=gcc
> +	CC=gcc-9
>   	;;
>   osx-clang)
>   	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"

Re: [PATCH v4 30/31] CI: have osx-gcc use gcc, not clang

[Junio C Hamano]

Phillip Wood <phillip.wood@talktalk.net> writes:

> On 18/04/2022 17:29, Ævar Arnfjörð Bjarmason wrote:
>> Fix a regression in 707d2f2fe86 (CI: use "$runs_on_pool", not
>> "$jobname" to select packages & config, 2021-11-23). In that commit I
>> changed CC=gcc from CC=gcc-9, but on OSX the "gcc" in $PATH points to
>> clang, we need to use gcc-9 instead.
>
> As this is fixing a bug in master, perhaps it would be better as a
> separate patch that can be merged before this series (this series
> could be rebased on to the fix)

Yeah, that sounds quite sensible.

Or perhaps start from 'master', merge this (and other fixes, if
there are, in this series), merge Dscho's CI updates and then use
that as an updated base?

Re: [PATCH v4 30/31] CI: have osx-gcc use gcc, not clang

[Phillip Wood]

On 20/04/2022 09:13, Junio C Hamano wrote:
> Phillip Wood <phillip.wood@talktalk.net> writes:
> 
>> On 18/04/2022 17:29, Ævar Arnfjörð Bjarmason wrote:
>>> Fix a regression in 707d2f2fe86 (CI: use "$runs_on_pool", not
>>> "$jobname" to select packages & config, 2021-11-23). In that commit I
>>> changed CC=gcc from CC=gcc-9, but on OSX the "gcc" in $PATH points to
>>> clang, we need to use gcc-9 instead.
>>
>> As this is fixing a bug in master, perhaps it would be better as a
>> separate patch that can be merged before this series (this series
>> could be rebased on to the fix)
> 
> Yeah, that sounds quite sensible.
> 
> Or perhaps start from 'master', merge this (and other fixes, if
> there are, in this series), merge Dscho's CI updates and then use
> that as an updated base?

It would certainly be nice to get Dscho's updates merged sooner rather 
than later as I think they represent a more significant improvement for 
CI users. There is a complication though in that Dscho's series adds 
github markup to the build output and this series separates the build 
from the tests which means that is not necessary. I think it should be 
easy enough to change Dscho's series so it only uses github markup for 
the tests which is the main improvement and just wait for the build and 
tests to be separated in this series (ideally they'd be a short easy to 
review series that did just that).

Best Wishes

Phillip

Re: [PATCH v4 30/31] CI: have osx-gcc use gcc, not clang

[Junio C Hamano]

Phillip Wood <phillip.wood123@gmail.com> writes:

> ... There is a complication though in that Dscho's series
> adds github markup to the build output and this series separates the
> build from the tests which means that is not necessary. I think it
> should be easy enough to change Dscho's series so it only uses github
> markup for the tests which is the main improvement and just wait for
> the build and tests to be separated in this series (ideally they'd be
> a short easy to review series that did just that).

OK, Dscho, Ævar, does that sound like a workable plan?  See a reroll
of Dscho's series (which Phillip considers "should be easy enough")
first, get it solid enough and merge down to 'next', and then see
the refactoring by Ævar on top, hopefully with a minumum churn that
makes it impossible for people to review?

Thanks, all.

js/ci-github-workflow-markup, was Re: [PATCH v4 30/31] CI: have osx-gcc use gcc, not clang

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 523 bytes --]

Hi Junio,

On Wed, 20 Apr 2022, Junio C Hamano wrote:

> OK, Dscho, Ævar, does that sound like a workable plan?  See a reroll
> of Dscho's series (which Phillip considers "should be easy enough")
> first, get it solid enough and merge down to 'next', and then see
> the refactoring by Ævar on top, hopefully with a minumum churn that
> makes it impossible for people to review?

That works for me. Give me a few more days to catch up and then I will
send a new iteration of my patch series.

Thanks,
Dscho

Re: [PATCH v4 30/31] CI: have osx-gcc use gcc, not clang

[Ævar Arnfjörð Bjarmason]

On Wed, Apr 20 2022, Junio C Hamano wrote:

> Phillip Wood <phillip.wood123@gmail.com> writes:
>
>> ... There is a complication though in that Dscho's series
>> adds github markup to the build output and this series separates the
>> build from the tests which means that is not necessary. I think it
>> should be easy enough to change Dscho's series so it only uses github
>> markup for the tests which is the main improvement and just wait for
>> the build and tests to be separated in this series (ideally they'd be
>> a short easy to review series that did just that).
>
> OK, Dscho, Ævar, does that sound like a workable plan?  See a reroll
> of Dscho's series (which Phillip considers "should be easy enough")
> first, get it solid enough and merge down to 'next', and then see
> the refactoring by Ævar on top, hopefully with a minumum churn that
> makes it impossible for people to review?

First. I see you re-queued a the v2 of Johannes's patches. I had some
outstanding fixes for my series [2] + the derived version of his on top
[3] which I thought I'd submit regardless.

To reply a bit to the thread at large, quoting Phillip:

On Wed, Apr 20 2022, Phillip Wood wrote:

> It would certainly be nice to get Dscho's updates merged sooner rather
> than later as I think they represent a more significant improvement
> for CI users. There is a complication though in that Dscho's series
> adds github markup to the build output and this series separates the
> build from the tests which means that is not necessary. I think it
> should be easy enough to change Dscho's series so it only uses github
> markup for the tests which is the main improvement and just wait for
> the build and tests to be separated in this series (ideally they'd be
> a short easy to review series that did just that).

Phillip: I'd be very happy to see a version of my [2] rebased on [2] (or
[3]), rather than the other way around. I'm fairly sure that if you
attempt to do so you won't find it to be all that easy.

If you do and come up with some version which Johannes is OK with I'd be
very happy with it. Without that I don't see how I wouldn't be in the
same rut of wanting to improve other parts of the CI and that being
brought up as a perceived blocker, without a suggested way forward (per
[4]).

But as to a way forward. Even though my series is larger I still think a
much better way forward is to do it first.

There's been some bugs here & there (all of which I believe are
resolved), but I don't think anyone's spotted anything that's
fundamentally wrong with that needs to be addressed in some open-ended
way.

Whereas Johannes's v2 changes some UX things arguably for the better,
but also introduces severe performance regressions in the UX. See [5]
and [6]. I was routinely seeing e.g. ~5s loading times go to ~20s.

Now, I'm not sure (I haven't re-done the measurements), but I have a
hunch that basing it on top of my series made it much better.

That's because in Johannes's the GitHub folding JS needs to fold the
equivalent of make+make test+his GitHub markup. If it's based my series
the the "make" steps are in their own "step", which it looks as though
the GitHub UX needs to do less work to fold/unfold.

1. https://lore.kernel.org/git/pull.1117.v2.git.1646130289.gitgitgadget@gmail.com/
2. https://lore.kernel.org/git/cover-v5-00.29-00000000000-20220421T181526Z-avarab@gmail.com/
3. https://lore.kernel.org/git/RFC-cover-v5-00.10-00000000000-20220421T183001Z-avarab@gmail.com/
4. https://lore.kernel.org/git/220421.86fsm66zmz.gmgdl@evledraar.gmail.com/
5. https://lore.kernel.org/git/220220.86bkz1d7hm.gmgdl@evledraar.gmail.com/
6. https://lore.kernel.org/git/220222.86tucr6kz5.gmgdl@evledraar.gmail.com/

Re: [PATCH v4 30/31] CI: have osx-gcc use gcc, not clang

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> First. I see you re-queued a the v2 of Johannes's patches. I had some
> outstanding fixes for my series [2] + the derived version of his on top
> [3] which I thought I'd submit regardless.

I do not mind flip-flopping on alternate days for a week or so.

Quite honestly I do not care which ones come first all that much.
All I care about is (1) we keep CI functional and (2) we do not
butcher Makefile too badly in the process.  Part of "functional" is
"not unusably slow".

Making the CI process less locked into a particular vendor and
allowing the build/test with a configuration as similar as what is
run on CI locally are nice to have that can come at the same time,
but I wouldn't shed tears if we defer them to later, as long as they
eventually materialize.

If you can fix Dscho's series by adding a series on top, that sounds
like one approach of lessor resistance.

[PATCH v4 31/31] CI: make it easy to use ci/*.sh outside of CI

[Ævar Arnfjörð Bjarmason]

In preceding commits the ci/.sh scripts have lost most of their
CI-specific assumptions. Let's go even further and explicitly support
running ci/lib.sh outside of CI.

This was possible before by faking up enough CI-specific variables,
but as shown in the new "help" output being added here using the
ci/lib.sh to provide "CI-like" has now become trivial.

The ci/print-test-failures.sh scripts can now be used outside of CI as
well, the only GitHub CI-specific part is now guarded by a check that
we'll pass if outside of GitHub CI.

There's also a special-case here to not clobber $MAKEFLAGS in the
environment if we're outside of CI, in case the user has e.g. "jN" or
other flags to "make" that they'd prefer configured already.

Using "ci/lib.sh" as a stand-alone script is much more useful if it
doesn't hardcode NPROC=10, let's provide a poor shellscript
replacement for the online_cpus() we have in thread-utils.c to cover
the most common OS's. It was suggested to use "2>&1" to invoke
"command -v", but per my reading of [2] and my own testing that
doesn't seem to be needed. Perhaps it's only needed for "which(1)"?

1. https://lore.kernel.org/git/214f8670-91d5-f4b6-efa1-76966c3ab1ee@sunshineco.com/
2. https://pubs.opengroup.org/onlinepubs/009604499/utilities/command.html

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib-ci-type.sh         |  3 ---
 ci/lib-online_cpus.sh     | 26 ++++++++++++++++++
 ci/lib.sh                 | 57 ++++++++++++++++++++++++++++++++++++---
 ci/print-test-failures.sh |  4 ---
 4 files changed, 79 insertions(+), 11 deletions(-)
 create mode 100644 ci/lib-online_cpus.sh

diff --git a/ci/lib-ci-type.sh b/ci/lib-ci-type.sh
index 6f01fd9e5d9..09acab7aaec 100644
--- a/ci/lib-ci-type.sh
+++ b/ci/lib-ci-type.sh
@@ -3,7 +3,4 @@
 if test "$GITHUB_ACTIONS" = "true"
 then
 	CI_TYPE=github-actions
-else
-	echo "Could not identify CI type" >&2
-	exit 1
 fi
diff --git a/ci/lib-online_cpus.sh b/ci/lib-online_cpus.sh
new file mode 100644
index 00000000000..6ce7ec5692d
--- /dev/null
+++ b/ci/lib-online_cpus.sh
@@ -0,0 +1,26 @@
+#!/bin/sh
+
+# TODO: Ideally we'd compile t/helper/test-online-cpus.c early, but
+# that presents a chicken & egg problem. But if we move it to a
+# stand-oline command...
+online_cpus() {
+	NPROC=
+
+	if command -v nproc >/dev/null
+	then
+		# GNU coreutils
+		NPROC=$(nproc)
+	elif command -v sysctl >/dev/null
+	then
+		# BSD & Mac OS X
+		NPROC=$(sysctl -n hw.ncpu)
+	elif test -n "$NUMBER_OF_PROCESSORS"
+	then
+		# Windows
+		NPROC="$NUMBER_OF_PROCESSORS"
+	else
+		NPROC=1
+	fi
+
+	echo $NPROC
+}
diff --git a/ci/lib.sh b/ci/lib.sh
index 8f80be4c3df..362bd031df0 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,6 +1,30 @@
 #!/bin/sh
 set -e
 
+#  Usage
+CI_TYPE_HELP_COMMANDS='
+	# run "make all test" like the "linux-leaks" job
+	(eval $(jobname=linux-leaks ci/lib.sh --all) && make test)
+
+	# run "make all test" like the "linux-musl" job
+	(eval $(jobname=linux-musl ci/lib.sh --all) && make test)
+
+	# run "make test" like the "linux-TEST-vars" job (uses various GIT_TEST_* modes)
+	make && (eval $(jobname=linux-TEST-vars ci/lib.sh --test) && make test)
+
+	# run "make test" like the "linux-sha256" job
+	make && (eval $(jobname=linux-sha256 ci/lib.sh --test) && make test)
+'
+
+CI_TYPE_HELP="
+running $0 outside of CI? You can use ci/lib.sh to set up your
+environment like a given CI job. E.g.:
+$CI_TYPE_HELP_COMMANDS
+
+note that some of these (e.g. the linux-musl one) may not work as
+expected due to the CI job configuring a platform that may not match
+yours."
+
 # Helper libraries
 . ${0%/*}/lib-ci-type.sh
 
@@ -9,6 +33,10 @@ mode=$1
 if test -z "$mode"
 then
 	echo "need a $0 mode, e.g. --build or --test" >&2
+	if test -z "$CI_TYPE"
+	then
+		echo "$CI_TYPE_HELP" >&2
+	fi
 	exit 1
 fi
 
@@ -35,7 +63,7 @@ setenv () {
 	do
 		case "$1" in
 		--build | --test)
-			if test "$1" != "$mode"
+			if test "$1" != "$mode" && test "$mode" != "--all"
 			then
 				skip=t
 			fi
@@ -65,6 +93,10 @@ setenv () {
 	if test -n "$GITHUB_ENV"
 	then
 		echo "$key=$val" >>"$GITHUB_ENV"
+	elif test -z "$CI_TYPE"
+	then
+		echo "$key=\"$val\""
+		echo "export $key"
 	fi
 
 	echo "SET: '$key=$val'" >&2
@@ -75,10 +107,25 @@ CC=
 
 # How many jobs to run in parallel?
 NPROC=10
+case "$CI_TYPE" in
+'')
+	. ${0%/*}/lib-online_cpus.sh
+	NPROC=$(online_cpus)
+
+	if test -n "$MAKEFLAGS"
+	then
+		COMMON_MAKEFLAGS="$MAKEFLAGS"
+	else
+		COMMON_MAKEFLAGS=--jobs=$NPROC
+	fi
+	;;
+*)
+	# For "--test" we carry the MAKEFLAGS over from earlier steps, except
+	# in stand-alone jobs which will use $COMMON_MAKEFLAGS.
+	COMMON_MAKEFLAGS=--jobs=$NPROC
+	;;
+esac
 
-# For "--test" we carry the MAKEFLAGS over from earlier steps, except
-# in stand-alone jobs which will use $COMMON_MAKEFLAGS.
-COMMON_MAKEFLAGS=--jobs=$NPROC
 
 # Clear MAKEFLAGS that may come from the outside world.
 MAKEFLAGS=$COMMON_MAKEFLAGS
@@ -98,6 +145,8 @@ github-actions)
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
 	;;
+'')
+	;;
 *)
 	echo "Unhandled CI type: $CI_TYPE" >&2
 	exit 1
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 29f8c332eca..ec395c79ccd 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -36,10 +36,6 @@ do
 			cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
 			tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
 			;;
-		*)
-			echo "Unhandled CI type: $CI_TYPE" >&2
-			exit 1
-			;;
 		esac
 	fi
 done
-- 
2.36.0.rc2.898.gdf6bbac50ff

Re: [PATCH v4 31/31] CI: make it easy to use ci/*.sh outside of CI

[Eric Sunshine]

On Mon, Apr 18, 2022 at 12:29 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> In preceding commits the ci/.sh scripts have lost most of their
> CI-specific assumptions. Let's go even further and explicitly support
> running ci/lib.sh outside of CI.
>
> This was possible before by faking up enough CI-specific variables,
> but as shown in the new "help" output being added here using the
> ci/lib.sh to provide "CI-like" has now become trivial.
>
> The ci/print-test-failures.sh scripts can now be used outside of CI as
> well, the only GitHub CI-specific part is now guarded by a check that
> we'll pass if outside of GitHub CI.
>
> There's also a special-case here to not clobber $MAKEFLAGS in the
> environment if we're outside of CI, in case the user has e.g. "jN" or
> other flags to "make" that they'd prefer configured already.
>
> Using "ci/lib.sh" as a stand-alone script is much more useful if it
> doesn't hardcode NPROC=10, let's provide a poor shellscript
> replacement for the online_cpus() we have in thread-utils.c to cover
> the most common OS's. It was suggested to use "2>&1" to invoke
> "command -v", but per my reading of [2] and my own testing that
> doesn't seem to be needed. Perhaps it's only needed for "which(1)"?

Not redirecting stderr makes sense. I almost certainly accidentally
typed "type foo" reflexively even though I meant to type "command -v
foo", which explains why I thought redirecting stderr was needed. The
explanation above probably would have best been done simply as a reply
to the review thread rather than as part of the commit message, but
dropping it from the commit message is not worth a reroll.

> 1. https://lore.kernel.org/git/214f8670-91d5-f4b6-efa1-76966c3ab1ee@sunshineco.com/
> 2. https://pubs.opengroup.org/onlinepubs/009604499/utilities/command.html
>
> Helped-by: Eric Sunshine <sunshine@sunshineco.com>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
> diff --git a/ci/lib-online_cpus.sh b/ci/lib-online_cpus.sh
> @@ -0,0 +1,26 @@
> +# TODO: Ideally we'd compile t/helper/test-online-cpus.c early, but
> +# that presents a chicken & egg problem. But if we move it to a
> +# stand-oline command...

Um, what's a "stand-oline"?

Re: [PATCH v4 31/31] CI: make it easy to use ci/*.sh outside of CI

[Ævar Arnfjörð Bjarmason]

On Mon, Apr 18 2022, Eric Sunshine wrote:

> On Mon, Apr 18, 2022 at 12:29 PM Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>> In preceding commits the ci/.sh scripts have lost most of their
>> CI-specific assumptions. Let's go even further and explicitly support
>> running ci/lib.sh outside of CI.
>>
>> This was possible before by faking up enough CI-specific variables,
>> but as shown in the new "help" output being added here using the
>> ci/lib.sh to provide "CI-like" has now become trivial.
>>
>> The ci/print-test-failures.sh scripts can now be used outside of CI as
>> well, the only GitHub CI-specific part is now guarded by a check that
>> we'll pass if outside of GitHub CI.
>>
>> There's also a special-case here to not clobber $MAKEFLAGS in the
>> environment if we're outside of CI, in case the user has e.g. "jN" or
>> other flags to "make" that they'd prefer configured already.
>>
>> Using "ci/lib.sh" as a stand-alone script is much more useful if it
>> doesn't hardcode NPROC=10, let's provide a poor shellscript
>> replacement for the online_cpus() we have in thread-utils.c to cover
>> the most common OS's. It was suggested to use "2>&1" to invoke
>> "command -v", but per my reading of [2] and my own testing that
>> doesn't seem to be needed. Perhaps it's only needed for "which(1)"?
>
> Not redirecting stderr makes sense. I almost certainly accidentally
> typed "type foo" reflexively even though I meant to type "command -v
> foo", which explains why I thought redirecting stderr was needed. The
> explanation above probably would have best been done simply as a reply
> to the review thread rather than as part of the commit message, but
> dropping it from the commit message is not worth a reroll.

Thanks. I figured including it was worthwhile, since I'd probably on
another day have looked at it & mixed them up the same way, i.e. you
need the redirection for "which"...

>> 1. https://lore.kernel.org/git/214f8670-91d5-f4b6-efa1-76966c3ab1ee@sunshineco.com/
>> 2. https://pubs.opengroup.org/onlinepubs/009604499/utilities/command.html
>>
>> Helped-by: Eric Sunshine <sunshine@sunshineco.com>
>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>> ---
>> diff --git a/ci/lib-online_cpus.sh b/ci/lib-online_cpus.sh
>> @@ -0,0 +1,26 @@
>> +# TODO: Ideally we'd compile t/helper/test-online-cpus.c early, but
>> +# that presents a chicken & egg problem. But if we move it to a
>> +# stand-oline command...
>
> Um, what's a "stand-oline"?

Typo for stand-alone, sorry.

[PATCH v5 00/29] CI: run "make" in CI "steps", improve UX

[Ævar Arnfjörð Bjarmason]

This series changes CI "step" targets that are shellscripts that do N
things to instead be single command invocations at the "step" level,
driven by the CI recipe itself.

To do that we need to pass state that we previously re-setup for every
"step" via $GITHUB_ENV, whose state we then helpfully show (this is just
a standard GitHub CI feature) in a drop-down at the start of every
"step".

I.e. at the tip of this series you can reliably look at that
$GITHUB_ENV view to see what the full and relevant environment was for
that "make", "make test" or whatever.

Change since v4:

 * Minor UX improvements for 29/29, e.g. "error: " prefixes for error
   messages.
 * Addressed feedback from Phillip Wood at
   https://lore.kernel.org/git/1157a463-f6c6-1df5-59cd-419d73eed1df@gmail.com/:

   - That bash-specific ${../../} syntax is gone
   - The inverted "skip-test: no" is now a "skip-test: yes", as it should be.

   - The new "trace" output from ci/lib.sh now doesn't have "SKIP"
     lines, i.e. we just say what variables we do set, not what we
     don't set. E.g.:
     https://github.com/avar/git/runs/6093145633?check_suite_focus=true#step:4:13

   - I looked at the "a lot of churn" comment, there's e.g. "CI:
     export variables via a wrapper" where we first make a wrapper to
     "export" variables, then make further use of it, some of which
     involves touching the same lines again.

     Likewise first invoking ci/lib.sh, then splitting that up into
     ci/lib.sh --build and ci/lib.sh --test.

     I looked at squashing some of that together, but I think it
     really makes it harder to review & reason about to do N things at
     the same time, even if the net diff size is bigger.
 * Squashed together two commits that were adjusting bash syntax to be
   POSIX syntax.
 * For the pedantic job that skips tests the "ci/lib.sh --test" step
   of setting up variables for it is now also skipped. See:
   https://github.com/avar/git/runs/6093144782?check_suite_focus=true
 * Made ci/select-test-slice.sh do the same "are we in GitHub CI?"
   check as the rest of the scripts, it's now easier to run it
   stand-alone for testing.
 * Adjusted usage output, made it friendlier, moved those UX bits over
   to a test_commit-like getopts parsing.
 * Fixed a typo/commit message suggestion by Eric Sunshine.

 * This series contained a regression fix for an issue already on
   "master", which I've split out in
   https://lore.kernel.org/git/patch-v2-1.1-92acf9420a9-20220421T174733Z-avarab@gmail.com/

   When the two are merged there's a conflict that's resolved thusly,
   per a --remerge-diff (which I snipped away with [...]). I.e. gcc-8
   and gcc-9 need to be added to the linux-gcc and osx-gcc part of
   ci/lib.sh:
	
	diff --git a/ci/lib.sh b/ci/lib.sh
	remerge CONFLICT (content): Merge conflict in ci/lib.sh
	index 407e994edcd..a4e9d51ad6c 100755
	--- a/ci/lib.sh
	+++ b/ci/lib.sh
	@@ -30,126 +30,6 @@ usage() {
	 	then
	 		echo "$CI_TYPE_HELP" >&2
	 	fi
	-<<<<<<< ca24940186b (Merge branch 'avar/ci-have-osx-p4-install-fall-back-on-sha256-no-check' into build-master)
	-=======
	[...snipped ~100 lines ... ]
	->>>>>>> 92acf9420a9 (CI: select CC based on CC_PACKAGE (again))
	 	exit 1
	 }
	 
	@@ -373,7 +253,7 @@ vs-test)
	 	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
	 	;;
	 linux-gcc)
	-	CC=gcc
	+	CC=gcc-8
	 	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
	 	;;
	 linux-gcc-default)
	@@ -397,7 +277,7 @@ linux-TEST-vars)
	 	;;
	 osx-gcc)
	 	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
	-	CC=gcc
	+	CC=gcc-9
	 	;;
	 osx-clang)
	 	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"

Ævar Arnfjörð Bjarmason (29):
  CI: run "set -ex" early in ci/lib.sh
  CI: make "$jobname" explicit, remove fallback
  CI: remove more dead Travis CI support
  CI: remove dead "tree skipping" code
  CI: remove unused Azure ci/* code
  CI/lib.sh: stop adding leading whitespace to $MAKEFLAGS
  CI: don't have "git grep" invoke a pager in tree content check
  CI: have "static-analysis" run a "make ci-static-analysis" target
  CI: have "static-analysis" run "check-builtins", not "documentation"
  CI: move p4 and git-lfs variables to ci/install-dependencies.sh
  CI: consistently use "export" in ci/lib.sh
  CI: export variables via a wrapper
  CI: remove "run-build-and-tests.sh", run "make [test]" directly
  CI: make ci/{lib,install-dependencies}.sh POSIX-compatible
  CI: check ignored unignored build artifacts in "win[+VS] build" too
  CI: invoke "make artifacts-tar" directly in windows-build
  CI: split up and reduce "ci/test-documentation.sh"
  CI: combine ci/install{,-docker}-dependencies.sh
  CI: move "env" definitions into ci/lib.sh
  ci/run-test-slice.sh: replace shelling out with "echo"
  CI: pre-select test slice in Windows & VS tests
  CI: only invoke ci/lib.sh as "steps" in main.yml
  CI: narrow down variable definitions in --build and --test
  CI: add more variables to MAKEFLAGS, except under vs-build
  CI: set CC in MAKEFLAGS directly, don't add it to the environment
  CI: set SANITIZE=leak in MAKEFLAGS directly
  CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case
  CI: don't use "set -x" in "ci/lib.sh" output
  CI: make it easy to use ci/*.sh outside of CI

 .github/workflows/main.yml            | 101 +++---
 Makefile                              |  31 +-
 ci/check-directional-formatting.bash  |   2 +-
 ci/check-unignored-build-artifacts.sh |  20 ++
 ci/install-dependencies.sh            |  82 +++--
 ci/install-docker-dependencies.sh     |  22 --
 ci/lib-ci-type.sh                     |   6 +
 ci/lib-online_cpus.sh                 |  26 ++
 ci/lib-tput.sh                        |   5 +
 ci/lib.sh                             | 424 ++++++++++++++++----------
 ci/make-test-artifacts.sh             |  12 -
 ci/mount-fileshare.sh                 |  25 --
 ci/print-test-failures.sh             |  53 +---
 ci/run-build-and-tests.sh             |  54 ----
 ci/run-docker-build.sh                |  66 ----
 ci/run-docker.sh                      |  47 ---
 ci/run-static-analysis.sh             |  32 --
 ci/run-test-slice.sh                  |  17 --
 ci/select-test-slice.sh               |  13 +
 ci/test-documentation.sh              |  37 +--
 ci/util/extract-trash-dirs.sh         |  50 ---
 shared.mak                            |   1 +
 22 files changed, 508 insertions(+), 618 deletions(-)
 create mode 100755 ci/check-unignored-build-artifacts.sh
 delete mode 100755 ci/install-docker-dependencies.sh
 create mode 100644 ci/lib-ci-type.sh
 create mode 100644 ci/lib-online_cpus.sh
 create mode 100644 ci/lib-tput.sh
 delete mode 100755 ci/make-test-artifacts.sh
 delete mode 100755 ci/mount-fileshare.sh
 delete mode 100755 ci/run-build-and-tests.sh
 delete mode 100755 ci/run-docker-build.sh
 delete mode 100755 ci/run-docker.sh
 delete mode 100755 ci/run-static-analysis.sh
 delete mode 100755 ci/run-test-slice.sh
 create mode 100755 ci/select-test-slice.sh
 delete mode 100755 ci/util/extract-trash-dirs.sh

Range-diff against v4:
 1:  ed27ca0b7e7 =  1:  84d8098e365 CI: run "set -ex" early in ci/lib.sh
 2:  737fe1e2c4c !  2:  3f5c071b0f6 CI: make "$jobname" explicit, remove fallback
    @@ ci/lib.sh
     +# Starting assertions
     +if test -z "$jobname"
     +then
    -+	echo "must set a CI jobname in the environment" >&2
    ++	echo "error: must set a CI jobname in the environment" >&2
     +	exit 1
     +fi
     +
 3:  d875069fc28 =  3:  c64845a4efb CI: remove more dead Travis CI support
 4:  2590459db33 =  4:  45856e67d5d CI: remove dead "tree skipping" code
 5:  e311ea8b9b3 =  5:  d4671148eb7 CI: remove unused Azure ci/* code
 6:  151f28e3818 =  6:  45ea80d0495 CI/lib.sh: stop adding leading whitespace to $MAKEFLAGS
 7:  6c2a8ee9c4e =  7:  a264b40ba0d CI: don't have "git grep" invoke a pager in tree content check
 8:  64a0a1fc8ce =  8:  4a58f7be8d4 CI: have "static-analysis" run a "make ci-static-analysis" target
 9:  0f780bf9e3c =  9:  78c0c81af43 CI: have "static-analysis" run "check-builtins", not "documentation"
10:  3cdeae9c141 = 10:  07e36035a1e CI: move p4 and git-lfs variables to ci/install-dependencies.sh
11:  1d3d357dcef = 11:  272bd590a95 CI: consistently use "export" in ci/lib.sh
12:  5ef0756d095 = 12:  ea35327e24c CI: export variables via a wrapper
13:  e0ce614eb6d ! 13:  f6ff81f48ab CI: remove "run-build-and-tests.sh", run "make [test]" directly
    @@ .github/workflows/main.yml: jobs:
                image: daald/ubuntu32:xenial
              - jobname: pedantic
                image: fedora
    -+          skip-tests: no
    ++          skip-tests: yes
          env:
            jobname: ${{matrix.vector.jobname}}
          runs-on: ubuntu-latest
    @@ .github/workflows/main.yml: jobs:
     +    - run: ci/lib.sh
     +    - run: make
     +    - run: make test
    -+      if: success() && matrix.vector.skip-tests != 'no'
    ++      if: success() && matrix.vector.skip-tests != 'yes'
          - run: ci/print-test-failures.sh
     -      if: failure()
    -+      if: failure() && matrix.vector.skip-tests != 'no'
    ++      if: failure() && matrix.vector.skip-tests != 'yes'
          - name: Upload failed tests' directories
            if: failure() && env.FAILED_TEST_ARTIFACTS != ''
            uses: actions/upload-artifact@v1
14:  a8e70124929 <  -:  ----------- ci/lib.sh: use "test" instead of "["
18:  eec7a11376e ! 14:  533dbc50c4f CI: make ci/install-dependencies.sh POSIX-compatible
    @@ Metadata
     Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## Commit message ##
    -    CI: make ci/install-dependencies.sh POSIX-compatible
    +    CI: make ci/{lib,install-dependencies}.sh POSIX-compatible
     
         Change the unportable constructs in the "ci/install-dependencies.sh"
         scripts to use POSIX replacements, and have it run under "/bin/sh"
    @@ Commit message
         "ci/install-docker-dependencies.sh", which doesn't have access to
         "bash".
     
    +    While we're at it change "if [" to "if test" in ci/lib.sh. This isn't
    +    strictly speaking a case of POSIX-incompatibility, but that syntax was
    +    being used there because this code was using idiomatic bash
    +    constructs. Let's have it use our usual style of "if test" instead.
    +
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## ci/install-dependencies.sh ##
    @@ ci/install-dependencies.sh: ubuntu-latest)
      	# Uncomment this if you want to run perf tests:
      	# brew install gnu-time
      	brew link --force gettext
    +@@ ci/install-dependencies.sh: macos-latest)
    + 
    + 	if test -n "$CC_PACKAGE"
    + 	then
    +-		BREW_PACKAGE=${CC_PACKAGE/-/@}
    ++		BREW_PACKAGE=$(echo $CC_PACKAGE | tr '-' '@')
    + 		brew install "$BREW_PACKAGE"
    + 		brew link "$BREW_PACKAGE"
    + 	fi
    +
    + ## ci/lib.sh ##
    +@@ ci/lib.sh: ubuntu-latest)
    + 		break
    + 	fi
    + 
    +-	if [ "$jobname" = linux-gcc ]
    ++	if test "$jobname" = linux-gcc
    + 	then
    + 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python3"
    + 	else
    +@@ ci/lib.sh: ubuntu-latest)
    + 	setenv --test GIT_TEST_HTTPD true
    + 	;;
    + macos-latest)
    +-	if [ "$jobname" = osx-gcc ]
    ++	if test "$jobname" = osx-gcc
    + 	then
    + 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
    + 	else
15:  bb335d1c0ca = 15:  6044c2b383f CI: check ignored unignored build artifacts in "win[+VS] build" too
16:  5458e6dab0b = 16:  4d1a9f88c32 CI: invoke "make artifacts-tar" directly in windows-build
17:  5e5a5b71700 = 17:  5c8f464a60f CI: split up and reduce "ci/test-documentation.sh"
19:  a9c98582de1 = 18:  a218be76819 CI: combine ci/install{,-docker}-dependencies.sh
20:  96e7eb68906 = 19:  83e21b4f501 CI: move "env" definitions into ci/lib.sh
21:  fdbe8554f5d = 20:  fd0e1610577 ci/run-test-slice.sh: replace shelling out with "echo"
22:  f257ab59971 ! 21:  0c7cd9d64ba CI: pre-select test slice in Windows & VS tests
    @@ ci/run-test-slice.sh => ci/select-test-slice.sh
      tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \
      	t[0-9]*.sh))
     -make --quiet -C t T="$tests"
    -+echo T="$tests" >>$GITHUB_ENV
    ++if test -n "$GITHUB_ENV"
    ++then
    ++	echo T="$tests" >>$GITHUB_ENV
    ++fi
23:  1d95088da29 = 22:  1b077665352 CI: only invoke ci/lib.sh as "steps" in main.yml
24:  a7434b215db ! 23:  1867da7eddb CI: narrow down variable definitions in --build and --test
    @@ Commit message
         fewer, which makes diagnosing anything going on there easier, as we
         won't have to look at a deluge of e.g. GIT_TEST_* variables.
     
    +    For the "pedantic" job (which has no test phase) we won't run the
    +    "ci/lib.sh --test" step, which will be clearly visible as a skipped
    +    step in the UX.
    +
         Since we'll now always run "--build" for "make" and "--test" for "make
         test" we can stop setting the "fat" MAKEFLAGS entirely on the "test"
         steps, i.e. the one with PYTHON_PATH, CC etc. These will all be
    @@ .github/workflows/main.yml: jobs:
     +    - run: ci/lib.sh --build
          - run: make
     +    - run: ci/lib.sh --test
    ++      if: success() && matrix.vector.skip-tests != 'yes'
          - run: make test
    -       if: success() && matrix.vector.skip-tests != 'no'
    +       if: success() && matrix.vector.skip-tests != 'yes'
          - run: ci/print-test-failures.sh
     @@ .github/workflows/main.yml: jobs:
          steps:
    @@ .github/workflows/main.yml: jobs:
     
      ## ci/lib.sh ##
     @@ ci/lib.sh: set -ex
    + # Helper libraries
      . ${0%/*}/lib-ci-type.sh
      
    ++# Parse options
    ++mode_build=
    ++mode_test=
    ++while test $# != 0
    ++do
    ++	case "$1" in
    ++	--build)
    ++		mode_build=t
    ++		;;
    ++	--test)
    ++		mode_test=t
    ++		;;
    ++	-*)
    ++		echo "error: invalid option: $1" >&2
    ++		exit 1
    ++		;;
    ++	*)
    ++		echo "error: invalid argument: $1" >&2
    ++		exit 1
    ++		;;
    ++	esac
    ++	shift
    ++done
    ++
      # Starting assertions
    -+mode=$1
    -+if test -z "$mode"
    + if test -z "$jobname"
    + then
    +@@ ci/lib.sh: then
    + 	exit 1
    + fi
    + 
    ++if test "$mode_test$mode_build" != "t"
     +then
    -+	echo "need a $0 mode, e.g. --build or --test" >&2
    ++	echo "error: need one mode, e.g. --build or --test" >&2
     +	exit 1
     +fi
     +
    - if test -z "$jobname"
    - then
    - 	echo "must set a CI jobname in the environment" >&2
    -@@ ci/lib.sh: setenv () {
    + # Helper functions
    + setenv () {
      	while test $# != 0
      	do
      		case "$1" in
    --		--build)
    --			;;
    --		--test)
    + 		--build)
    ++			if test -z "$mode_build"
    ++			then
    ++				return 0
    ++			fi
    + 			;;
    + 		--test)
     -			;;
     -		--all)
    -+		--build | --test)
    -+			if test "$1" != "$mode"
    ++			if test -z "$mode_test"
     +			then
     +				return 0
     +			fi
    -+
      			;;
      		-*)
      			echo "BUG: bad setenv() option '$1'" >&2
25:  78f4609a9c1 = 24:  a4e3a9d2c28 CI: add more variables to MAKEFLAGS, except under vs-build
26:  26a34f1d4b9 ! 25:  48d99cee809 CI: set CC in MAKEFLAGS directly, don't add it to the environment
    @@ Commit message
         "$jobname" to select packages & config, 2021-11-23), i.e. we're now
         aiming to only set those variables specific jobs need.
     
    +    Since we don't need to make this generic enough to handle "gcc-9"
    +    turning into "gcc@9" for "brew install" let's remove what was
    +    originally a bash-specific replacement in 707d2f2fe86 (CI: use
    +    "$runs_on_pool", not "$jobname" to select packages & config,
    +    2021-11-23), and which a preceding commit changed to a "tr"
    +    invocation.
    +
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## .github/workflows/main.yml ##
    @@ ci/install-dependencies.sh: UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl
       libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl"
      
     +CC_PACKAGE=
    ++BREW_CC_PACKAGE=
     +case "$jobname" in
     +linux-gcc | linux-TEST-vars)
     +	CC_PACKAGE=gcc-8
     +	;;
     +osx-gcc)
    -+	CC_PACKAGE=gcc-9
    ++	BREW_CC_PACKAGE=gcc@9
     +	;;
     +esac
     +
      case "$runs_on_pool" in
      ubuntu-latest)
      	# The Linux build installs the defined dependency versions below.
    +@@ ci/install-dependencies.sh: macos-latest)
    + 	} ||
    + 	brew install homebrew/cask/perforce
    + 
    +-	if test -n "$CC_PACKAGE"
    ++	if test -n "$BREW_CC_PACKAGE"
    + 	then
    +-		BREW_PACKAGE=$(echo $CC_PACKAGE | tr '-' '@')
    +-		brew install "$BREW_PACKAGE"
    +-		brew link "$BREW_PACKAGE"
    ++		brew install "$BREW_CC_PACKAGE"
    ++		brew link "$BREW_CC_PACKAGE"
    + 	fi
    + 	;;
    + esac
     
      ## ci/lib.sh ##
     @@ ci/lib.sh: setenv () {
27:  514de8d16b0 = 26:  07a6c087297 CI: set SANITIZE=leak in MAKEFLAGS directly
28:  6501059c594 = 27:  50f21ffdffe CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case
29:  1bdcf1399d9 ! 28:  e35dff2bef3 CI: don't use "set -x" in "ci/lib.sh" output
    @@ Commit message
         script doesn't "set -x" and will act differently depending on the
         $CI_TYPE.
     
    +    It's useful when developing this script to see what variables
    +    it *doesn't* set, but such output isn't useful to users, so let's hide
    +    the "SKIP" output behind a --debug option.
    +
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## ci/lib.sh ##
    @@ ci/lib.sh
      
      # Helper libraries
      . ${0%/*}/lib-ci-type.sh
    +@@ ci/lib.sh: set -ex
    + # Parse options
    + mode_build=
    + mode_test=
    ++mode_debug=
    + while test $# != 0
    + do
    + 	case "$1" in
    + 	--build)
    ++		echo "MODE: $1" >&2
    + 		mode_build=t
    + 		;;
    + 	--test)
    ++		echo "MODE: $1" >&2
    + 		mode_test=t
    + 		;;
    ++	--debug)
    ++		echo "DEBUG: $1" >&2
    ++		mode_debug=t
    ++		;;
    + 	-*)
    + 		echo "error: invalid option: $1" >&2
    + 		exit 1
     @@ ci/lib.sh: then
      	exit 1
      fi
    @@ ci/lib.sh: then
     +echo "CONFIG: CI_TYPE=$CI_TYPE" >&2
     +echo "CONFIG: jobname=$jobname" >&2
     +echo "CONFIG: runs_on_pool=$runs_on_pool" >&2
    -+echo "CONFIG: GITHUB_ENV=$GITHUB_ENV" >&2
     +if test -n "$GITHUB_ENV"
     +then
     +	echo "CONFIG: GITHUB_ENV=$GITHUB_ENV" >&2
     +fi
    ++echo >&2
     +
      # Helper functions
      setenv () {
    -+	local skip=
    ++	skip=
      	while test $# != 0
      	do
      		case "$1" in
    - 		--build | --test)
    - 			if test "$1" != "$mode"
    + 		--build)
    + 			if test -z "$mode_build"
    + 			then
    +-				return 0
    ++				skip=t
    + 			fi
    + 			;;
    + 		--test)
    + 			if test -z "$mode_test"
      			then
     -				return 0
     +				skip=t
      			fi
    - 
      			;;
    + 		-*)
     @@ ci/lib.sh: setenv () {
      	val=$2
      	shift 2
      
     +	if test -n "$skip"
     +	then
    -+		echo "SKIP '$key=$val'" >&2
    ++		if test -n "$mode_debug"
    ++		then
    ++			echo "SKIP: '$key=$val'" >&2
    ++		fi
     +		return 0
     +	fi
     +
30:  94abb826627 <  -:  ----------- CI: have osx-gcc use gcc, not clang
31:  4fc67e668da ! 29:  13d8c6997c3 CI: make it easy to use ci/*.sh outside of CI
    @@ Commit message
         Using "ci/lib.sh" as a stand-alone script is much more useful if it
         doesn't hardcode NPROC=10, let's provide a poor shellscript
         replacement for the online_cpus() we have in thread-utils.c to cover
    -    the most common OS's. It was suggested to use "2>&1" to invoke
    -    "command -v", but per my reading of [2] and my own testing that
    -    doesn't seem to be needed. Perhaps it's only needed for "which(1)"?
    -
    -    1. https://lore.kernel.org/git/214f8670-91d5-f4b6-efa1-76966c3ab1ee@sunshineco.com/
    -    2. https://pubs.opengroup.org/onlinepubs/009604499/utilities/command.html
    +    the most common OS's.
     
         Helped-by: Eric Sunshine <sunshine@sunshineco.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    @@ ci/lib-online_cpus.sh (new)
     +#!/bin/sh
     +
     +# TODO: Ideally we'd compile t/helper/test-online-cpus.c early, but
    -+# that presents a chicken & egg problem. But if we move it to a
    -+# stand-oline command...
    ++# that currently presents a chicken & egg problem. We need this before
    ++# we build (much of) anything.
     +online_cpus() {
     +	NPROC=
     +
    @@ ci/lib.sh
      #!/bin/sh
      set -e
      
    -+#  Usage
    ++# Usage
     +CI_TYPE_HELP_COMMANDS='
     +	# run "make all test" like the "linux-leaks" job
     +	(eval $(jobname=linux-leaks ci/lib.sh --all) && make test)
    @@ ci/lib.sh
     +'
     +
     +CI_TYPE_HELP="
    -+running $0 outside of CI? You can use ci/lib.sh to set up your
    ++Running $0 outside of CI? You can use ci/lib.sh to set up your
     +environment like a given CI job. E.g.:
     +$CI_TYPE_HELP_COMMANDS
    -+
    -+note that some of these (e.g. the linux-musl one) may not work as
    ++Note that some of these (e.g. the linux-musl one) may not work as
     +expected due to the CI job configuring a platform that may not match
     +yours."
     +
    - # Helper libraries
    - . ${0%/*}/lib-ci-type.sh
    - 
    -@@ ci/lib.sh: mode=$1
    - if test -z "$mode"
    - then
    - 	echo "need a $0 mode, e.g. --build or --test" >&2
    ++usage() {
    ++	echo "usage: jobname=<job-name> [runs_on_pool=<pool-name>] $0 (--build | --test)" >&2
     +	if test -z "$CI_TYPE"
     +	then
     +		echo "$CI_TYPE_HELP" >&2
     +	fi
    - 	exit 1
    ++	exit 1
    ++}
    ++
    + # Helper libraries
    + . ${0%/*}/lib-ci-type.sh
    + 
    + # Parse options
    ++mode_all=
    + mode_build=
    + mode_test=
    + mode_debug=
    + while test $# != 0
    + do
    + 	case "$1" in
    ++	--all)
    ++		echo "MODE: $1" >&2
    ++		mode_all=t
    ++		mode_build=t
    ++		mode_test=t
    ++		;;
    + 	--build)
    + 		echo "MODE: $1" >&2
    + 		mode_build=t
    +@@ ci/lib.sh: do
    + 		;;
    + 	-*)
    + 		echo "error: invalid option: $1" >&2
    +-		exit 1
    ++		usage
    + 		;;
    + 	*)
    + 		echo "error: invalid argument: $1" >&2
    +-		exit 1
    ++		usage
    + 		;;
    + 	esac
    + 	shift
    +@@ ci/lib.sh: done
    + if test -z "$jobname"
    + then
    + 	echo "error: must set a CI jobname in the environment" >&2
    +-	exit 1
    ++	usage
      fi
      
    +-if test "$mode_test$mode_build" != "t"
    ++if test "$mode_test$mode_build" != "t" && test -z "$mode_all"
    + then
    + 	echo "error: need one mode, e.g. --build or --test" >&2
    +-	exit 1
    ++	usage
    + fi
    + 
    + # Show our configuration
     @@ ci/lib.sh: setenv () {
      	do
      		case "$1" in
    - 		--build | --test)
    --			if test "$1" != "$mode"
    -+			if test "$1" != "$mode" && test "$mode" != "--all"
    + 		--build)
    +-			if test -z "$mode_build"
    ++			if test -z "$mode_build$mode_all"
    + 			then
    + 				skip=t
    + 			fi
    + 			;;
    + 		--test)
    +-			if test -z "$mode_test"
    ++			if test -z "$mode_test$mode_all"
      			then
      				skip=t
      			fi
-- 
2.36.0.879.g3659959fcca

[PATCH v5 01/29] CI: run "set -ex" early in ci/lib.sh

[Ævar Arnfjörð Bjarmason]

Change ci/lib.sh to run "set -ex" before it does anything else, before
this we were omitting a few "export" commands in our trace output that
happened before we enabled tracing.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index cbc2f8f1caa..57141d38a85 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,5 +1,11 @@
 # Library of functions shared by all CI scripts
 
+# Set 'exit on error' for all CI scripts to let the caller know that
+# something went wrong.
+# Set tracing executed commands, primarily setting environment variables
+# and installing dependencies.
+set -ex
+
 skip_branch_tip_with_tag () {
 	# Sometimes, a branch is pushed at the same time the tag that points
 	# at the same commit as the tip of the branch is pushed, and building
@@ -85,12 +91,6 @@ export TERM=${TERM:-dumb}
 # Clear MAKEFLAGS that may come from the outside world.
 export MAKEFLAGS=
 
-# Set 'exit on error' for all CI scripts to let the caller know that
-# something went wrong.
-# Set tracing executed commands, primarily setting environment variables
-# and installing dependencies.
-set -ex
-
 if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
 then
 	CI_TYPE=azure-pipelines
-- 
2.36.0.879.g3659959fcca

[PATCH v5 02/29] CI: make "$jobname" explicit, remove fallback

[Ævar Arnfjörð Bjarmason]

Remove the $jobname fallback last adjusted in b011fabd6e0 (ci/lib.sh:
encapsulate Travis-specific things, 2019-01-27).

This makes all of our CI jobs have an explicit $jobname, and removes
the special-case of the Windows jobs implicitly setting one (that they
don't currently use). In subsequent commits we'll use this new
$jobname explicitly.

This is a follow-up to my 25715419bf4 (CI: don't run "make test" twice
in one job, 2021-11-23), since then all the jobs except the Windows
jobs have had an explicit $jobname.

In the case of the Windows jobs we were using a jobname of windows-gcc
for both, due to picking "gcc" as a default fallback for "$CC" in
ci/lib.sh.

Since we'll need this to run ci/lib.sh it makes sense to move the
check to the top of the file. Subsequent commits will add more such
assertions to it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml |  8 ++++++++
 ci/lib.sh                  | 12 +++++++-----
 2 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index c35200defb9..309b9141249 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -79,6 +79,8 @@ jobs:
 
   windows-build:
     name: win build
+    env:
+      jobname: windows-build
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
     runs-on: windows-latest
@@ -100,6 +102,8 @@ jobs:
         path: artifacts
   windows-test:
     name: win test
+    env:
+      jobname: windows-test
     runs-on: windows-latest
     needs: [windows-build]
     strategy:
@@ -131,6 +135,8 @@ jobs:
         path: ${{env.FAILED_TEST_ARTIFACTS}}
   vs-build:
     name: win+VS build
+    env:
+      jobname: vs-build
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
     env:
@@ -183,6 +189,8 @@ jobs:
         path: artifacts
   vs-test:
     name: win+VS test
+    env:
+      jobname: vs-test
     runs-on: windows-latest
     needs: vs-build
     strategy:
diff --git a/ci/lib.sh b/ci/lib.sh
index 57141d38a85..6d76c2d406b 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -6,6 +6,13 @@
 # and installing dependencies.
 set -ex
 
+# Starting assertions
+if test -z "$jobname"
+then
+	echo "error: must set a CI jobname in the environment" >&2
+	exit 1
+fi
+
 skip_branch_tip_with_tag () {
 	# Sometimes, a branch is pushed at the same time the tag that points
 	# at the same commit as the tip of the branch is pushed, and building
@@ -146,11 +153,6 @@ test -n "${DONT_SKIP_TAGS-}" ||
 skip_branch_tip_with_tag
 skip_good_tree
 
-if test -z "$jobname"
-then
-	jobname="$CI_OS_NAME-$CC"
-fi
-
 export DEVELOPER=1
 export DEFAULT_TEST_TARGET=prove
 export GIT_TEST_CLONE_2GB=true
-- 
2.36.0.879.g3659959fcca

[PATCH v5 03/29] CI: remove more dead Travis CI support

[Ævar Arnfjörð Bjarmason]

Remove code that I missed in 4a6e4b96026 (CI: remove Travis CI
support, 2021-11-23). This code was only called from or used by the
now-removed .travis.yml, or needed by the Travis CI environment.

For the symlinking in ci/run-build-and-tests.sh: Back when
3c93b829205 (travis-ci: build Git during the 'script' phase,
2018-01-08) and 4b060a4d973 (ci: use a junction on Windows instead of
a symlink, 2019-01-27) adjusted this "ln" command, the Windows build
would use ci/run-build-and-tests.sh.

As seen in 889cacb6897 (ci: configure GitHub Actions for CI/PR,
2020-04-11) the current windows build uses a different entry point
under the GitHub CI, which doesn't use this .prove caching. Namely
"ci/run-test-slice.sh".

We can be certain that it's never used in "ci/run-test-slice.sh"
because to have a ".prove" file we'd need to use "--state=save", which
we only do in the dead Azure codepath in ci/lib.sh. If it were used it
would do the wrong thing, because the different test slices would each
try to clobber the same "t/.prove" file.

If a subsequent run then used the "--state=failed,slow,save" it would
defeat the purpose of "ci/run-test-slice.sh", since all slices would
then run all tests. I.e. behavior of prove's "--state" options is to
select tests to run from the provided "--state" file, in addition to
those specified on the command-line.

For ci/run-docker{,-build}.sh: It was likewise last referenced in the
.travis.yml removed in my 4a6e4b96026. The current "dockerized" run in
".github/workflows/main.yml" calls the same entry points as the main
"regular" job.

For "ci/print-test-failures.sh" the creation of the tarball added in
aea8879a6ac (travis-ci: include the trash directories of failed tests
in the trace log, 2018-08-01) has been dead since my 4a6e4b96026,
which removed the fall-through case from the "case/esac" statement.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                     | 10 ++----
 ci/print-test-failures.sh     | 39 ---------------------
 ci/run-build-and-tests.sh     |  5 ---
 ci/run-docker-build.sh        | 66 -----------------------------------
 ci/run-docker.sh              | 47 -------------------------
 ci/run-test-slice.sh          |  5 ---
 ci/util/extract-trash-dirs.sh | 50 --------------------------
 7 files changed, 2 insertions(+), 220 deletions(-)
 delete mode 100755 ci/run-docker-build.sh
 delete mode 100755 ci/run-docker.sh
 delete mode 100755 ci/util/extract-trash-dirs.sh

diff --git a/ci/lib.sh b/ci/lib.sh
index 6d76c2d406b..648940da877 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -106,9 +106,6 @@ then
 	CI_COMMIT="$BUILD_SOURCEVERSION"
 	CI_JOB_ID="$BUILD_BUILDID"
 	CI_JOB_NUMBER="$BUILD_BUILDNUMBER"
-	CI_OS_NAME="$(echo "$AGENT_OS" | tr A-Z a-z)"
-	test darwin != "$CI_OS_NAME" || CI_OS_NAME=osx
-	CI_REPO_SLUG="$(expr "$BUILD_REPOSITORY_URI" : '.*/\([^/]*/[^/]*\)$')"
 	CC="${CC:-gcc}"
 
 	# use a subdirectory of the cache dir (because the file share is shared
@@ -118,16 +115,13 @@ then
 	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
-	test windows_nt != "$CI_OS_NAME" ||
+	test Windows_NT != "$AGENT_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 elif test true = "$GITHUB_ACTIONS"
 then
 	CI_TYPE=github-actions
 	CI_BRANCH="$GITHUB_REF"
 	CI_COMMIT="$GITHUB_SHA"
-	CI_OS_NAME="$(echo "$RUNNER_OS" | tr A-Z a-z)"
-	test macos != "$CI_OS_NAME" || CI_OS_NAME=osx
-	CI_REPO_SLUG="$GITHUB_REPOSITORY"
 	CI_JOB_ID="$GITHUB_RUN_ID"
 	CC="${CC:-gcc}"
 	DONT_SKIP_TAGS=t
@@ -137,7 +131,7 @@ then
 	export GIT_PROVE_OPTS="--timer --jobs 10"
 	export GIT_TEST_OPTS="--verbose-log -x"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
-	test windows != "$CI_OS_NAME" ||
+	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 else
 	echo "Could not identify CI type" >&2
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 57277eefcd0..740ebb7c529 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -16,15 +16,6 @@ then
 	exit
 fi
 
-case "$jobname" in
-osx-clang|osx-gcc)
-	# base64 in OSX doesn't wrap its output at 76 columns by
-	# default, but prints a single, very long line.
-	base64_opts="-b 76"
-	;;
-esac
-
-combined_trash_size=0
 for TEST_EXIT in test-results/*.exit
 do
 	if [ "$(cat "$TEST_EXIT")" != "0" ]
@@ -42,47 +33,17 @@ do
 		azure-pipelines)
 			mkdir -p failed-test-artifacts
 			mv "$trash_dir" failed-test-artifacts
-			continue
 			;;
 		github-actions)
 			mkdir -p failed-test-artifacts
 			echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
 			cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
 			tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
-			continue
 			;;
 		*)
 			echo "Unhandled CI type: $CI_TYPE" >&2
 			exit 1
 			;;
 		esac
-		trash_tgz_b64="trash.$test_name.base64"
-		if [ -d "$trash_dir" ]
-		then
-			tar czp "$trash_dir" |base64 $base64_opts >"$trash_tgz_b64"
-
-			trash_size=$(wc -c <"$trash_tgz_b64")
-			if [ $trash_size -gt 1048576 ]
-			then
-				# larger than 1MB
-				echo "$(tput setaf 1)Didn't include the trash directory of '$test_name' in the trace log, it's too big$(tput sgr0)"
-				continue
-			fi
-
-			new_combined_trash_size=$(($combined_trash_size + $trash_size))
-			if [ $new_combined_trash_size -gt 1048576 ]
-			then
-				echo "$(tput setaf 1)Didn't include the trash directory of '$test_name' in the trace log, there is plenty of trash in there already.$(tput sgr0)"
-				continue
-			fi
-			combined_trash_size=$new_combined_trash_size
-
-			# DO NOT modify these two 'echo'-ed strings below
-			# without updating 'ci/util/extract-trash-dirs.sh'
-			# as well.
-			echo "$(tput setaf 1)Start of trash directory of '$test_name':$(tput sgr0)"
-			cat "$trash_tgz_b64"
-			echo "$(tput setaf 1)End of trash directory of '$test_name'$(tput sgr0)"
-		fi
 	fi
 done
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 280dda7d285..0a1ec8c2bae 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -5,11 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-case "$CI_OS_NAME" in
-windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
-*) ln -s "$cache_dir/.prove" t/.prove;;
-esac
-
 export MAKE_TARGETS="all test"
 
 case "$jobname" in
diff --git a/ci/run-docker-build.sh b/ci/run-docker-build.sh
deleted file mode 100755
index 6cd832efb9c..00000000000
--- a/ci/run-docker-build.sh
+++ /dev/null
@@ -1,66 +0,0 @@
-#!/bin/sh
-#
-# Build and test Git inside container
-#
-# Usage:
-#   run-docker-build.sh <host-user-id>
-#
-
-set -ex
-
-if test $# -ne 1 || test -z "$1"
-then
-	echo >&2 "usage: run-docker-build.sh <host-user-id>"
-	exit 1
-fi
-
-case "$jobname" in
-linux32)
-	switch_cmd="linux32 --32bit i386"
-	;;
-linux-musl)
-	switch_cmd=
-	useradd () { adduser -D "$@"; }
-	;;
-*)
-	exit 1
-	;;
-esac
-
-"${0%/*}/install-docker-dependencies.sh"
-
-# If this script runs inside a docker container, then all commands are
-# usually executed as root. Consequently, the host user might not be
-# able to access the test output files.
-# If a non 0 host user id is given, then create a user "ci" with that
-# user id to make everything accessible to the host user.
-HOST_UID=$1
-if test $HOST_UID -eq 0
-then
-	# Just in case someone does want to run the test suite as root.
-	CI_USER=root
-else
-	CI_USER=ci
-	if test "$(id -u $CI_USER 2>/dev/null)" = $HOST_UID
-	then
-		echo "user '$CI_USER' already exists with the requested ID $HOST_UID"
-	else
-		useradd -u $HOST_UID $CI_USER
-	fi
-fi
-
-# Build and test
-command $switch_cmd su -m -l $CI_USER -c "
-	set -ex
-	export DEVELOPER='$DEVELOPER'
-	export DEFAULT_TEST_TARGET='$DEFAULT_TEST_TARGET'
-	export GIT_PROVE_OPTS='$GIT_PROVE_OPTS'
-	export GIT_TEST_OPTS='$GIT_TEST_OPTS'
-	export GIT_TEST_CLONE_2GB='$GIT_TEST_CLONE_2GB'
-	export MAKEFLAGS='$MAKEFLAGS'
-	export cache_dir='$cache_dir'
-	cd /usr/src/git
-	test -n '$cache_dir' && ln -s '$cache_dir/.prove' t/.prove
-	make
-	make test
-"
diff --git a/ci/run-docker.sh b/ci/run-docker.sh
deleted file mode 100755
index af89d1624a4..00000000000
--- a/ci/run-docker.sh
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/bin/sh
-#
-# Download and run Docker image to build and test Git
-#
-
-. ${0%/*}/lib.sh
-
-case "$jobname" in
-linux32)
-	CI_CONTAINER="daald/ubuntu32:xenial"
-	;;
-linux-musl)
-	CI_CONTAINER=alpine
-	;;
-*)
-	exit 1
-	;;
-esac
-
-docker pull "$CI_CONTAINER"
-
-# Use the following command to debug the docker build locally:
-# <host-user-id> must be 0 if podman is used as drop-in replacement for docker
-# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/sh "$CI_CONTAINER"
-# root@container:/# export jobname=<jobname>
-# root@container:/# /usr/src/git/ci/run-docker-build.sh <host-user-id>
-
-container_cache_dir=/tmp/container-cache
-
-docker run \
-	--interactive \
-	--env DEVELOPER \
-	--env DEFAULT_TEST_TARGET \
-	--env GIT_PROVE_OPTS \
-	--env GIT_TEST_OPTS \
-	--env GIT_TEST_CLONE_2GB \
-	--env MAKEFLAGS \
-	--env jobname \
-	--env cache_dir="$container_cache_dir" \
-	--volume "${PWD}:/usr/src/git" \
-	--volume "$cache_dir:$container_cache_dir" \
-	"$CI_CONTAINER" \
-	/usr/src/git/ci/run-docker-build.sh $(id -u $USER)
-
-check_unignored_build_artifacts
-
-save_good_tree
diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index f8c2c3106a2..b9a682b4bcd 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -5,11 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-case "$CI_OS_NAME" in
-windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
-*) ln -s "$cache_dir/.prove" t/.prove;;
-esac
-
 make --quiet -C t T="$(cd t &&
 	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
 	tr '\n' ' ')"
diff --git a/ci/util/extract-trash-dirs.sh b/ci/util/extract-trash-dirs.sh
deleted file mode 100755
index 8e67bec21a2..00000000000
--- a/ci/util/extract-trash-dirs.sh
+++ /dev/null
@@ -1,50 +0,0 @@
-#!/bin/sh
-
-error () {
-	echo >&2 "error: $@"
-	exit 1
-}
-
-find_embedded_trash () {
-	while read -r line
-	do
-		case "$line" in
-		*Start\ of\ trash\ directory\ of\ \'t[0-9][0-9][0-9][0-9]-*\':*)
-			test_name="${line#*\'}"
-			test_name="${test_name%\'*}"
-
-			return 0
-		esac
-	done
-
-	return 1
-}
-
-extract_embedded_trash () {
-	while read -r line
-	do
-		case "$line" in
-		*End\ of\ trash\ directory\ of\ \'$test_name\'*)
-			return
-			;;
-		*)
-			printf '%s\n' "$line"
-			;;
-		esac
-	done
-
-	error "unexpected end of input"
-}
-
-# Raw logs from Linux build jobs have CRLF line endings, while OSX
-# build jobs mostly have CRCRLF, except an odd line every now and
-# then that has CRCRCRLF.  'base64 -d' from 'coreutils' doesn't like
-# CRs and complains about "invalid input", so remove all CRs at the
-# end of lines.
-sed -e 's/\r*$//' | \
-while find_embedded_trash
-do
-	echo "Extracting trash directory of '$test_name'"
-
-	extract_embedded_trash |base64 -d |tar xzp
-done
-- 
2.36.0.879.g3659959fcca

[PATCH v5 04/29] CI: remove dead "tree skipping" code

[Ævar Arnfjörð Bjarmason]

Remove the code related to skipping CI runs if the tree was
successfully tested before. With the GitHub CI this is done with the
"ci-config" step, see e76eec35540 (ci: allow per-branch config for
GitHub Actions, 2020-05-07) and 7d78d5fc1a9 (ci: skip GitHub workflow
runs for already-tested commits/trees, 2020-10-08).

This code hasn't been used since 4a6e4b96026 (CI: remove Travis CI
support, 2021-11-23), and before that for the retired Azure pipeline
support removed in 6081d3898fe (ci: retire the Azure Pipelines
definition, 2020-04-11).

This change is needed because this and subsequent commits are turning
"ci/lib.sh" into a dumber library that'll only be tasked with setting
variables for CI jobs, or for specific steps within those jobs. Now we
don't need to worry about it potentially skipping the run on its own.

This change also removes a subtle potential logic error introduced in
0e7696c64db (ci: disallow directional formatting, 2021-11-04). The
"ci/check-directional-formatting.bash" script would have been made to
run after the "save_good_tree" in invoked in
"ci/run-static-analysis.sh". So if e.g. the "make coccicheck" failed
we'd still mark the tree as good.

I.e. the addition of "ci/check-directional-formatting.bash" didn't
take into account that the various users of the "save_good_tree"
function in ci/*.sh made a hard assumption that they're the only
ci/*.sh script that's being run (and they use "set -e").

This is entirely academic since we weren't actually running this code,
but is something to be careful of if anyone ever needs to resurrect
parts of this.

Finally, the cache_dir="$HOME/none" added in a3f2eec862b (ci/lib:
allow running in GitHub Actions, 2020-04-08) for the GitHub CI can be
removed. It appears to have been added to appease the subsequent
'mkdir -p "$cache_dir"', which wasn't needed in GitHub CI.

1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                 | 91 ---------------------------------------
 ci/run-build-and-tests.sh |  2 -
 ci/run-static-analysis.sh |  2 -
 ci/test-documentation.sh  |  2 -
 4 files changed, 97 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 648940da877..a30ab35d9c2 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -13,75 +13,6 @@ then
 	exit 1
 fi
 
-skip_branch_tip_with_tag () {
-	# Sometimes, a branch is pushed at the same time the tag that points
-	# at the same commit as the tip of the branch is pushed, and building
-	# both at the same time is a waste.
-	#
-	# When the build is triggered by a push to a tag, $CI_BRANCH will
-	# have that tagname, e.g. v2.14.0.  Let's see if $CI_BRANCH is
-	# exactly at a tag, and if so, if it is different from $CI_BRANCH.
-	# That way, we can tell if we are building the tip of a branch that
-	# is tagged and we can skip the build because we won't be skipping a
-	# build of a tag.
-
-	if TAG=$(git describe --exact-match "$CI_BRANCH" 2>/dev/null) &&
-		test "$TAG" != "$CI_BRANCH"
-	then
-		echo "$(tput setaf 2)Tip of $CI_BRANCH is exactly at $TAG$(tput sgr0)"
-		exit 0
-	fi
-}
-
-# Save some info about the current commit's tree, so we can skip the build
-# job if we encounter the same tree again and can provide a useful info
-# message.
-save_good_tree () {
-	echo "$(git rev-parse $CI_COMMIT^{tree}) $CI_COMMIT $CI_JOB_NUMBER $CI_JOB_ID" >>"$good_trees_file"
-	# limit the file size
-	tail -1000 "$good_trees_file" >"$good_trees_file".tmp
-	mv "$good_trees_file".tmp "$good_trees_file"
-}
-
-# Skip the build job if the same tree has already been built and tested
-# successfully before (e.g. because the branch got rebased, changing only
-# the commit messages).
-skip_good_tree () {
-	if test true = "$GITHUB_ACTIONS"
-	then
-		return
-	fi
-
-	if ! good_tree_info="$(grep "^$(git rev-parse $CI_COMMIT^{tree}) " "$good_trees_file")"
-	then
-		# Haven't seen this tree yet, or no cached good trees file yet.
-		# Continue the build job.
-		return
-	fi
-
-	echo "$good_tree_info" | {
-		read tree prev_good_commit prev_good_job_number prev_good_job_id
-
-		if test "$CI_JOB_ID" = "$prev_good_job_id"
-		then
-			cat <<-EOF
-			$(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0)
-			This commit has already been built and tested successfully by this build job.
-			To force a re-build delete the branch's cache and then hit 'Restart job'.
-			EOF
-		else
-			cat <<-EOF
-			$(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0)
-			This commit's tree has already been built and tested successfully in build job $prev_good_job_number for commit $prev_good_commit.
-			The log of that build job is available at $SYSTEM_TASKDEFINITIONSURI$SYSTEM_TEAMPROJECT/_build/results?buildId=$prev_good_job_id
-			To force a re-build delete the branch's cache and then hit 'Restart job'.
-			EOF
-		fi
-	}
-
-	exit 0
-}
-
 check_unignored_build_artifacts ()
 {
 	! git ls-files --other --exclude-standard --error-unmatch \
@@ -102,16 +33,8 @@ if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
 then
 	CI_TYPE=azure-pipelines
 	# We are running in Azure Pipelines
-	CI_BRANCH="$BUILD_SOURCEBRANCH"
-	CI_COMMIT="$BUILD_SOURCEVERSION"
-	CI_JOB_ID="$BUILD_BUILDID"
-	CI_JOB_NUMBER="$BUILD_BUILDNUMBER"
 	CC="${CC:-gcc}"
 
-	# use a subdirectory of the cache dir (because the file share is shared
-	# among *all* phases)
-	cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME"
-
 	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
@@ -120,13 +43,7 @@ then
 elif test true = "$GITHUB_ACTIONS"
 then
 	CI_TYPE=github-actions
-	CI_BRANCH="$GITHUB_REF"
-	CI_COMMIT="$GITHUB_SHA"
-	CI_JOB_ID="$GITHUB_RUN_ID"
 	CC="${CC:-gcc}"
-	DONT_SKIP_TAGS=t
-
-	cache_dir="$HOME/none"
 
 	export GIT_PROVE_OPTS="--timer --jobs 10"
 	export GIT_TEST_OPTS="--verbose-log -x"
@@ -139,14 +56,6 @@ else
 	exit 1
 fi
 
-good_trees_file="$cache_dir/good-trees"
-
-mkdir -p "$cache_dir"
-
-test -n "${DONT_SKIP_TAGS-}" ||
-skip_branch_tip_with_tag
-skip_good_tree
-
 export DEVELOPER=1
 export DEFAULT_TEST_TARGET=prove
 export GIT_TEST_CLONE_2GB=true
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 0a1ec8c2bae..e5a21985b44 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -45,5 +45,3 @@ esac
 # start running tests.
 make $MAKE_TARGETS
 check_unignored_build_artifacts
-
-save_good_tree
diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh
index 65bcebda41a..5a87b1cac96 100755
--- a/ci/run-static-analysis.sh
+++ b/ci/run-static-analysis.sh
@@ -28,5 +28,3 @@ fi
 
 make hdr-check ||
 exit 1
-
-save_good_tree
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index de41888430a..9e0652c30dd 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -41,5 +41,3 @@ grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
 
 rm -f stdout.log stderr.log stderr.raw
 check_unignored_build_artifacts
-
-save_good_tree
-- 
2.36.0.879.g3659959fcca

[PATCH v5 05/29] CI: remove unused Azure ci/* code

[Ævar Arnfjörð Bjarmason]

Remove Azure-specific code that's been unused since 6081d3898fe (ci:
retire the Azure Pipelines definition, 2020-04-11). As noted in a
larger removal of all of the Azure-supporting code in [1] (although
that missed some of this) there is more of it in-tree, but let's focus
on only the ci/* code for now.

This is needed because in subsequent commits this unused code would
either need to be changed to accommodate changes in ci/*, or be
removed.

As we'll see in those subsequent commits the end-state we're heading
towards will actually make it easier to add new CI types in the
future, even though the only one we're left with now is the GitHub
CI. I.e. the "ci/*" framework will be made to do less, not more. We'll
be offloading more of what it does to our generic build and test
system.

While I'm at it (since the line needs to be touched anyway) change an
odd 'if test true == "$GITHUB_ACTIONS"' to the more usual style used
in other places of 'if test "$GITHUB_ACTIONS" = "true"'.

1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/install-dependencies.sh |  3 ---
 ci/lib.sh                  | 13 +------------
 ci/mount-fileshare.sh      | 25 -------------------------
 ci/print-test-failures.sh  |  4 ----
 4 files changed, 1 insertion(+), 44 deletions(-)
 delete mode 100755 ci/mount-fileshare.sh

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index dbcebad2fb2..e7ea8799411 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -34,8 +34,6 @@ macos-latest)
 	export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
 	# Uncomment this if you want to run perf tests:
 	# brew install gnu-time
-	test -z "$BREW_INSTALL_PACKAGES" ||
-	brew install $BREW_INSTALL_PACKAGES
 	brew link --force gettext
 	brew install --cask --no-quarantine perforce || {
 		# Update the definitions and try again
@@ -69,7 +67,6 @@ Documentation)
 	sudo apt-get -q update
 	sudo apt-get -q -y install asciidoc xmlto docbook-xsl-ns make
 
-	test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
 	sudo gem install --version 1.5.8 asciidoctor
 	;;
 linux-gcc-default)
diff --git a/ci/lib.sh b/ci/lib.sh
index a30ab35d9c2..a6cdcfa014b 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -29,18 +29,7 @@ export TERM=${TERM:-dumb}
 # Clear MAKEFLAGS that may come from the outside world.
 export MAKEFLAGS=
 
-if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
-then
-	CI_TYPE=azure-pipelines
-	# We are running in Azure Pipelines
-	CC="${CC:-gcc}"
-
-	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
-	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
-	MAKEFLAGS="$MAKEFLAGS --jobs=10"
-	test Windows_NT != "$AGENT_OS" ||
-	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
-elif test true = "$GITHUB_ACTIONS"
+if test "$GITHUB_ACTIONS" = "true"
 then
 	CI_TYPE=github-actions
 	CC="${CC:-gcc}"
diff --git a/ci/mount-fileshare.sh b/ci/mount-fileshare.sh
deleted file mode 100755
index 26b58a80960..00000000000
--- a/ci/mount-fileshare.sh
+++ /dev/null
@@ -1,25 +0,0 @@
-#!/bin/sh
-
-die () {
-	echo "$*" >&2
-	exit 1
-}
-
-test $# = 4 ||
-die "Usage: $0 <share> <username> <password> <mountpoint>"
-
-mkdir -p "$4" || die "Could not create $4"
-
-case "$(uname -s)" in
-Linux)
-	sudo mount -t cifs -o vers=3.0,username="$2",password="$3",dir_mode=0777,file_mode=0777,serverino "$1" "$4"
-	;;
-Darwin)
-	pass="$(echo "$3" | sed -e 's/\//%2F/g' -e 's/+/%2B/g')" &&
-	mount -t smbfs,soft "smb://$2:$pass@${1#//}" "$4"
-	;;
-*)
-	die "No support for $(uname -s)"
-	;;
-esac ||
-die "Could not mount $4"
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 740ebb7c529..47e2ea1ad10 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -30,10 +30,6 @@ do
 		test_name="${test_name##*/}"
 		trash_dir="trash directory.$test_name"
 		case "$CI_TYPE" in
-		azure-pipelines)
-			mkdir -p failed-test-artifacts
-			mv "$trash_dir" failed-test-artifacts
-			;;
 		github-actions)
 			mkdir -p failed-test-artifacts
 			echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
-- 
2.36.0.879.g3659959fcca

[PATCH v5 06/29] CI/lib.sh: stop adding leading whitespace to $MAKEFLAGS

[Ævar Arnfjörð Bjarmason]

In a8c51f77d16 (ci: clear and mark MAKEFLAGS exported just once,
2019-02-07) the assignment of MAKEFLAGS was done up-front to
future-proof this code for the bug fixed in 406f93ae488 (ci: make sure
we build Git parallel, 2019-02-07).

But since 4a6e4b96026 (CI: remove Travis CI support, 2021-11-23) the
--jobs=10 has been an invariant here, so let's assign --jobs=$NPROC to
the MAKEFLAGS right away, this gets rid of the leading whitespace from
the $MAKEFLAGS variable, as it would previously be empty on the first
assignment.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index a6cdcfa014b..edf26984aeb 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -26,17 +26,19 @@ check_unignored_build_artifacts ()
 # GitHub Action doesn't set TERM, which is required by tput
 export TERM=${TERM:-dumb}
 
+# How many jobs to run in parallel?
+NPROC=10
+
 # Clear MAKEFLAGS that may come from the outside world.
-export MAKEFLAGS=
+export MAKEFLAGS=--jobs=$NPROC
 
 if test "$GITHUB_ACTIONS" = "true"
 then
 	CI_TYPE=github-actions
 	CC="${CC:-gcc}"
 
-	export GIT_PROVE_OPTS="--timer --jobs 10"
+	export GIT_PROVE_OPTS="--timer --jobs $NPROC"
 	export GIT_TEST_OPTS="--verbose-log -x"
-	MAKEFLAGS="$MAKEFLAGS --jobs=10"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 else
-- 
2.36.0.879.g3659959fcca

[PATCH v5 07/29] CI: don't have "git grep" invoke a pager in tree content check

[Ævar Arnfjörð Bjarmason]

Fix the omission of not adding "-P" (a.k.a. "--no-pager") to the "git"
invocation that spaws "git grep", added in 0e7696c64db (ci: disallow
directional formatting, 2021-11-04).

This didn't matter for the GitHub CI, but does when running the check
locally. A subsequent commit will start invoking it via a new check
from the top-level Makefile, and without this we'd visibly spawn a
pager when doing so.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/check-directional-formatting.bash | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ci/check-directional-formatting.bash b/ci/check-directional-formatting.bash
index e6211b141aa..de8e6e486f0 100755
--- a/ci/check-directional-formatting.bash
+++ b/ci/check-directional-formatting.bash
@@ -23,5 +23,5 @@
 # U+2066..U+2069: LRI, RLI, FSI and PDI
 regex='(\u202a|\u202b|\u202c|\u202d|\u202e|\u2066|\u2067|\u2068|\u2069)'
 
-! LC_CTYPE=C git grep -El "$(LC_CTYPE=C.UTF-8 printf "$regex")" \
+! LC_CTYPE=C git -P grep -El "$(LC_CTYPE=C.UTF-8 printf "$regex")" \
 	-- ':(exclude,attr:binary)' ':(exclude)*.po'
-- 
2.36.0.879.g3659959fcca

[PATCH v5 08/29] CI: have "static-analysis" run a "make ci-static-analysis" target

[Ævar Arnfjörð Bjarmason]

Make it easier to run the CI logic locally by creating
"ci-static-analysis" target.

Now the "check-coccicheck" logic added in 0860a7641ba (travis-ci: fail
if Coccinelle static analysis found something to transform,
2018-07-23) can be easily tested outside of CI, and likewise the
sanity check added in 0e7696c64db (ci: disallow directional
formatting, 2021-11-04).

These targets could be improved, the "hdr-check" target will re-do all
of its work every time it's run, and could with a change similar to my
c234e8a0ecf (Makefile: make the "sparse" target non-.PHONY,
2021-09-23). The same goes for the new
"ci-check-directional-formatting" target.

But without those improvements being able to easily run a 1=1
equivalent of the checks we do in CI during a local build is already
an improvement. Subsequent commits will expand on this ability.

This change will also make the CI check faster, since we can take
advantage of parallelism. The "make coccicheck" takes a very long
time, and at the tail end we might only have one job pegging 100% of
one CPU core for a long time. Now any extra cores will be free to run
the rest of the targets under "ci-static-analysis".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml |  3 +--
 Makefile                   | 28 ++++++++++++++++++++++++++++
 ci/run-static-analysis.sh  | 30 ------------------------------
 shared.mak                 |  1 +
 4 files changed, 30 insertions(+), 32 deletions(-)
 delete mode 100755 ci/run-static-analysis.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 309b9141249..ad752010102 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -317,8 +317,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/run-static-analysis.sh
-    - run: ci/check-directional-formatting.bash
+    - run: make ci-static-analysis
   sparse:
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
diff --git a/Makefile b/Makefile
index f8bccfab5e9..8717ea7e745 100644
--- a/Makefile
+++ b/Makefile
@@ -3031,6 +3031,20 @@ coccicheck: $(addsuffix .patch,$(filter-out %.pending.cocci,$(wildcard contrib/c
 # See contrib/coccinelle/README
 coccicheck-pending: $(addsuffix .patch,$(wildcard contrib/coccinelle/*.pending.cocci))
 
+.PHONY: check-coccicheck
+check-coccicheck: coccicheck
+	$(QUIET_CHECK)for cocci_patch in contrib/coccinelle/*.patch; do \
+		if test -s "$$cocci_patch"; then \
+			echo "Coccinelle suggests the following changes in '$$cocci_patch':"; \
+			cat "$$cocci_patch"; \
+			fail=UnfortunatelyYes; \
+		fi \
+	done; \
+	if test -n "$$fail"; then \
+		echo "error: Coccinelle suggested some changes"; \
+		exit 1; \
+	fi
+
 .PHONY: coccicheck coccicheck-pending
 
 ### Installation rules
@@ -3466,3 +3480,17 @@ $(FUZZ_PROGRAMS): all
 		$(XDIFF_OBJS) $(EXTLIBS) git.o $@.o $(LIB_FUZZING_ENGINE) -o $@
 
 fuzz-all: $(FUZZ_PROGRAMS)
+
+### CI "check" targets
+#
+# These targets are run from the CI, see .github/workflows/main.yml,
+# but can also be run manually to run the same assertions locally.
+
+.PHONY: ci-check-directional-formatting
+ci-check-directional-formatting:
+	$(QUIET_CHECK)ci/check-directional-formatting.bash
+
+.PHONY: ci-static-analysis
+ci-static-analysis: ci-check-directional-formatting
+ci-static-analysis: check-coccicheck
+ci-static-analysis: hdr-check
diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh
deleted file mode 100755
index 5a87b1cac96..00000000000
--- a/ci/run-static-analysis.sh
+++ /dev/null
@@ -1,30 +0,0 @@
-#!/bin/sh
-#
-# Perform various static code analysis checks
-#
-
-. ${0%/*}/lib.sh
-
-make coccicheck
-
-set +x
-
-fail=
-for cocci_patch in contrib/coccinelle/*.patch
-do
-	if test -s "$cocci_patch"
-	then
-		echo "$(tput setaf 1)Coccinelle suggests the following changes in '$cocci_patch':$(tput sgr0)"
-		cat "$cocci_patch"
-		fail=UnfortunatelyYes
-	fi
-done
-
-if test -n "$fail"
-then
-	echo "$(tput setaf 1)error: Coccinelle suggested some changes$(tput sgr0)"
-	exit 1
-fi
-
-make hdr-check ||
-exit 1
diff --git a/shared.mak b/shared.mak
index 50d4596f0d6..83e0aa16e7d 100644
--- a/shared.mak
+++ b/shared.mak
@@ -68,6 +68,7 @@ ifndef V
 	QUIET_HDR      = @echo '   ' HDR $(<:hcc=h);
 	QUIET_RC       = @echo '   ' RC $@;
 	QUIET_SPATCH   = @echo '   ' SPATCH $<;
+	QUIET_CHECK    = @echo '   ' CHECK $@;
 
 ## Used in "Documentation/Makefile"
 	QUIET_ASCIIDOC	= @echo '   ' ASCIIDOC $@;
-- 
2.36.0.879.g3659959fcca

[PATCH v5 09/29] CI: have "static-analysis" run "check-builtins", not "documentation"

[Ævar Arnfjörð Bjarmason]

Move the "make check-builtins" check from the "documentation" job to
the "static-analysis" job.

The "check-builtins" target added in c74390e4a1d (cherry is built-in,
do not ship git-cherry.sh, 2006-11-05) is unrelated to the
documentation, so it's odd that b98712b9aa9 (travis-ci: build
documentation, 2016-05-04) added it to the "documentation" job.

Let's just move it to the "ci-static-analysis" target, and while we're
at it improve the output with $(QUIET_CHECK).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile                 | 3 ++-
 ci/test-documentation.sh | 1 -
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 8717ea7e745..2e1ae6c6067 100644
--- a/Makefile
+++ b/Makefile
@@ -3402,7 +3402,7 @@ check-docs::
 ### Make sure built-ins do not have dups and listed in git.c
 #
 check-builtins::
-	./check-builtins.sh
+	$(QUIET_CHECK)./check-builtins.sh
 
 ### Test suite coverage testing
 #
@@ -3492,5 +3492,6 @@ ci-check-directional-formatting:
 
 .PHONY: ci-static-analysis
 ci-static-analysis: ci-check-directional-formatting
+ci-static-analysis: check-builtins
 ci-static-analysis: check-coccicheck
 ci-static-analysis: hdr-check
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 9e0652c30dd..41e2b126311 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -14,7 +14,6 @@ filter_log () {
 	    "$1"
 }
 
-make check-builtins
 make check-docs
 
 # Build docs with AsciiDoc
-- 
2.36.0.879.g3659959fcca

[PATCH v5 10/29] CI: move p4 and git-lfs variables to ci/install-dependencies.sh

[Ævar Arnfjörð Bjarmason]

Move the declaration of variables that are only used by the
"ubuntu-latest" block in "ci/install-dependencies.sh" there from
"ci/lib.sh".

This makes the code easier to follow, and changes "ci/lib.sh" to a
library that only exports CI variables for general use, we didn't need
to export these $P4_PATH and $GIT_LFS_PATH variables.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/install-dependencies.sh | 17 +++++++++++++++--
 ci/lib.sh                  | 12 ------------
 2 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index e7ea8799411..34b4400f38b 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -5,14 +5,27 @@
 
 . ${0%/*}/lib.sh
 
-P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
-LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
 UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
  tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
  libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl"
 
 case "$runs_on_pool" in
 ubuntu-latest)
+	# The Linux build installs the defined dependency versions below.
+	# The OS X build installs much more recent versions, whichever
+	# were recorded in the Homebrew database upon creating the OS X
+	# image.
+	# Keep that in mind when you encounter a broken OS X build!
+	LINUX_P4_VERSION="16.2"
+	LINUX_GIT_LFS_VERSION="1.5.2"
+
+	P4_PATH="$HOME/custom/p4"
+	GIT_LFS_PATH="$HOME/custom/git-lfs"
+	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
+
+	P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
+	LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
+
 	sudo apt-get -q update
 	sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \
 		$UBUNTU_COMMON_PKGS $CC_PACKAGE
diff --git a/ci/lib.sh b/ci/lib.sh
index edf26984aeb..3ceb36c7058 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -67,18 +67,6 @@ ubuntu-latest)
 	fi
 
 	export GIT_TEST_HTTPD=true
-
-	# The Linux build installs the defined dependency versions below.
-	# The OS X build installs much more recent versions, whichever
-	# were recorded in the Homebrew database upon creating the OS X
-	# image.
-	# Keep that in mind when you encounter a broken OS X build!
-	export LINUX_P4_VERSION="16.2"
-	export LINUX_GIT_LFS_VERSION="1.5.2"
-
-	P4_PATH="$HOME/custom/p4"
-	GIT_LFS_PATH="$HOME/custom/git-lfs"
-	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
 	;;
 macos-latest)
 	if [ "$jobname" = osx-gcc ]
-- 
2.36.0.879.g3659959fcca

[PATCH] fixup! CI: move p4 and git-lfs variables to ci/install-dependencies.sh

[Carlo Marcelo Arenas Belón]

Once the code was moved into an ubuntu specific branch, it doesn't make
sense anymore to refer to logic that only applies to a different branch

Cleanup the comment to allow for the implementation to change without
making it stale, and while at it rename the variable used to track the
P4 version to something slightly less confusing.

Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
---
 ci/install-dependencies.sh | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 098baeef3fb..7bea45fd339 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -23,11 +23,9 @@ esac
 case "$runs_on_pool" in
 ubuntu-latest)
 	# The Linux build installs the defined dependency versions below.
-	# The OS X build installs much more recent versions, whichever
-	# were recorded in the Homebrew database upon creating the OS X
-	# image.
+	# The OS X build installs much more recent versions
 	# Keep that in mind when you encounter a broken OS X build!
-	LINUX_P4_VERSION="16.2"
+	P4_AT_LINUX_VERSION="16.2"
 	LINUX_GIT_LFS_VERSION="1.5.2"
 
 	P4_PATH="$HOME/custom/p4"
@@ -39,7 +37,7 @@ ubuntu-latest)
 		echo "$PATH" >>"$GITHUB_PATH"
 	fi
 
-	P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
+	P4WHENCE=http://filehost.perforce.com/perforce/r$P4_AT_LINUX_VERSION
 	LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
 
 	sudo apt-get -q update
-- 
2.36.0.266.g59f845bde02

Re: [PATCH] fixup! CI: move p4 and git-lfs variables to ci/install-dependencies.sh

[Junio C Hamano]

Carlo Marcelo Arenas Belón  <carenas@gmail.com> writes:

> Once the code was moved into an ubuntu specific branch, it doesn't make
> sense anymore to refer to logic that only applies to a different branch
>
> Cleanup the comment to allow for the implementation to change without
> making it stale, and while at it rename the variable used to track the
> P4 version to something slightly less confusing.
>
> Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
> ---
>  ci/install-dependencies.sh | 8 +++-----
>  1 file changed, 3 insertions(+), 5 deletions(-)

OK.  The place we run wget from is

    $P4WHENCE/bin.linux26x86_64/{p4,p4d}

for Linux.  And P4WHENCE is defined to be

    http://{filehost,cdist2}.perforce.com/perforce/r$LINUX_P4_VERSION

Unfortunately, what we grab are not these two files for macOS.

https://cdist2.perforce.com/perforce/r21.2/bin.macosx1015x86_64/helix-core-server.tgz

If p4 and p4d were the only files that we grabbed for macOS, then
what I wanted to suggest in the previous message would have worked,
i.e. 

We define, per $runs_on_pool, two variables:

     P4VERSION (e.g. "r16.2", "r21.2")
     P4ARCH (e.g. "bin.macosx1015x86_64", "bin.linux26x86_64")

Then a single definition, after the "case $runs_on_pool ... esac",
of a leading URL, i.e.

     P4DIST=https://cdist2.perforce.com/perforce/$P4VERSION/$P4ARCH/

can be used to help two wget commands, i.e.

     wget --quiet "$P4DIST/p4"
     wget --quiet "$P4DIST/p4d"

in install-dependencies.sh for both Linux-x86_64 and macOS.

But if we cannot unify the way the installation is done for macOS
and Linux that way, I do not think renaming $LINUX_P4_VERSION to
another name is worth doing (notice that the point of the above
exercise was to abstract these into just two variables, VERSION and
ARCH).

By the way, I spelunked https://filehost.perforce.com/perforce/ from
a browser to see how the files are layed out over there, not cdist2.
I am guessing that "filehost" is the name they want end-users like us
to use when downloading what they distribute?

> diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
> index 098baeef3fb..7bea45fd339 100755
> --- a/ci/install-dependencies.sh
> +++ b/ci/install-dependencies.sh
> @@ -23,11 +23,9 @@ esac
>  case "$runs_on_pool" in
>  ubuntu-latest)
>  	# The Linux build installs the defined dependency versions below.
> -	# The OS X build installs much more recent versions, whichever
> -	# were recorded in the Homebrew database upon creating the OS X
> -	# image.
> +	# The OS X build installs much more recent versions
>  	# Keep that in mind when you encounter a broken OS X build!
> -	LINUX_P4_VERSION="16.2"
> +	P4_AT_LINUX_VERSION="16.2"
>  	LINUX_GIT_LFS_VERSION="1.5.2"
>  
>  	P4_PATH="$HOME/custom/p4"
> @@ -39,7 +37,7 @@ ubuntu-latest)
>  		echo "$PATH" >>"$GITHUB_PATH"
>  	fi
>  
> -	P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
> +	P4WHENCE=http://filehost.perforce.com/perforce/r$P4_AT_LINUX_VERSION
>  	LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
>  
>  	sudo apt-get -q update

Re: [PATCH] fixup! CI: move p4 and git-lfs variables to ci/install-dependencies.sh

[Carlo Arenas]

On Sat, Apr 23, 2022 at 3:59 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> If p4 and p4d were the only files that we grabbed for macOS, then
> what I wanted to suggest in the previous message would have worked,

true, but it can still work by making them both use the same version
(which would be likely a CI variable) and pull the three files from
that directory (which was my end objective as shown) in :

  https://github.com/carenas/git/commit/4fd79fa1770bc9e8e57074dbac4e44de173aeb75

> By the way, I spelunked https://filehost.perforce.com/perforce/ from
> a browser to see how the files are layed out over there, not cdist2.
> I am guessing that "filehost" is the name they want end-users like us
> to use when downloading what they distribute?

I presume that was the URL they were using when this code was written,
FWIW brew uses cdist2, and that is what I get after going to their
website at the end after clicking through their download forms so I
assume it is less likely to break as well.

agree though that renaming the variable could be avoided but it looks
nicer as an intermediate state, and we might even keep it if having
two different versions is preferred :

  https://github.com/carenas/git/commit/b6699fa3f082ef6d0e8c32718ce5142612003de8

Carlo

Re: [PATCH] fixup! CI: move p4 and git-lfs variables to ci/install-dependencies.sh

[Carlo Arenas]

On Sat, Apr 23, 2022 at 3:59 PM Junio C Hamano <gitster@pobox.com> wrote:
> But if we cannot unify the way the installation is done for macOS
> and Linux that way, I do not think renaming $LINUX_P4_VERSION to
> another name is worth doing (notice that the point of the above
> exercise was to abstract these into just two variables, VERSION and
> ARCH).

BTW, I was intentionally holding those changes for the same reason; if
renaming the variable causes any pain (my guess was that it will be
unlikely since the code got moved already at that point, and so
renaming it at the same time would be unlikely to cause further
conflicts) then it shouldn't be done now (or maybe ever).

and those changes should only be done after all the refactoring going
on now is gone, and only if discussed first with a lengthy period of
RFC to make sure we get it right and go quickly.

Of course, the final version of it will be unlikely to NOT have a
single function that does the right thing based on those two variables
and those patches were ever going to be sent as RFC.

Carlo

[PATCH v5 11/29] CI: consistently use "export" in ci/lib.sh

[Ævar Arnfjörð Bjarmason]

Change the "ci/lib.sh" script co consistently use "export", for
e.g. MAKEFLAGS we were exporting it, and then assigning to it, let's
do it the other way around.

Right now this doesn't matter, since we in
e.g. "ci/install-dependencies.sh" source this file, and don't use
something like "env(1)" to retrieve these variables. But in a
subsequent commit we'll "export" these variables through a wrapper (to
additionally write them to a GitHub CI-specific $GITHUB_ENV
file). This change makes that subsequent change easier to read, as it
won't need to do any control flow refactoring.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 3ceb36c7058..2e425cb787c 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -30,7 +30,7 @@ export TERM=${TERM:-dumb}
 NPROC=10
 
 # Clear MAKEFLAGS that may come from the outside world.
-export MAKEFLAGS=--jobs=$NPROC
+MAKEFLAGS=--jobs=$NPROC
 
 if test "$GITHUB_ACTIONS" = "true"
 then
@@ -38,9 +38,10 @@ then
 	CC="${CC:-gcc}"
 
 	export GIT_PROVE_OPTS="--timer --jobs $NPROC"
-	export GIT_TEST_OPTS="--verbose-log -x"
+	GIT_TEST_OPTS="--verbose-log -x"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
+	export GIT_TEST_OPTS
 else
 	echo "Could not identify CI type" >&2
 	env >&2
@@ -94,4 +95,4 @@ linux-leaks)
 	;;
 esac
 
-MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
+export MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
-- 
2.36.0.879.g3659959fcca

[PATCH v5 12/29] CI: export variables via a wrapper

[Ævar Arnfjörð Bjarmason]

Add and use a "setenv" wrapper to export the variables set in
ci/lib.sh.

For now this merely adds debugging information about what variables
are set, but in a subsequent commit we'll expand this to set variables
in $GITHUB_ENV instead of the environment, and thus carry variable
settings between steps in a job.

We'll also eventually be interested in what type of variable we're
setting, and when it needs to be used, because we'll prominently
expose what variables we set in what CI "step", so having only the
ones we actually use will improve readability.

To that end call the "setenv" function with optional "--build",
"--test" and "--all" parameters to indicate whether they're needed for
building, testing or both.

We currently ignore the "varname" parameter, but will make use of it in a
subsequent commit. Adding it here reduces the needless churn, as we
won't need to change all these lines again for that subsequent change.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                 | 51 ++++++++++++++++++++++++++++++---------
 ci/run-build-and-tests.sh | 34 +++++++++++++-------------
 2 files changed, 57 insertions(+), 28 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 2e425cb787c..e36e350e74c 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -13,6 +13,35 @@ then
 	exit 1
 fi
 
+# Helper functions
+setenv () {
+	while test $# != 0
+	do
+		case "$1" in
+		--build)
+			;;
+		--test)
+			;;
+		--all)
+			;;
+		-*)
+			echo "BUG: bad setenv() option '$1'" >&2
+			exit 1
+			;;
+		*)
+			break
+			;;
+		esac
+		shift
+	done
+
+	key=$1
+	val=$2
+	shift 2
+
+	eval "export $key=\"$val\""
+}
+
 check_unignored_build_artifacts ()
 {
 	! git ls-files --other --exclude-standard --error-unmatch \
@@ -24,7 +53,7 @@ check_unignored_build_artifacts ()
 }
 
 # GitHub Action doesn't set TERM, which is required by tput
-export TERM=${TERM:-dumb}
+setenv TERM ${TERM:-dumb}
 
 # How many jobs to run in parallel?
 NPROC=10
@@ -37,21 +66,21 @@ then
 	CI_TYPE=github-actions
 	CC="${CC:-gcc}"
 
-	export GIT_PROVE_OPTS="--timer --jobs $NPROC"
+	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
 	GIT_TEST_OPTS="--verbose-log -x"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
-	export GIT_TEST_OPTS
+	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
 else
 	echo "Could not identify CI type" >&2
 	env >&2
 	exit 1
 fi
 
-export DEVELOPER=1
-export DEFAULT_TEST_TARGET=prove
-export GIT_TEST_CLONE_2GB=true
-export SKIP_DASHED_BUILT_INS=YesPlease
+setenv --build DEVELOPER 1
+setenv --test DEFAULT_TEST_TARGET prove
+setenv --test GIT_TEST_CLONE_2GB true
+setenv --build SKIP_DASHED_BUILT_INS YesPlease
 
 case "$runs_on_pool" in
 ubuntu-latest)
@@ -67,7 +96,7 @@ ubuntu-latest)
 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python2"
 	fi
 
-	export GIT_TEST_HTTPD=true
+	setenv --test GIT_TEST_HTTPD true
 	;;
 macos-latest)
 	if [ "$jobname" = osx-gcc ]
@@ -90,9 +119,9 @@ linux-musl)
 	MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8"
 	;;
 linux-leaks)
-	export SANITIZE=leak
-	export GIT_TEST_PASSING_SANITIZE_LEAK=true
+	setenv --build SANITIZE leak
+	setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true
 	;;
 esac
 
-export MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
+setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index e5a21985b44..35d45a9373e 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -9,33 +9,33 @@ export MAKE_TARGETS="all test"
 
 case "$jobname" in
 linux-gcc)
-	export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
 	;;
 linux-TEST-vars)
-	export GIT_TEST_SPLIT_INDEX=yes
-	export GIT_TEST_MERGE_ALGORITHM=recursive
-	export GIT_TEST_FULL_IN_PACK_ARRAY=true
-	export GIT_TEST_OE_SIZE=10
-	export GIT_TEST_OE_DELTA_SIZE=5
-	export GIT_TEST_COMMIT_GRAPH=1
-	export GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS=1
-	export GIT_TEST_MULTI_PACK_INDEX=1
-	export GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP=1
-	export GIT_TEST_ADD_I_USE_BUILTIN=1
-	export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master
-	export GIT_TEST_WRITE_REV_INDEX=1
-	export GIT_TEST_CHECKOUT_WORKERS=2
+	setenv --test GIT_TEST_SPLIT_INDEX yes
+	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
+	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
+	setenv --test GIT_TEST_OE_SIZE 10
+	setenv --test GIT_TEST_OE_DELTA_SIZE 5
+	setenv --test GIT_TEST_COMMIT_GRAPH 1
+	setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
+	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
+	setenv --test GIT_TEST_WRITE_REV_INDEX 1
+	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
 	;;
 linux-clang)
-	export GIT_TEST_DEFAULT_HASH=sha1
+	setenv --test GIT_TEST_DEFAULT_HASH sha1
 	;;
 linux-sha256)
-	export GIT_TEST_DEFAULT_HASH=sha256
+	setenv --test GIT_TEST_DEFAULT_HASH sha256
 	;;
 pedantic)
 	# Don't run the tests; we only care about whether Git can be
 	# built.
-	export DEVOPTS=pedantic
+	setenv --build DEVOPTS pedantic
 	export MAKE_TARGETS=all
 	;;
 esac
-- 
2.36.0.879.g3659959fcca

[PATCH v5 13/29] CI: remove "run-build-and-tests.sh", run "make [test]" directly

[Ævar Arnfjörð Bjarmason]

Remove the already thin "ci/run-build-and-tests.sh" wrapper and
instead make the CI run "make" or "make test" directly. By doing this
we'll be able to easily see at a glance whether our failure was in the
compilation or testing, whether that's via human eyes or improve
machine readability.

We also need to run our new "ci/check-unignored-build-artifacts.sh" on
success() in the CI now, just like we already had a step conditional
on failure() running ci/print-test-failures.sh.

The reason we used a "ci/run-build-and-tests.sh" wrapper in the first
place had to do with Travis CI-specific constraints that no longer
apply to us, as the Travis CI support has been removed.

Instead we can configure the CI in an earlier step by running
"ci/lib.sh", which under GitHub CI will write the environment
variables we need to the "$GITHUB_ENV" file.

We'll then have access to them in subsequent steps, and crucially
those variables will be prominently visible at the start of each step
via an expandable drop-down in the UI.drop-do.

I.e. this changes the CI run from a top-down flow like (pseudocode):

 - job:
   - step1:
     - use ci/lib.sh to set env vars
     - run a script like ci/run-build-and-tests.sh
   - step2:
     - if: failure()
     - use ci/lib.sh to set env vars
     - run ci/print-test-failures.sh

To:

 - job:
   - step1:
     - set variables in $GITHUB_ENV using ci/lib.sh
   - step2:
     - make
   - step3:
     - make test
   - step4:
     - if: failure()
     - run ci/print-test-failures.sh
   - step5:
     - if: success()
     - run ci/check-unignored-build-artifacts.sh

There is a proposal[2] to get some of the benefits of this approach by
not re-arranging our variable setup in this way, but to instead use
the GitHub CI grouping syntax to focus on the relevant parts of "make"
or "make test" when we have failures.

Doing it this way makes for better looking GitHub CI UI, and lays much
better ground work for our CI going forward. Because:

 * The CI logic will be more portable to a future CI system, since a
   common feature of them is to run various commands in sequence, but
   a future system won't necessarily support the GitHub-specifics
   syntax of "grouping" output within a "step".

   Even if those systems don't support a "$GITHUB_ENV" emulating will
   be much easier than to deal with some CI-specific grouping syntax.

 * At the start of every step the GitHub CI presents an expandable
   list of environment variables from "$GITHUB_ENV". We'll now see
   exactly what variables affected that step (although we currently
   overshoot that a bit, and always define all variables).

 * CI failures will be easier to reproduce locally, as this makes the
   relevant ci/* scripts something that sets up our environment, but
   leaves "make" and "make test" working as they do locally.

   To reproduce a run the user only needs to set the variables
   discussed in the drop-down above, either manually or by running
   "ci/lib.sh".

 * The output will be less verbose. The "ci/lib.sh" script uses "set
   -x", and before this e.g. "ci/static-analysis.sh" would start with
   40 lines of trace output, culminating in using "export" to export
   the relevant environment variables.

   Now that verbosity is in the earlier "ci/lib.sh" step, and not in
   any subsequent one. The "make" targets then start out with the
   relevant non-trace output right away.

 * If we do want to use the grouping syntax within a "step" it'll now
   be easier to do so. It doesn't support nesting, so we'd have to
   make a choice between using it for e.g. "make" v.s. "make test", or
   individual test failures. See "sadly" in [3].

1. https://lore.kernel.org/git/211120.86k0h30zuw.gmgdl@evledraar.gmail.com/
2. https://lore.kernel.org/git/pull.1117.git.1643050574.gitgitgadget@gmail.com/
3. https://lore.kernel.org/git/9333ba781b8240f704e739b00d274f8c3d887e39.1643050574.git.gitgitgadget@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml            | 14 ++++++--
 ci/check-unignored-build-artifacts.sh |  8 +++++
 ci/install-dependencies.sh            |  4 +++
 ci/lib.sh                             | 36 +++++++++++++++++++-
 ci/make-test-artifacts.sh             |  2 --
 ci/run-build-and-tests.sh             | 47 ---------------------------
 ci/run-test-slice.sh                  |  2 --
 ci/test-documentation.sh              |  1 -
 8 files changed, 58 insertions(+), 56 deletions(-)
 create mode 100755 ci/check-unignored-build-artifacts.sh
 delete mode 100755 ci/run-build-and-tests.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index ad752010102..57b2db06923 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -268,7 +268,10 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/run-build-and-tests.sh
+    - run: ci/lib.sh
+    - run: make
+    - run: make test
+      if: success()
     - run: ci/print-test-failures.sh
       if: failure()
     - name: Upload failed tests' directories
@@ -292,6 +295,7 @@ jobs:
           image: daald/ubuntu32:xenial
         - jobname: pedantic
           image: fedora
+          skip-tests: yes
     env:
       jobname: ${{matrix.vector.jobname}}
     runs-on: ubuntu-latest
@@ -299,9 +303,12 @@ jobs:
     steps:
     - uses: actions/checkout@v1
     - run: ci/install-docker-dependencies.sh
-    - run: ci/run-build-and-tests.sh
+    - run: ci/lib.sh
+    - run: make
+    - run: make test
+      if: success() && matrix.vector.skip-tests != 'yes'
     - run: ci/print-test-failures.sh
-      if: failure()
+      if: failure() && matrix.vector.skip-tests != 'yes'
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v1
@@ -317,6 +324,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
+    - run: ci/lib.sh
     - run: make ci-static-analysis
   sparse:
     needs: ci-config
diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh
new file mode 100755
index 00000000000..56d04b0db9a
--- /dev/null
+++ b/ci/check-unignored-build-artifacts.sh
@@ -0,0 +1,8 @@
+#!/bin/sh
+#
+# Check whether the build created anything not in our .gitignore
+#
+
+. ${0%/*}/lib.sh
+
+check_unignored_build_artifacts
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 34b4400f38b..01006379fd0 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -22,6 +22,10 @@ ubuntu-latest)
 	P4_PATH="$HOME/custom/p4"
 	GIT_LFS_PATH="$HOME/custom/git-lfs"
 	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
+	if test -n "$GITHUB_PATH"
+	then
+		echo "$PATH" >>"$GITHUB_PATH"
+	fi
 
 	P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
 	LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
diff --git a/ci/lib.sh b/ci/lib.sh
index e36e350e74c..43231862aa5 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,3 +1,5 @@
+#!/bin/sh
+
 # Library of functions shared by all CI scripts
 
 # Set 'exit on error' for all CI scripts to let the caller know that
@@ -39,7 +41,10 @@ setenv () {
 	val=$2
 	shift 2
 
-	eval "export $key=\"$val\""
+	if test -n "$GITHUB_ENV"
+	then
+		echo "$key=$val" >>"$GITHUB_ENV"
+	fi
 }
 
 check_unignored_build_artifacts ()
@@ -109,6 +114,35 @@ macos-latest)
 esac
 
 case "$jobname" in
+linux-gcc)
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
+	;;
+linux-TEST-vars)
+	setenv --test GIT_TEST_SPLIT_INDEX yes
+	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
+	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
+	setenv --test GIT_TEST_OE_SIZE 10
+	setenv --test GIT_TEST_OE_DELTA_SIZE 5
+	setenv --test GIT_TEST_COMMIT_GRAPH 1
+	setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
+	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
+	setenv --test GIT_TEST_WRITE_REV_INDEX 1
+	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
+	;;
+linux-clang)
+	setenv --test GIT_TEST_DEFAULT_HASH sha1
+	;;
+linux-sha256)
+	setenv --test GIT_TEST_DEFAULT_HASH sha256
+	;;
+pedantic)
+	# Don't run the tests; we only care about whether Git can be
+	# built.
+	setenv --build DEVOPTS pedantic
+	;;
 linux32)
 	CC=gcc
 	;;
diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh
index 646967481f6..45298562982 100755
--- a/ci/make-test-artifacts.sh
+++ b/ci/make-test-artifacts.sh
@@ -8,5 +8,3 @@ mkdir -p "$1" # in case ci/lib.sh decides to quit early
 . ${0%/*}/lib.sh
 
 make artifacts-tar ARTIFACTS_DIRECTORY="$1"
-
-check_unignored_build_artifacts
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
deleted file mode 100755
index 35d45a9373e..00000000000
--- a/ci/run-build-and-tests.sh
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/bin/sh
-#
-# Build and test Git
-#
-
-. ${0%/*}/lib.sh
-
-export MAKE_TARGETS="all test"
-
-case "$jobname" in
-linux-gcc)
-	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
-	;;
-linux-TEST-vars)
-	setenv --test GIT_TEST_SPLIT_INDEX yes
-	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
-	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
-	setenv --test GIT_TEST_OE_SIZE 10
-	setenv --test GIT_TEST_OE_DELTA_SIZE 5
-	setenv --test GIT_TEST_COMMIT_GRAPH 1
-	setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
-	setenv --test GIT_TEST_MULTI_PACK_INDEX 1
-	setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
-	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
-	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
-	setenv --test GIT_TEST_WRITE_REV_INDEX 1
-	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
-	;;
-linux-clang)
-	setenv --test GIT_TEST_DEFAULT_HASH sha1
-	;;
-linux-sha256)
-	setenv --test GIT_TEST_DEFAULT_HASH sha256
-	;;
-pedantic)
-	# Don't run the tests; we only care about whether Git can be
-	# built.
-	setenv --build DEVOPTS pedantic
-	export MAKE_TARGETS=all
-	;;
-esac
-
-# Any new "test" targets should not go after this "make", but should
-# adjust $MAKE_TARGETS. Otherwise compilation-only targets above will
-# start running tests.
-make $MAKE_TARGETS
-check_unignored_build_artifacts
diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index b9a682b4bcd..70326961454 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -8,5 +8,3 @@
 make --quiet -C t T="$(cd t &&
 	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
 	tr '\n' ' ')"
-
-check_unignored_build_artifacts
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 41e2b126311..6b5cce03bd7 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -39,4 +39,3 @@ test -s Documentation/git.html
 grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
 
 rm -f stdout.log stderr.log stderr.raw
-check_unignored_build_artifacts
-- 
2.36.0.879.g3659959fcca

[PATCH v5 14/29] CI: make ci/{lib,install-dependencies}.sh POSIX-compatible

[Ævar Arnfjörð Bjarmason]

Change the unportable constructs in the "ci/install-dependencies.sh"
scripts to use POSIX replacements, and have it run under "/bin/sh"
instead of "/usr/bin/env bash". We needed to:

* Replace pushd/popd with a sub-shell.
* Change "mkdir --parents" to "mkdir -p".
* Use POSIX "export" syntax.

This is in preparation for combining this script and
"ci/install-docker-dependencies.sh", which doesn't have access to
"bash".

While we're at it change "if [" to "if test" in ci/lib.sh. This isn't
strictly speaking a case of POSIX-incompatibility, but that syntax was
being used there because this code was using idiomatic bash
constructs. Let's have it use our usual style of "if test" instead.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/install-dependencies.sh | 27 +++++++++++++++++----------
 ci/lib.sh                  |  4 ++--
 2 files changed, 19 insertions(+), 12 deletions(-)

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 01006379fd0..1ad4dd7db39 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -1,4 +1,4 @@
-#!/usr/bin/env bash
+#!/bin/sh
 #
 # Install dependencies required to build and test Git on Linux and macOS
 #
@@ -21,7 +21,8 @@ ubuntu-latest)
 
 	P4_PATH="$HOME/custom/p4"
 	GIT_LFS_PATH="$HOME/custom/git-lfs"
-	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
+	PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
+	export PATH
 	if test -n "$GITHUB_PATH"
 	then
 		echo "$PATH" >>"$GITHUB_PATH"
@@ -33,22 +34,28 @@ ubuntu-latest)
 	sudo apt-get -q update
 	sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \
 		$UBUNTU_COMMON_PKGS $CC_PACKAGE
-	mkdir --parents "$P4_PATH"
-	pushd "$P4_PATH"
+	mkdir -p "$P4_PATH"
+	(
+		cd "$P4_PATH"
 		wget --quiet "$P4WHENCE/bin.linux26x86_64/p4d"
 		wget --quiet "$P4WHENCE/bin.linux26x86_64/p4"
 		chmod u+x p4d
 		chmod u+x p4
-	popd
-	mkdir --parents "$GIT_LFS_PATH"
-	pushd "$GIT_LFS_PATH"
+	)
+	mkdir -p "$GIT_LFS_PATH"
+	(
+		cd "$GIT_LFS_PATH"
 		wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
 		tar --extract --gunzip --file "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
 		cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs .
-	popd
+	)
 	;;
 macos-latest)
-	export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
+	HOMEBREW_NO_AUTO_UPDATE=1
+	export HOMEBREW_NO_AUTO_UPDATE
+	HOMEBREW_NO_INSTALL_CLEANUP=1
+	export HOMEBREW_NO_INSTALL_CLEANUP
+
 	# Uncomment this if you want to run perf tests:
 	# brew install gnu-time
 	brew link --force gettext
@@ -62,7 +69,7 @@ macos-latest)
 
 	if test -n "$CC_PACKAGE"
 	then
-		BREW_PACKAGE=${CC_PACKAGE/-/@}
+		BREW_PACKAGE=$(echo $CC_PACKAGE | tr '-' '@')
 		brew install "$BREW_PACKAGE"
 		brew link "$BREW_PACKAGE"
 	fi
diff --git a/ci/lib.sh b/ci/lib.sh
index 43231862aa5..2fd022c94bc 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -94,7 +94,7 @@ ubuntu-latest)
 		break
 	fi
 
-	if [ "$jobname" = linux-gcc ]
+	if test "$jobname" = linux-gcc
 	then
 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python3"
 	else
@@ -104,7 +104,7 @@ ubuntu-latest)
 	setenv --test GIT_TEST_HTTPD true
 	;;
 macos-latest)
-	if [ "$jobname" = osx-gcc ]
+	if test "$jobname" = osx-gcc
 	then
 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
 	else
-- 
2.36.0.879.g3659959fcca

[PATCH v5 15/29] CI: check ignored unignored build artifacts in "win[+VS] build" too

[Ævar Arnfjörð Bjarmason]

Use the newly created "ci/check-unignored-build-artifacts.sh" script
in the "win build" and "win+VS build" steps too. These steps weren't
doing this sanity check of untracked files against .gitignore before,
but as the check passes for them let's keep doing that extra sanity
check.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 57b2db06923..c97fbd0f0b3 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -93,6 +93,9 @@ jobs:
         HOME: ${{runner.workspace}}
         NO_PERL: 1
       run: . /etc/profile && ci/make-test-artifacts.sh artifacts
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
+      shell: bash
     - name: zip up tracked files
       run: git archive -o artifacts/tracked.tar.gz HEAD
     - name: upload tracked files and build artifacts
@@ -180,6 +183,9 @@ jobs:
       run: |
         mkdir -p artifacts &&
         eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts NO_GETTEXT=YesPlease 2>&1 | grep ^tar)"
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
+      shell: bash
     - name: zip up tracked files
       run: git archive -o artifacts/tracked.tar.gz HEAD
     - name: upload tracked files and build artifacts
-- 
2.36.0.879.g3659959fcca

[PATCH v5 16/29] CI: invoke "make artifacts-tar" directly in windows-build

[Ævar Arnfjörð Bjarmason]

Change the windows-build job to invoke the "make artifacts-tar" step
directly, instead of calling a "ci/make-test-artifacts.sh" script.

The script was needed because "ci/lib.sh" would set up various
environment variables for us, but now we can instead use the
"ci/lib.sh" in its script mode.

The "mkdir -p" added in b819f1d2cec (ci: parallelize testing on
Windows, 2019-01-29) isn't needed, the same commit added that "mkdir
-p" to the "artifacts-tar" rule itself, so we can have "make" create
the directory for us.

This also has the benefit of making the "build" step less chatty,
since it won't start with the verbose "set -x" output, that's now
contained in the "ci/lib.sh" step.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml |  5 ++++-
 ci/make-test-artifacts.sh  | 10 ----------
 2 files changed, 4 insertions(+), 11 deletions(-)
 delete mode 100755 ci/make-test-artifacts.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index c97fbd0f0b3..ca993afd098 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -87,12 +87,15 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
+    - run: ci/lib.sh
+      shell: bash
     - name: build
       shell: bash
       env:
         HOME: ${{runner.workspace}}
         NO_PERL: 1
-      run: . /etc/profile && ci/make-test-artifacts.sh artifacts
+      run: . /etc/profile && make artifacts-tar ARTIFACTS_DIRECTORY=artifacts
+      if: success()
     - run: ci/check-unignored-build-artifacts.sh
       if: success()
       shell: bash
diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh
deleted file mode 100755
index 45298562982..00000000000
--- a/ci/make-test-artifacts.sh
+++ /dev/null
@@ -1,10 +0,0 @@
-#!/bin/sh
-#
-# Build Git and store artifacts for testing
-#
-
-mkdir -p "$1" # in case ci/lib.sh decides to quit early
-
-. ${0%/*}/lib.sh
-
-make artifacts-tar ARTIFACTS_DIRECTORY="$1"
-- 
2.36.0.879.g3659959fcca

[PATCH v5 17/29] CI: split up and reduce "ci/test-documentation.sh"

[Ævar Arnfjörð Bjarmason]

Change the "ci/test-documentation.sh" script to run the bash-specific
parts in as one command in the CI job itself, and to run the two "make
doc" commands at the top-level.

It'll now be obvious from the title of the step if we failed in the
asciidoc or asciidoctor step.

Since the "check_unignored_build_artifacts()" function is now only
used in "ci/check-unignored-build-artifacts.sh" move that function
there.

The recipe for the job in ".github/workflows/main.yml" is now a bit
verbose because it's effectively the same job twice, with a "make
clean" in-between. It would be better for the verbosity to run it via
a matrix as done in the alternate approach in [1] does, but then we'd
sacrifice overall CPU time for the brevity. It might still be worth
doing, but let's go for this simpler approach for now.

1. https://lore.kernel.org/git/patch-v2-6.6-7c423c8283d-20211120T030848Z-avarab@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml            | 16 +++++++++++++-
 ci/check-unignored-build-artifacts.sh | 10 +++++++++
 ci/lib.sh                             | 10 ---------
 ci/test-documentation.sh              | 31 +++++++++------------------
 4 files changed, 35 insertions(+), 32 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index ca993afd098..88eb2ef2dad 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -365,4 +365,18 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/test-documentation.sh
+    - run: ci/lib.sh
+    - run: make check-docs
+    - run: "make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)"
+      shell: bash
+    - run: ci/test-documentation.sh AsciiDoc
+      if: success()
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
+    - run: make clean
+    - run: "make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)"
+      shell: bash
+    - run: ci/test-documentation.sh Asciidoctor
+      if: success()
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh
index 56d04b0db9a..0bc04f32804 100755
--- a/ci/check-unignored-build-artifacts.sh
+++ b/ci/check-unignored-build-artifacts.sh
@@ -5,4 +5,14 @@
 
 . ${0%/*}/lib.sh
 
+check_unignored_build_artifacts ()
+{
+	! git ls-files --other --exclude-standard --error-unmatch \
+		-- ':/*' 2>/dev/null ||
+	{
+		echo "$(tput setaf 1)error: found unignored build artifacts$(tput sgr0)"
+		false
+	}
+}
+
 check_unignored_build_artifacts
diff --git a/ci/lib.sh b/ci/lib.sh
index 2fd022c94bc..63c14f504b3 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -47,16 +47,6 @@ setenv () {
 	fi
 }
 
-check_unignored_build_artifacts ()
-{
-	! git ls-files --other --exclude-standard --error-unmatch \
-		-- ':/*' 2>/dev/null ||
-	{
-		echo "$(tput setaf 1)error: found unignored build artifacts$(tput sgr0)"
-		false
-	}
-}
-
 # GitHub Action doesn't set TERM, which is required by tput
 setenv TERM ${TERM:-dumb}
 
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 6b5cce03bd7..b8a6a6f664e 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -1,10 +1,12 @@
-#!/usr/bin/env bash
+#!/bin/sh
 #
-# Perform sanity checks on documentation and build it.
+# Perform sanity checks on "make doc" output and built documentation
 #
 
 . ${0%/*}/lib.sh
 
+generator=$1
+
 filter_log () {
 	sed -e '/^GIT_VERSION = /d' \
 	    -e "/constant Gem::ConfigMap is deprecated/d" \
@@ -14,28 +16,15 @@ filter_log () {
 	    "$1"
 }
 
-make check-docs
-
-# Build docs with AsciiDoc
-make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
-cat stderr.raw
-filter_log stderr.raw >stderr.log
-test ! -s stderr.log
-test -s Documentation/git.html
-test -s Documentation/git.xml
-test -s Documentation/git.1
-grep '<meta name="generator" content="AsciiDoc ' Documentation/git.html
-
-rm -f stdout.log stderr.log stderr.raw
-check_unignored_build_artifacts
-
-# Build docs with AsciiDoctor
-make clean
-make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
 cat stderr.raw
 filter_log stderr.raw >stderr.log
 test ! -s stderr.log
 test -s Documentation/git.html
-grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
+if test "$generator" = "Asciidoctor"
+then
+	test -s Documentation/git.xml
+	test -s Documentation/git.1
+fi
+grep "<meta name=\"generator\" content=\"$generator " Documentation/git.html
 
 rm -f stdout.log stderr.log stderr.raw
-- 
2.36.0.879.g3659959fcca

[PATCH v5 18/29] CI: combine ci/install{,-docker}-dependencies.sh

[Ævar Arnfjörð Bjarmason]

Combine the "ci/install-dependencies.sh" and
"ci/install-docker-dependencies.sh" into one script. The "case"
statement in the latter only cared about "$jobname", and can be folded
into the same "case" statement in the former.

The reason they split up is historical, until a preceding commit
"ci/lib.sh" required "bash", which might not have been available in
"docker".

This also fixes issue in "ci/install-docker-dependencies.sh" where
we'd hide errors due to not using "set -e". Now that we include
"ci/lib.sh" we won't have that potential issue.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml        |  2 +-
 ci/install-dependencies.sh        | 15 +++++++++++++++
 ci/install-docker-dependencies.sh | 22 ----------------------
 3 files changed, 16 insertions(+), 23 deletions(-)
 delete mode 100755 ci/install-docker-dependencies.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 88eb2ef2dad..42944020436 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -311,7 +311,7 @@ jobs:
     container: ${{matrix.vector.image}}
     steps:
     - uses: actions/checkout@v1
-    - run: ci/install-docker-dependencies.sh
+    - run: ci/install-dependencies.sh
     - run: ci/lib.sh
     - run: make
     - run: make test
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 1ad4dd7db39..adece989491 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -97,6 +97,21 @@ linux-gcc-default)
 	sudo apt-get -q update
 	sudo apt-get -q -y install $UBUNTU_COMMON_PKGS
 	;;
+linux32)
+	linux32 --32bit i386 sh -c '
+		apt update >/dev/null &&
+		apt install -y build-essential libcurl4-openssl-dev \
+			libssl-dev libexpat-dev gettext python >/dev/null
+	'
+	;;
+linux-musl)
+	apk add --update build-base curl-dev openssl-dev expat-dev gettext \
+		pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null
+	;;
+pedantic)
+	dnf -yq update >/dev/null &&
+	dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null
+	;;
 esac
 
 if type p4d >/dev/null && type p4 >/dev/null
diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh
deleted file mode 100755
index 78b7e326da6..00000000000
--- a/ci/install-docker-dependencies.sh
+++ /dev/null
@@ -1,22 +0,0 @@
-#!/bin/sh
-#
-# Install dependencies required to build and test Git inside container
-#
-
-case "$jobname" in
-linux32)
-	linux32 --32bit i386 sh -c '
-		apt update >/dev/null &&
-		apt install -y build-essential libcurl4-openssl-dev \
-			libssl-dev libexpat-dev gettext python >/dev/null
-	'
-	;;
-linux-musl)
-	apk add --update build-base curl-dev openssl-dev expat-dev gettext \
-		pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null
-	;;
-pedantic)
-	dnf -yq update >/dev/null &&
-	dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null
-	;;
-esac
-- 
2.36.0.879.g3659959fcca

[PATCH v5 19/29] CI: move "env" definitions into ci/lib.sh

[Ævar Arnfjörð Bjarmason]

Have the GitHub CI jobs use the "ci/lib".sh as a source of truth for
environment variables that affect their builds and tests. This brings
them in line with the rest of the jobs, and make it easier to see the
entire CI configuration per jobname at a glance.

To do this we need to add a ci/lib.sh "step" to the remaining jobs
that didn't have it added in preceding commits.

The Makefile parameters are luckily all accepted via the environment,
so we can export these instead of adding them to MAKEFLAGS.

Let's also use the documented GIT_CONIFG_* mechanism for setting
config instead of the internal GIT_CONFIG_PARAMETERS variable. This
adjusts code added in 889cacb6897 (ci: configure GitHub Actions for
CI/PR, 2020-04-11), we could probably use the GIT_AUTHOR_NAME etc, but
let's keep setting this via config, just with the documented
mechanism.

By setting "ARTIFACTS_DIRECTORY=artifacts" once we don't need to
repeat it in various places, let's get it from the environment
instead.

Finally, the "DEVELOPER: 1" was only needed by the "sparse" job, which
wasn't using "ci/lib.sh" (and which sets DEVELOPER=1). Let's instead
have the "sparse" job use the "ci/lib.sh" and remove DEVELOPER=1 from
".github/workflows/main.yml".

This substantially speeds up the "sparse" job, since it'll now pick up
the "--jobs" setting in MAKEFLAGS that we use everywhere else. Before
it ran in around 4m30s, now in around 2m30s.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 29 +++++++++++------------------
 ci/lib.sh                  | 21 +++++++++++++++++++++
 2 files changed, 32 insertions(+), 18 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 42944020436..5f167ac5be6 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -2,9 +2,6 @@ name: CI
 
 on: [push, pull_request]
 
-env:
-  DEVELOPER: 1
-
 jobs:
   ci-config:
     name: config
@@ -93,8 +90,7 @@ jobs:
       shell: bash
       env:
         HOME: ${{runner.workspace}}
-        NO_PERL: 1
-      run: . /etc/profile && make artifacts-tar ARTIFACTS_DIRECTORY=artifacts
+      run: . /etc/profile && make artifacts-tar
       if: success()
     - run: ci/check-unignored-build-artifacts.sh
       if: success()
@@ -105,7 +101,7 @@ jobs:
       uses: actions/upload-artifact@v2
       with:
         name: windows-artifacts
-        path: artifacts
+        path: ${{env.ARTIFACTS_DIRECTORY}}
   windows-test:
     name: win test
     env:
@@ -126,6 +122,8 @@ jobs:
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
+    - run: ci/lib.sh
+      shell: bash
     - name: test
       shell: bash
       run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
@@ -145,9 +143,6 @@ jobs:
       jobname: vs-build
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
-    env:
-      NO_PERL: 1
-      GIT_CONFIG_PARAMETERS: "'user.name=CI' 'user.email=ci@git'"
     runs-on: windows-latest
     steps:
     - uses: actions/checkout@v2
@@ -171,6 +166,8 @@ jobs:
     - name: copy dlls to root
       shell: cmd
       run: compat\vcbuild\vcpkg_copy_dlls.bat release
+    - run: ci/lib.sh
+      shell: bash
     - name: generate Visual Studio solution
       shell: bash
       run: |
@@ -181,21 +178,16 @@ jobs:
     - name: bundle artifact tar
       shell: bash
       env:
-        MSVC: 1
         VCPKG_ROOT: ${{github.workspace}}\compat\vcbuild\vcpkg
       run: |
-        mkdir -p artifacts &&
-        eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts NO_GETTEXT=YesPlease 2>&1 | grep ^tar)"
-    - run: ci/check-unignored-build-artifacts.sh
-      if: success()
-      shell: bash
+        eval "$(make -n artifacts-tar 2>&1 | grep -e ^mkdir -e ^tar)"
     - name: zip up tracked files
       run: git archive -o artifacts/tracked.tar.gz HEAD
     - name: upload tracked files and build artifacts
       uses: actions/upload-artifact@v2
       with:
         name: vs-artifacts
-        path: artifacts
+        path: ${{env.ARTIFACTS_DIRECTORY}}
   vs-test:
     name: win+VS test
     env:
@@ -216,10 +208,10 @@ jobs:
     - name: extract tracked files and build artifacts
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
+    - run: ci/lib.sh
+      shell: bash
     - name: test
       shell: bash
-      env:
-        NO_SVN_TESTS: 1
       run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
     - name: ci/print-test-failures.sh
       if: failure()
@@ -354,6 +346,7 @@ jobs:
     - uses: actions/checkout@v2
     - name: Install other dependencies
       run: ci/install-dependencies.sh
+    - run: ci/lib.sh
     - run: make sparse
   documentation:
     name: documentation
diff --git a/ci/lib.sh b/ci/lib.sh
index 63c14f504b3..cae324e2fd7 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -104,6 +104,27 @@ macos-latest)
 esac
 
 case "$jobname" in
+windows-build)
+	setenv --build NO_PERL NoThanks
+	setenv --build ARTIFACTS_DIRECTORY artifacts
+	;;
+vs-build)
+	setenv --build NO_PERL NoThanks
+	setenv --build NO_GETTEXT NoThanks
+	setenv --build ARTIFACTS_DIRECTORY artifacts
+	setenv --build INCLUDE_DLLS_IN_ARTIFACTS YesPlease
+	setenv --build MSVC YesPlease
+
+	setenv --build GIT_CONFIG_COUNT 2
+	setenv --build GIT_CONFIG_KEY_0 user.name
+	setenv --build GIT_CONFIG_VALUE_0 CI
+	setenv --build GIT_CONFIG_KEY_1 user.emailname
+	setenv --build GIT_CONFIG_VALUE_1 ci@git
+	setenv --build GIT_CONFIG_VALUE_1 ci@git
+	;;
+vs-test)
+	setenv --test NO_SVN_TESTS YesPlease
+	;;
 linux-gcc)
 	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
 	;;
-- 
2.36.0.879.g3659959fcca

[PATCH v5 20/29] ci/run-test-slice.sh: replace shelling out with "echo"

[Ævar Arnfjörð Bjarmason]

Amend this code added in b819f1d2cec (ci: parallelize testing on
Windows, 2019-01-29) to stop shelling out to "tr", all we're trying to
do here is to turn '\n'-delimited output into a ' '-delimited list. To
do that we can just use "echo".

None of these files will contain the odd characters using "echo" might
choke on in the general case (i.e. '\'-escapes and the like).

This has the added benefit of not adding a redundant ' ' to the end of
the T-parameter, which didn't matter in practice, but as we'll see in
a subsequent commit makes for slightly nicer output, as we'll show
this variable using a GitHub CI-native feature.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/run-test-slice.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index 70326961454..1de6a18ca47 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -5,6 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-make --quiet -C t T="$(cd t &&
-	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
-	tr '\n' ' ')"
+tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \
+	t[0-9]*.sh))
+make --quiet -C t T="$tests"
-- 
2.36.0.879.g3659959fcca

[PATCH v5 21/29] CI: pre-select test slice in Windows & VS tests

[Ævar Arnfjörð Bjarmason]

In preceding commits the tests have been changed to do their setup via
$GITHUB_ENV in one step, and to then have subsequent steps that re-use
that environment.

Let's change the "test slice" tests added in b819f1d2cec (ci:
parallelize testing on Windows, 2019-01-29) to do the same. These
tests select 10% of the tests to run in 10 "test slices". Now we'll
select those in a step that immediately precedes the testing step, and
then simply invoke "make -C t -e".

This has the advantage that the tests to be run are now listed in the
standard "Run" drop-down at the start of the "test" step.

Since the "T" variable in "t/Makefile" doesn't normally accept
overrides from the environment we need to invoke "make" with the "-e"
option (a.k.a. "--environment-overrides"). We could also make $(T) in
t/Makefile be a "?=" assigned variable, but this way works, and is
arguably clearer as it's more obvious that we're injecting a special
list of tests that override the normal behavior of that Makefile.

Note that we cannot run the top-level "make test" here, because of how
the Windows CI builds git, i.e. either via CMake or some option that
would cause "make test" to recompile git itself. Instead we run "make
-C t [...]".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml                     | 10 ++++++++--
 ci/{run-test-slice.sh => select-test-slice.sh} |  7 +++++--
 2 files changed, 13 insertions(+), 4 deletions(-)
 rename ci/{run-test-slice.sh => select-test-slice.sh} (50%)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 5f167ac5be6..83e0aa1f469 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -124,9 +124,12 @@ jobs:
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
     - run: ci/lib.sh
       shell: bash
+    - name: select tests
+      run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
+      shell: bash
     - name: test
       shell: bash
-      run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
+      run: . /etc/profile && make -C t -e
     - name: ci/print-test-failures.sh
       if: failure()
       shell: bash
@@ -210,9 +213,12 @@ jobs:
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
     - run: ci/lib.sh
       shell: bash
+    - name: select tests
+      run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
+      shell: bash
     - name: test
       shell: bash
-      run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
+      run: . /etc/profile && make -C t -e
     - name: ci/print-test-failures.sh
       if: failure()
       shell: bash
diff --git a/ci/run-test-slice.sh b/ci/select-test-slice.sh
similarity index 50%
rename from ci/run-test-slice.sh
rename to ci/select-test-slice.sh
index 1de6a18ca47..f59d8cadda5 100755
--- a/ci/run-test-slice.sh
+++ b/ci/select-test-slice.sh
@@ -1,10 +1,13 @@
 #!/bin/sh
 #
-# Test Git in parallel
+# Select a portion of the tests for testing Git in parallel
 #
 
 . ${0%/*}/lib.sh
 
 tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \
 	t[0-9]*.sh))
-make --quiet -C t T="$tests"
+if test -n "$GITHUB_ENV"
+then
+	echo T="$tests" >>$GITHUB_ENV
+fi
-- 
2.36.0.879.g3659959fcca

[PATCH v5 22/29] CI: only invoke ci/lib.sh as "steps" in main.yml

[Ævar Arnfjörð Bjarmason]

Change the scripts in ci/ to stop using ci/lib.sh as a library, now
that the only thing it did for them was to "set -ex" and possibly set
TERM=dumb.

Let's create a ci/lib-tput.sh for those that need to use "tput"
instead, and have these scripts invoke "set -ex" themselves.

This makes their invocation a lot less verbose, since they'll be
relying on an earlier step in the CI job to have set the variables in
$GITHUB_ENV, and won't be spewing their own trace output to set those
variables again.

Let's also create a ci/lib-ci-type.sh, and have ci/lib.sh and
ci/print-test-failures.sh share the logic to discover the CI type. We
could have set the CI_TYPE in the environment with "setenv", but let's
avoid that verbosity for this purely internal variable.

The "ci/lib.sh" is now no longer a "Library of functions shared by all
CI scripts", so let's remove that commentary, and the misleading
comment about "set -ex" being for "installing dependencies", we're now
no longer using it in "ci/install-dependencies.sh" (but it does its
own "set -ex").

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/check-unignored-build-artifacts.sh |  4 +++-
 ci/install-dependencies.sh            |  2 +-
 ci/lib-ci-type.sh                     |  9 +++++++++
 ci/lib-tput.sh                        |  5 +++++
 ci/lib.sh                             | 27 ++++++++++-----------------
 ci/print-test-failures.sh             |  6 +++---
 ci/select-test-slice.sh               |  2 +-
 ci/test-documentation.sh              |  2 +-
 8 files changed, 33 insertions(+), 24 deletions(-)
 create mode 100644 ci/lib-ci-type.sh
 create mode 100644 ci/lib-tput.sh

diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh
index 0bc04f32804..c27d6a97f45 100755
--- a/ci/check-unignored-build-artifacts.sh
+++ b/ci/check-unignored-build-artifacts.sh
@@ -3,7 +3,9 @@
 # Check whether the build created anything not in our .gitignore
 #
 
-. ${0%/*}/lib.sh
+set -ex
+
+. ${0%/*}/lib-tput.sh
 
 check_unignored_build_artifacts ()
 {
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index adece989491..4b4cdbe26e0 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -3,7 +3,7 @@
 # Install dependencies required to build and test Git on Linux and macOS
 #
 
-. ${0%/*}/lib.sh
+set -ex
 
 UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
  tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
diff --git a/ci/lib-ci-type.sh b/ci/lib-ci-type.sh
new file mode 100644
index 00000000000..6f01fd9e5d9
--- /dev/null
+++ b/ci/lib-ci-type.sh
@@ -0,0 +1,9 @@
+#!/bin/sh
+
+if test "$GITHUB_ACTIONS" = "true"
+then
+	CI_TYPE=github-actions
+else
+	echo "Could not identify CI type" >&2
+	exit 1
+fi
diff --git a/ci/lib-tput.sh b/ci/lib-tput.sh
new file mode 100644
index 00000000000..b62ef13c44c
--- /dev/null
+++ b/ci/lib-tput.sh
@@ -0,0 +1,5 @@
+#!/bin/sh
+
+# GitHub Action doesn't set TERM, which is required by tput
+TERM=${TERM:-dumb}
+export TERM
diff --git a/ci/lib.sh b/ci/lib.sh
index cae324e2fd7..fea45a1a8d3 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,13 +1,9 @@
 #!/bin/sh
-
-# Library of functions shared by all CI scripts
-
-# Set 'exit on error' for all CI scripts to let the caller know that
-# something went wrong.
-# Set tracing executed commands, primarily setting environment variables
-# and installing dependencies.
 set -ex
 
+# Helper libraries
+. ${0%/*}/lib-ci-type.sh
+
 # Starting assertions
 if test -z "$jobname"
 then
@@ -47,18 +43,14 @@ setenv () {
 	fi
 }
 
-# GitHub Action doesn't set TERM, which is required by tput
-setenv TERM ${TERM:-dumb}
-
 # How many jobs to run in parallel?
 NPROC=10
 
 # Clear MAKEFLAGS that may come from the outside world.
 MAKEFLAGS=--jobs=$NPROC
 
-if test "$GITHUB_ACTIONS" = "true"
-then
-	CI_TYPE=github-actions
+case "$CI_TYPE" in
+github-actions)
 	CC="${CC:-gcc}"
 
 	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
@@ -66,11 +58,12 @@ then
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
-else
-	echo "Could not identify CI type" >&2
-	env >&2
+	;;
+*)
+	echo "Unhandled CI type: $CI_TYPE" >&2
 	exit 1
-fi
+	;;
+esac
 
 setenv --build DEVELOPER 1
 setenv --test DEFAULT_TEST_TARGET prove
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 47e2ea1ad10..29f8c332eca 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -3,10 +3,10 @@
 # Print output of failing tests
 #
 
-. ${0%/*}/lib.sh
+set -e
 
-# Tracing executed commands would produce too much noise in the loop below.
-set +x
+. ${0%/*}/lib-ci-type.sh
+. ${0%/*}/lib-tput.sh
 
 cd t/
 
diff --git a/ci/select-test-slice.sh b/ci/select-test-slice.sh
index f59d8cadda5..eccf9a48104 100755
--- a/ci/select-test-slice.sh
+++ b/ci/select-test-slice.sh
@@ -3,7 +3,7 @@
 # Select a portion of the tests for testing Git in parallel
 #
 
-. ${0%/*}/lib.sh
+set -ex
 
 tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \
 	t[0-9]*.sh))
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index b8a6a6f664e..64ff212cdaa 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -3,7 +3,7 @@
 # Perform sanity checks on "make doc" output and built documentation
 #
 
-. ${0%/*}/lib.sh
+set -ex
 
 generator=$1
 
-- 
2.36.0.879.g3659959fcca

[PATCH v5 23/29] CI: narrow down variable definitions in --build and --test

[Ævar Arnfjörð Bjarmason]

In a preceding step the "setvar" function was made to take a
"--build", "--test" or "--all" argument to indicate where the
variables it sets were used.

Let's make use of that by having the relevant parts of
".github/workflows/main.yml" invoke "ci/lib.sh" with those options.

By doing this the set of variables shown in build-only steps will be
fewer, which makes diagnosing anything going on there easier, as we
won't have to look at a deluge of e.g. GIT_TEST_* variables.

For the "pedantic" job (which has no test phase) we won't run the
"ci/lib.sh --test" step, which will be clearly visible as a skipped
step in the UX.

Since we'll now always run "--build" for "make" and "--test" for "make
test" we can stop setting the "fat" MAKEFLAGS entirely on the "test"
steps, i.e. the one with PYTHON_PATH, CC etc. These will all be
carried over from the earlier --build step.

That won't be true in the case of the "windows-test" and "vs-test"
jobs, since they run separately from the corresponding "build"
step. So we'll need to make sure that we have the --jobs=N argument
for those.

This doesn't matter that much, as we'll still have --jobs=N in
GIT_PROVE_OPTS. So the only thing we'll use it for is parallelism in
the t/Makefile before we get to running "prove". Still, it's good to
be consistent for good measure, and to run the t/Makefile itself in
parallel.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 21 ++++++++-------
 ci/lib.sh                  | 52 +++++++++++++++++++++++++++++++++++---
 2 files changed, 60 insertions(+), 13 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 83e0aa1f469..bb62b4ff725 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -84,7 +84,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
       shell: bash
     - name: build
       shell: bash
@@ -122,7 +122,7 @@ jobs:
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
-    - run: ci/lib.sh
+    - run: ci/lib.sh --test
       shell: bash
     - name: select tests
       run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
@@ -169,7 +169,7 @@ jobs:
     - name: copy dlls to root
       shell: cmd
       run: compat\vcbuild\vcpkg_copy_dlls.bat release
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
       shell: bash
     - name: generate Visual Studio solution
       shell: bash
@@ -211,7 +211,7 @@ jobs:
     - name: extract tracked files and build artifacts
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
-    - run: ci/lib.sh
+    - run: ci/lib.sh --test
       shell: bash
     - name: select tests
       run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
@@ -275,8 +275,9 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make
+    - run: ci/lib.sh --test
     - run: make test
       if: success()
     - run: ci/print-test-failures.sh
@@ -310,8 +311,10 @@ jobs:
     steps:
     - uses: actions/checkout@v1
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make
+    - run: ci/lib.sh --test
+      if: success() && matrix.vector.skip-tests != 'yes'
     - run: make test
       if: success() && matrix.vector.skip-tests != 'yes'
     - run: ci/print-test-failures.sh
@@ -331,7 +334,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make ci-static-analysis
   sparse:
     needs: ci-config
@@ -352,7 +355,7 @@ jobs:
     - uses: actions/checkout@v2
     - name: Install other dependencies
       run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make sparse
   documentation:
     name: documentation
@@ -364,7 +367,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make check-docs
     - run: "make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)"
       shell: bash
diff --git a/ci/lib.sh b/ci/lib.sh
index fea45a1a8d3..c875208817f 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -4,6 +4,30 @@ set -ex
 # Helper libraries
 . ${0%/*}/lib-ci-type.sh
 
+# Parse options
+mode_build=
+mode_test=
+while test $# != 0
+do
+	case "$1" in
+	--build)
+		mode_build=t
+		;;
+	--test)
+		mode_test=t
+		;;
+	-*)
+		echo "error: invalid option: $1" >&2
+		exit 1
+		;;
+	*)
+		echo "error: invalid argument: $1" >&2
+		exit 1
+		;;
+	esac
+	shift
+done
+
 # Starting assertions
 if test -z "$jobname"
 then
@@ -11,16 +35,28 @@ then
 	exit 1
 fi
 
+if test "$mode_test$mode_build" != "t"
+then
+	echo "error: need one mode, e.g. --build or --test" >&2
+	exit 1
+fi
+
 # Helper functions
 setenv () {
 	while test $# != 0
 	do
 		case "$1" in
 		--build)
+			if test -z "$mode_build"
+			then
+				return 0
+			fi
 			;;
 		--test)
-			;;
-		--all)
+			if test -z "$mode_test"
+			then
+				return 0
+			fi
 			;;
 		-*)
 			echo "BUG: bad setenv() option '$1'" >&2
@@ -46,8 +82,12 @@ setenv () {
 # How many jobs to run in parallel?
 NPROC=10
 
+# For "--test" we carry the MAKEFLAGS over from earlier steps, except
+# in stand-alone jobs which will use $COMMON_MAKEFLAGS.
+COMMON_MAKEFLAGS=--jobs=$NPROC
+
 # Clear MAKEFLAGS that may come from the outside world.
-MAKEFLAGS=--jobs=$NPROC
+MAKEFLAGS=$COMMON_MAKEFLAGS
 
 case "$CI_TYPE" in
 github-actions)
@@ -101,6 +141,9 @@ windows-build)
 	setenv --build NO_PERL NoThanks
 	setenv --build ARTIFACTS_DIRECTORY artifacts
 	;;
+windows-test)
+	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
+	;;
 vs-build)
 	setenv --build NO_PERL NoThanks
 	setenv --build NO_GETTEXT NoThanks
@@ -117,6 +160,7 @@ vs-build)
 	;;
 vs-test)
 	setenv --test NO_SVN_TESTS YesPlease
+	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
 	;;
 linux-gcc)
 	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
@@ -162,4 +206,4 @@ linux-leaks)
 	;;
 esac
 
-setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
+setenv --build MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
-- 
2.36.0.879.g3659959fcca

[PATCH v5 24/29] CI: add more variables to MAKEFLAGS, except under vs-build

[Ævar Arnfjörð Bjarmason]

It's clearer that "DEVELOPER" is a flag that affects the Makefile
itself in particular if it's put into "MAKEFLAGS than" if it
generically sits in the environment. Let's move both it and
"SKIP_DASHED_BUILT_INS" to "MAKEFLAGS".

We can't do this under vs-build, since that invokes cmake. Let's have
only that job set these in the environment.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index c875208817f..c8cc37c50b4 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -89,6 +89,13 @@ COMMON_MAKEFLAGS=--jobs=$NPROC
 # Clear MAKEFLAGS that may come from the outside world.
 MAKEFLAGS=$COMMON_MAKEFLAGS
 
+# Use common options for "make" (cmake in "vs-build" below uses the
+# intermediate variables directly)
+DEVELOPER=1
+MAKEFLAGS="$MAKEFLAGS DEVELOPER=$DEVELOPER"
+SKIP_DASHED_BUILT_INS=YesPlease
+MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
+
 case "$CI_TYPE" in
 github-actions)
 	CC="${CC:-gcc}"
@@ -105,10 +112,8 @@ github-actions)
 	;;
 esac
 
-setenv --build DEVELOPER 1
 setenv --test DEFAULT_TEST_TARGET prove
 setenv --test GIT_TEST_CLONE_2GB true
-setenv --build SKIP_DASHED_BUILT_INS YesPlease
 
 case "$runs_on_pool" in
 ubuntu-latest)
@@ -145,6 +150,9 @@ windows-test)
 	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
 	;;
 vs-build)
+	setenv --build DEVELOPER $DEVELOPER
+	setenv --build SKIP_DASHED_BUILT_INS $SKIP_DASHED_BUILT_INS
+
 	setenv --build NO_PERL NoThanks
 	setenv --build NO_GETTEXT NoThanks
 	setenv --build ARTIFACTS_DIRECTORY artifacts
-- 
2.36.0.879.g3659959fcca

[PATCH v5 25/29] CI: set CC in MAKEFLAGS directly, don't add it to the environment

[Ævar Arnfjörð Bjarmason]

Rather than pass a "$CC" in the environment to be picked up in
ci/lib.sh let's instead have ci/lib.sh itself add it directly to
MAKEFLAGS. For "$CC_PACKAGE" its setting and use can stay within
ci/install-dependencies.sh.

Setting CC=gcc by default made for confusing trace output, and since a
preceding change to carry it and others over across "steps" in the
GitHub CI it's been even more misleading.  E.g. the "win+VS build" job
confusingly has CC=gcc set, even though it builds with MSVC.

Let's instead rely on the Makefile default of CC=cc, and only override
it for those jobs where it's needed. This does mean that we'll need to
set it for the "pedantic" job, which previously relied on the default
CC=gcc in case "clang" become the default on that platform.

This partially reverts my 707d2f2fe86 (CI: use "$runs_on_pool", not
"$jobname" to select packages & config, 2021-11-23), i.e. we're now
aiming to only set those variables specific jobs need.

Since we don't need to make this generic enough to handle "gcc-9"
turning into "gcc@9" for "brew install" let's remove what was
originally a bash-specific replacement in 707d2f2fe86 (CI: use
"$runs_on_pool", not "$jobname" to select packages & config,
2021-11-23), and which a preceding commit changed to a "tr"
invocation.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 13 -------------
 ci/install-dependencies.sh | 18 ++++++++++++++----
 ci/lib.sh                  | 23 ++++++++++++++++++++---
 3 files changed, 34 insertions(+), 20 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index bb62b4ff725..6835e942280 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -238,37 +238,24 @@ jobs:
       matrix:
         vector:
           - jobname: linux-clang
-            cc: clang
             pool: ubuntu-latest
           - jobname: linux-sha256
-            cc: clang
             os: ubuntu
             pool: ubuntu-latest
           - jobname: linux-gcc
-            cc: gcc
-            cc_package: gcc-8
             pool: ubuntu-latest
           - jobname: linux-TEST-vars
-            cc: gcc
             os: ubuntu
-            cc_package: gcc-8
             pool: ubuntu-latest
           - jobname: osx-clang
-            cc: clang
             pool: macos-latest
           - jobname: osx-gcc
-            cc: gcc
-            cc_package: gcc-9
             pool: macos-latest
           - jobname: linux-gcc-default
-            cc: gcc
             pool: ubuntu-latest
           - jobname: linux-leaks
-            cc: gcc
             pool: ubuntu-latest
     env:
-      CC: ${{matrix.vector.cc}}
-      CC_PACKAGE: ${{matrix.vector.cc_package}}
       jobname: ${{matrix.vector.jobname}}
       runs_on_pool: ${{matrix.vector.pool}}
     runs-on: ${{matrix.vector.pool}}
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 4b4cdbe26e0..65548bf0b25 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -9,6 +9,17 @@ UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
  tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
  libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl"
 
+CC_PACKAGE=
+BREW_CC_PACKAGE=
+case "$jobname" in
+linux-gcc | linux-TEST-vars)
+	CC_PACKAGE=gcc-8
+	;;
+osx-gcc)
+	BREW_CC_PACKAGE=gcc@9
+	;;
+esac
+
 case "$runs_on_pool" in
 ubuntu-latest)
 	# The Linux build installs the defined dependency versions below.
@@ -67,11 +78,10 @@ macos-latest)
 	} ||
 	brew install homebrew/cask/perforce
 
-	if test -n "$CC_PACKAGE"
+	if test -n "$BREW_CC_PACKAGE"
 	then
-		BREW_PACKAGE=$(echo $CC_PACKAGE | tr '-' '@')
-		brew install "$BREW_PACKAGE"
-		brew link "$BREW_PACKAGE"
+		brew install "$BREW_CC_PACKAGE"
+		brew link "$BREW_CC_PACKAGE"
 	fi
 	;;
 esac
diff --git a/ci/lib.sh b/ci/lib.sh
index c8cc37c50b4..8d19ca5ced8 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -79,6 +79,9 @@ setenv () {
 	fi
 }
 
+# Clear variables that may come from the outside world.
+CC=
+
 # How many jobs to run in parallel?
 NPROC=10
 
@@ -98,8 +101,6 @@ MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
 
 case "$CI_TYPE" in
 github-actions)
-	CC="${CC:-gcc}"
-
 	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
 	GIT_TEST_OPTS="--verbose-log -x"
 	test Windows != "$RUNNER_OS" ||
@@ -171,9 +172,14 @@ vs-test)
 	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
 	;;
 linux-gcc)
+	CC=gcc
 	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
 	;;
+linux-gcc-default)
+	CC=gcc
+	;;
 linux-TEST-vars)
+	CC=gcc
 	setenv --test GIT_TEST_SPLIT_INDEX yes
 	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
 	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
@@ -188,13 +194,22 @@ linux-TEST-vars)
 	setenv --test GIT_TEST_WRITE_REV_INDEX 1
 	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
 	;;
+osx-gcc)
+	CC=gcc
+	;;
+osx-clang)
+	CC=clang
+	;;
 linux-clang)
+	CC=clang
 	setenv --test GIT_TEST_DEFAULT_HASH sha1
 	;;
 linux-sha256)
+	CC=clang
 	setenv --test GIT_TEST_DEFAULT_HASH sha256
 	;;
 pedantic)
+	CC=gcc
 	# Don't run the tests; we only care about whether Git can be
 	# built.
 	setenv --build DEVOPTS pedantic
@@ -209,9 +224,11 @@ linux-musl)
 	MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8"
 	;;
 linux-leaks)
+	CC=gcc
 	setenv --build SANITIZE leak
 	setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true
 	;;
 esac
 
-setenv --build MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
+MAKEFLAGS="$MAKEFLAGS${CC:+ CC=$CC}"
+setenv --build MAKEFLAGS "$MAKEFLAGS"
-- 
2.36.0.879.g3659959fcca

[PATCH v5 26/29] CI: set SANITIZE=leak in MAKEFLAGS directly

[Ævar Arnfjörð Bjarmason]

As in the preceding commit's change to do likewise for CC, let's not
set SANITIZE in the environment. When I added the "linux-leaks" job in
in 956d2e4639b (tests: add a test mode for SANITIZE=leak, run it in
CI, 2021-09-23) I missed that I could do that.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 8d19ca5ced8..b6e7a6647bb 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -225,7 +225,7 @@ linux-musl)
 	;;
 linux-leaks)
 	CC=gcc
-	setenv --build SANITIZE leak
+	MAKEFLAGS="$MAKEFLAGS SANITIZE=leak"
 	setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true
 	;;
 esac
-- 
2.36.0.879.g3659959fcca

[PATCH v5 27/29] CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case

[Ævar Arnfjörð Bjarmason]

Move the setting of "PYTHON_PATH=[...]" to the "$jobname" case
statement. This partially backs out of my 707d2f2fe86 (CI: use
"$runs_on_pool", not "$jobname" to select packages & config,
2021-11-23), now that we have a "osx-{clang,gcc}" anyway for setting
"$CC" we might as well do away with this part of the "$runs_on_pool"
case.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index b6e7a6647bb..f2f75901a9c 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -132,14 +132,6 @@ ubuntu-latest)
 
 	setenv --test GIT_TEST_HTTPD true
 	;;
-macos-latest)
-	if test "$jobname" = osx-gcc
-	then
-		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
-	else
-		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
-	fi
-	;;
 esac
 
 case "$jobname" in
@@ -195,9 +187,11 @@ linux-TEST-vars)
 	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
 	;;
 osx-gcc)
+	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
 	CC=gcc
 	;;
 osx-clang)
+	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
 	CC=clang
 	;;
 linux-clang)
-- 
2.36.0.879.g3659959fcca

[PATCH v5 28/29] CI: don't use "set -x" in "ci/lib.sh" output

[Ævar Arnfjörð Bjarmason]

Remove the "set -x" from the "ci/lib.sh" output. Before preceding
commits the logic in that file was much more complex, and likely to
fail in some scenarios.

Now we only task "ci/lib.sh" with setting various variables for
subsequent steps in our jobs, so we can start emitting more tailored
debugging output, which makes what it's doing easier to read.

This change also changes the output of the "ci/print-test-failures.sh"
script, since it's the only other user of "ci/lib-ci-type.sh". In that
case it's also helpful to know what "$CI_TYPE" we're using, as that
script doesn't "set -x" and will act differently depending on the
$CI_TYPE.

It's useful when developing this script to see what variables
it *doesn't* set, but such output isn't useful to users, so let's hide
the "SKIP" output behind a --debug option.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 35 ++++++++++++++++++++++++++++++++---
 1 file changed, 32 insertions(+), 3 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index f2f75901a9c..9e38c893890 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,5 +1,5 @@
 #!/bin/sh
-set -ex
+set -e
 
 # Helper libraries
 . ${0%/*}/lib-ci-type.sh
@@ -7,15 +7,22 @@ set -ex
 # Parse options
 mode_build=
 mode_test=
+mode_debug=
 while test $# != 0
 do
 	case "$1" in
 	--build)
+		echo "MODE: $1" >&2
 		mode_build=t
 		;;
 	--test)
+		echo "MODE: $1" >&2
 		mode_test=t
 		;;
+	--debug)
+		echo "DEBUG: $1" >&2
+		mode_debug=t
+		;;
 	-*)
 		echo "error: invalid option: $1" >&2
 		exit 1
@@ -41,21 +48,32 @@ then
 	exit 1
 fi
 
+# Show our configuration
+echo "CONFIG: CI_TYPE=$CI_TYPE" >&2
+echo "CONFIG: jobname=$jobname" >&2
+echo "CONFIG: runs_on_pool=$runs_on_pool" >&2
+if test -n "$GITHUB_ENV"
+then
+	echo "CONFIG: GITHUB_ENV=$GITHUB_ENV" >&2
+fi
+echo >&2
+
 # Helper functions
 setenv () {
+	skip=
 	while test $# != 0
 	do
 		case "$1" in
 		--build)
 			if test -z "$mode_build"
 			then
-				return 0
+				skip=t
 			fi
 			;;
 		--test)
 			if test -z "$mode_test"
 			then
-				return 0
+				skip=t
 			fi
 			;;
 		-*)
@@ -73,10 +91,21 @@ setenv () {
 	val=$2
 	shift 2
 
+	if test -n "$skip"
+	then
+		if test -n "$mode_debug"
+		then
+			echo "SKIP: '$key=$val'" >&2
+		fi
+		return 0
+	fi
+
 	if test -n "$GITHUB_ENV"
 	then
 		echo "$key=$val" >>"$GITHUB_ENV"
 	fi
+
+	echo "SET: '$key=$val'" >&2
 }
 
 # Clear variables that may come from the outside world.
-- 
2.36.0.879.g3659959fcca

[PATCH v5 29/29] CI: make it easy to use ci/*.sh outside of CI

[Ævar Arnfjörð Bjarmason]

In preceding commits the ci/.sh scripts have lost most of their
CI-specific assumptions. Let's go even further and explicitly support
running ci/lib.sh outside of CI.

This was possible before by faking up enough CI-specific variables,
but as shown in the new "help" output being added here using the
ci/lib.sh to provide "CI-like" has now become trivial.

The ci/print-test-failures.sh scripts can now be used outside of CI as
well, the only GitHub CI-specific part is now guarded by a check that
we'll pass if outside of GitHub CI.

There's also a special-case here to not clobber $MAKEFLAGS in the
environment if we're outside of CI, in case the user has e.g. "jN" or
other flags to "make" that they'd prefer configured already.

Using "ci/lib.sh" as a stand-alone script is much more useful if it
doesn't hardcode NPROC=10, let's provide a poor shellscript
replacement for the online_cpus() we have in thread-utils.c to cover
the most common OS's.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib-ci-type.sh         |  3 --
 ci/lib-online_cpus.sh     | 26 +++++++++++++
 ci/lib.sh                 | 80 ++++++++++++++++++++++++++++++++++-----
 ci/print-test-failures.sh |  4 --
 4 files changed, 96 insertions(+), 17 deletions(-)
 create mode 100644 ci/lib-online_cpus.sh

diff --git a/ci/lib-ci-type.sh b/ci/lib-ci-type.sh
index 6f01fd9e5d9..09acab7aaec 100644
--- a/ci/lib-ci-type.sh
+++ b/ci/lib-ci-type.sh
@@ -3,7 +3,4 @@
 if test "$GITHUB_ACTIONS" = "true"
 then
 	CI_TYPE=github-actions
-else
-	echo "Could not identify CI type" >&2
-	exit 1
 fi
diff --git a/ci/lib-online_cpus.sh b/ci/lib-online_cpus.sh
new file mode 100644
index 00000000000..7cf0cd042fa
--- /dev/null
+++ b/ci/lib-online_cpus.sh
@@ -0,0 +1,26 @@
+#!/bin/sh
+
+# TODO: Ideally we'd compile t/helper/test-online-cpus.c early, but
+# that currently presents a chicken & egg problem. We need this before
+# we build (much of) anything.
+online_cpus() {
+	NPROC=
+
+	if command -v nproc >/dev/null
+	then
+		# GNU coreutils
+		NPROC=$(nproc)
+	elif command -v sysctl >/dev/null
+	then
+		# BSD & Mac OS X
+		NPROC=$(sysctl -n hw.ncpu)
+	elif test -n "$NUMBER_OF_PROCESSORS"
+	then
+		# Windows
+		NPROC="$NUMBER_OF_PROCESSORS"
+	else
+		NPROC=1
+	fi
+
+	echo $NPROC
+}
diff --git a/ci/lib.sh b/ci/lib.sh
index 9e38c893890..80e89f89b7f 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,16 +1,55 @@
 #!/bin/sh
 set -e
 
+# Usage
+CI_TYPE_HELP_COMMANDS='
+	# run "make all test" like the "linux-leaks" job
+	(eval $(jobname=linux-leaks ci/lib.sh --all) && make test)
+
+	# run "make all test" like the "linux-musl" job
+	(eval $(jobname=linux-musl ci/lib.sh --all) && make test)
+
+	# run "make test" like the "linux-TEST-vars" job (uses various GIT_TEST_* modes)
+	make && (eval $(jobname=linux-TEST-vars ci/lib.sh --test) && make test)
+
+	# run "make test" like the "linux-sha256" job
+	make && (eval $(jobname=linux-sha256 ci/lib.sh --test) && make test)
+'
+
+CI_TYPE_HELP="
+Running $0 outside of CI? You can use ci/lib.sh to set up your
+environment like a given CI job. E.g.:
+$CI_TYPE_HELP_COMMANDS
+Note that some of these (e.g. the linux-musl one) may not work as
+expected due to the CI job configuring a platform that may not match
+yours."
+
+usage() {
+	echo "usage: jobname=<job-name> [runs_on_pool=<pool-name>] $0 (--build | --test)" >&2
+	if test -z "$CI_TYPE"
+	then
+		echo "$CI_TYPE_HELP" >&2
+	fi
+	exit 1
+}
+
 # Helper libraries
 . ${0%/*}/lib-ci-type.sh
 
 # Parse options
+mode_all=
 mode_build=
 mode_test=
 mode_debug=
 while test $# != 0
 do
 	case "$1" in
+	--all)
+		echo "MODE: $1" >&2
+		mode_all=t
+		mode_build=t
+		mode_test=t
+		;;
 	--build)
 		echo "MODE: $1" >&2
 		mode_build=t
@@ -25,11 +64,11 @@ do
 		;;
 	-*)
 		echo "error: invalid option: $1" >&2
-		exit 1
+		usage
 		;;
 	*)
 		echo "error: invalid argument: $1" >&2
-		exit 1
+		usage
 		;;
 	esac
 	shift
@@ -39,13 +78,13 @@ done
 if test -z "$jobname"
 then
 	echo "error: must set a CI jobname in the environment" >&2
-	exit 1
+	usage
 fi
 
-if test "$mode_test$mode_build" != "t"
+if test "$mode_test$mode_build" != "t" && test -z "$mode_all"
 then
 	echo "error: need one mode, e.g. --build or --test" >&2
-	exit 1
+	usage
 fi
 
 # Show our configuration
@@ -65,13 +104,13 @@ setenv () {
 	do
 		case "$1" in
 		--build)
-			if test -z "$mode_build"
+			if test -z "$mode_build$mode_all"
 			then
 				skip=t
 			fi
 			;;
 		--test)
-			if test -z "$mode_test"
+			if test -z "$mode_test$mode_all"
 			then
 				skip=t
 			fi
@@ -103,6 +142,10 @@ setenv () {
 	if test -n "$GITHUB_ENV"
 	then
 		echo "$key=$val" >>"$GITHUB_ENV"
+	elif test -z "$CI_TYPE"
+	then
+		echo "$key=\"$val\""
+		echo "export $key"
 	fi
 
 	echo "SET: '$key=$val'" >&2
@@ -113,10 +156,25 @@ CC=
 
 # How many jobs to run in parallel?
 NPROC=10
+case "$CI_TYPE" in
+'')
+	. ${0%/*}/lib-online_cpus.sh
+	NPROC=$(online_cpus)
+
+	if test -n "$MAKEFLAGS"
+	then
+		COMMON_MAKEFLAGS="$MAKEFLAGS"
+	else
+		COMMON_MAKEFLAGS=--jobs=$NPROC
+	fi
+	;;
+*)
+	# For "--test" we carry the MAKEFLAGS over from earlier steps, except
+	# in stand-alone jobs which will use $COMMON_MAKEFLAGS.
+	COMMON_MAKEFLAGS=--jobs=$NPROC
+	;;
+esac
 
-# For "--test" we carry the MAKEFLAGS over from earlier steps, except
-# in stand-alone jobs which will use $COMMON_MAKEFLAGS.
-COMMON_MAKEFLAGS=--jobs=$NPROC
 
 # Clear MAKEFLAGS that may come from the outside world.
 MAKEFLAGS=$COMMON_MAKEFLAGS
@@ -136,6 +194,8 @@ github-actions)
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
 	;;
+'')
+	;;
 *)
 	echo "Unhandled CI type: $CI_TYPE" >&2
 	exit 1
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 29f8c332eca..ec395c79ccd 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -36,10 +36,6 @@ do
 			cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
 			tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
 			;;
-		*)
-			echo "Unhandled CI type: $CI_TYPE" >&2
-			exit 1
-			;;
 		esac
 	fi
 done
-- 
2.36.0.879.g3659959fcca

[PATCH v6 00/29] CI: run "make" in CI "steps", improve UX

[Ævar Arnfjörð Bjarmason]

This series changes CI "step" targets that are shellscripts that do N
things to instead be single command invocations at the "step" level,
driven by the CI recipe itself.

To do that we need to pass state that we previously re-setup for every
"step" via $GITHUB_ENV, whose state we then helpfully show (this is just
a standard GitHub CI feature) in a drop-down at the start of every
"step".

I.e. at the tip of this series you can reliably look at that
$GITHUB_ENV view to see what the full and relevant environment was for
that "make", "make test" or whatever.

The only changes since v5 are to rebase this on an updated "master",
which had some small fixes in the ci/ directory. For the last
iteration see:
https://lore.kernel.org/git/cover-v5-00.29-00000000000-20220421T181526Z-avarab@gmail.com/

I'll follow this up with a re-roll of
https://lore.kernel.org/git/RFC-cover-v5-00.10-00000000000-20220421T183001Z-avarab@gmail.com/
which is a replacement for
https://lore.kernel.org/git/pull.1117.v3.git.1653171536.gitgitgadget@gmail.com/
based on this topic.

To view the CI output differences this topic makes see these two runs:

  master: https://github.com/avar/git/actions/runs/2377697338
  this: https://github.com/avar/git/actions/runs/2377257487

These runs have a commit on top to trigger CI build and test failures
in various areas. One example is here:

 master: https://github.com/avar/git/runs/6573059000?check_suite_focus=true
 this: https://github.com/avar/git/runs/6571804790?check_suite_focus=true

This shows that we fail on "make" instead of a monolothic
"ci/run-build-and-test.sh" step. Furthermore we can see that a
preceding "ci/lib.sh --build" step has been added to set & export (via
$GITHUB_ENV) the full set of variables "make" will use in the failing
step. Those are then visible at (click to expand it):

  https://github.com/avar/git/runs/6571804790?check_suite_focus=true#step:5:1

For every step we now do the same, i.e. it's immediately obvious what
parameters go into running that step. The last step in this series
then makes it easy to run builds and tests "like CI" using the same
tooling:

	$ ci/lib.sh 
	error: must set a CI jobname in the environment
	usage: jobname=<job-name> [runs_on_pool=<pool-name>] ci/lib.sh (--build | --test)

	Running ci/lib.sh outside of CI? You can use ci/lib.sh to set up your
	environment like a given CI job. E.g.:

	        # run "make all test" like the "linux-leaks" job
	        (eval $(jobname=linux-leaks ci/lib.sh --all) && make test)

	        # run "make all test" like the "linux-musl" job
	        (eval $(jobname=linux-musl ci/lib.sh --all) && make test)

	        # run "make test" like the "linux-TEST-vars" job (uses various GIT_TEST_* modes)
	        make && (eval $(jobname=linux-TEST-vars ci/lib.sh --test) && make test)

	        # run "make test" like the "linux-sha256" job
	        make && (eval $(jobname=linux-sha256 ci/lib.sh --test) && make test)

	Note that some of these (e.g. the linux-musl one) may not work as
	expected due to the CI job configuring a platform that may not match
	yours.

As noted in the usage this can then be directly eval'd:

	$ jobname=osx-gcc ci/lib.sh --build 2>/dev/null 
	MAKEFLAGS="j8 DEVELOPER=1 SKIP_DASHED_BUILT_INS=YesPlease PYTHON_PATH=/usr/bin/python3 CC=gcc"
	export MAKEFLAGS

Of course YMMV when doing so, running that on my Linux box won't
magically give me an OSX environment. But it's very useful to be able
to to e.g.:

	jobname=linux-TEST-vars ci/lib.sh --test

Which allows myself and others to tweak our local build scripts to
test things locally with setups that mirror these CI setups as far as
GIT_TEST_* etc. parameters are concerned.

Ævar Arnfjörð Bjarmason (29):
  CI: run "set -ex" early in ci/lib.sh
  CI: make "$jobname" explicit, remove fallback
  CI: remove more dead Travis CI support
  CI: remove dead "tree skipping" code
  CI: remove unused Azure ci/* code
  CI/lib.sh: stop adding leading whitespace to $MAKEFLAGS
  CI: don't have "git grep" invoke a pager in tree content check
  CI: have "static-analysis" run a "make ci-static-analysis" target
  CI: have "static-analysis" run "check-builtins", not "documentation"
  CI: move p4 and git-lfs variables to ci/install-dependencies.sh
  CI: consistently use "export" in ci/lib.sh
  CI: export variables via a wrapper
  CI: remove "run-build-and-tests.sh", run "make [test]" directly
  CI: make ci/{lib,install-dependencies}.sh POSIX-compatible
  CI: check ignored unignored build artifacts in "win[+VS] build" too
  CI: invoke "make artifacts-tar" directly in windows-build
  CI: split up and reduce "ci/test-documentation.sh"
  CI: combine ci/install{,-docker}-dependencies.sh
  CI: move "env" definitions into ci/lib.sh
  ci/run-test-slice.sh: replace shelling out with "echo"
  CI: pre-select test slice in Windows & VS tests
  CI: only invoke ci/lib.sh as "steps" in main.yml
  CI: narrow down variable definitions in --build and --test
  CI: add more variables to MAKEFLAGS, except under vs-build
  CI: set CC in MAKEFLAGS directly, don't add it to the environment
  CI: set SANITIZE=leak in MAKEFLAGS directly
  CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case
  CI: don't use "set -x" in "ci/lib.sh" output
  CI: make it easy to use ci/*.sh outside of CI

 .github/workflows/main.yml            | 101 +++---
 Makefile                              |  31 +-
 ci/check-directional-formatting.bash  |   2 +-
 ci/check-unignored-build-artifacts.sh |  20 ++
 ci/install-dependencies.sh            |  82 +++--
 ci/install-docker-dependencies.sh     |  22 --
 ci/lib-ci-type.sh                     |   6 +
 ci/lib-online_cpus.sh                 |  26 ++
 ci/lib-tput.sh                        |   5 +
 ci/lib.sh                             | 424 ++++++++++++++++----------
 ci/make-test-artifacts.sh             |  12 -
 ci/mount-fileshare.sh                 |  25 --
 ci/print-test-failures.sh             |  53 +---
 ci/run-build-and-tests.sh             |  54 ----
 ci/run-docker-build.sh                |  66 ----
 ci/run-docker.sh                      |  47 ---
 ci/run-static-analysis.sh             |  32 --
 ci/run-test-slice.sh                  |  17 --
 ci/select-test-slice.sh               |  13 +
 ci/test-documentation.sh              |  37 +--
 ci/util/extract-trash-dirs.sh         |  50 ---
 shared.mak                            |   1 +
 22 files changed, 508 insertions(+), 618 deletions(-)
 create mode 100755 ci/check-unignored-build-artifacts.sh
 delete mode 100755 ci/install-docker-dependencies.sh
 create mode 100644 ci/lib-ci-type.sh
 create mode 100644 ci/lib-online_cpus.sh
 create mode 100644 ci/lib-tput.sh
 delete mode 100755 ci/make-test-artifacts.sh
 delete mode 100755 ci/mount-fileshare.sh
 delete mode 100755 ci/run-build-and-tests.sh
 delete mode 100755 ci/run-docker-build.sh
 delete mode 100755 ci/run-docker.sh
 delete mode 100755 ci/run-static-analysis.sh
 delete mode 100755 ci/run-test-slice.sh
 create mode 100755 ci/select-test-slice.sh
 delete mode 100755 ci/util/extract-trash-dirs.sh

Range-diff against v5:
 1:  84d8098e365 =  1:  25cb08ad06c CI: run "set -ex" early in ci/lib.sh
 2:  3f5c071b0f6 =  2:  1be7910c728 CI: make "$jobname" explicit, remove fallback
 3:  c64845a4efb !  3:  234e248e831 CI: remove more dead Travis CI support
    @@ ci/lib.sh: then
     -	test macos != "$CI_OS_NAME" || CI_OS_NAME=osx
     -	CI_REPO_SLUG="$GITHUB_REPOSITORY"
      	CI_JOB_ID="$GITHUB_RUN_ID"
    - 	CC="${CC:-gcc}"
    + 	CC="${CC_PACKAGE:-${CC:-gcc}}"
      	DONT_SKIP_TAGS=t
     @@ ci/lib.sh: then
      	export GIT_PROVE_OPTS="--timer --jobs 10"
 4:  45856e67d5d !  4:  36a6287ee9b CI: remove dead "tree skipping" code
    @@ ci/lib.sh: then
     -	CI_BRANCH="$GITHUB_REF"
     -	CI_COMMIT="$GITHUB_SHA"
     -	CI_JOB_ID="$GITHUB_RUN_ID"
    - 	CC="${CC:-gcc}"
    + 	CC="${CC_PACKAGE:-${CC:-gcc}}"
     -	DONT_SKIP_TAGS=t
     -
     -	cache_dir="$HOME/none"
 5:  d4671148eb7 !  5:  66b0a383daf CI: remove unused Azure ci/* code
    @@ ci/install-dependencies.sh: macos-latest)
     -	test -z "$BREW_INSTALL_PACKAGES" ||
     -	brew install $BREW_INSTALL_PACKAGES
      	brew link --force gettext
    - 	brew install --cask --no-quarantine perforce || {
    - 		# Update the definitions and try again
    + 	mkdir -p $HOME/bin
    + 	(
     @@ ci/install-dependencies.sh: Documentation)
      	sudo apt-get -q update
      	sudo apt-get -q -y install asciidoc xmlto docbook-xsl-ns make
    @@ ci/lib.sh: export TERM=${TERM:-dumb}
     +if test "$GITHUB_ACTIONS" = "true"
      then
      	CI_TYPE=github-actions
    - 	CC="${CC:-gcc}"
    + 	CC="${CC_PACKAGE:-${CC:-gcc}}"
     
      ## ci/mount-fileshare.sh (deleted) ##
     @@
 6:  45ea80d0495 !  6:  d96333c1bd8 CI/lib.sh: stop adding leading whitespace to $MAKEFLAGS
    @@ ci/lib.sh: check_unignored_build_artifacts ()
      if test "$GITHUB_ACTIONS" = "true"
      then
      	CI_TYPE=github-actions
    - 	CC="${CC:-gcc}"
    + 	CC="${CC_PACKAGE:-${CC:-gcc}}"
      
     -	export GIT_PROVE_OPTS="--timer --jobs 10"
     +	export GIT_PROVE_OPTS="--timer --jobs $NPROC"
 7:  a264b40ba0d =  7:  d0780ab4a96 CI: don't have "git grep" invoke a pager in tree content check
 8:  4a58f7be8d4 =  8:  84e368736e2 CI: have "static-analysis" run a "make ci-static-analysis" target
 9:  78c0c81af43 =  9:  1a9c98b909c CI: have "static-analysis" run "check-builtins", not "documentation"
10:  07e36035a1e ! 10:  e208a9ab1e2 CI: move p4 and git-lfs variables to ci/install-dependencies.sh
    @@ ci/install-dependencies.sh
      
      . ${0%/*}/lib.sh
      
    --P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
    +-P4WHENCE=https://cdist2.perforce.com/perforce/r$LINUX_P4_VERSION
     -LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
      UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
       tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
    @@ ci/install-dependencies.sh
     +	GIT_LFS_PATH="$HOME/custom/git-lfs"
     +	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
     +
    -+	P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
    ++	P4WHENCE=https://cdist2.perforce.com/perforce/r$LINUX_P4_VERSION
     +	LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
     +
      	sudo apt-get -q update
11:  272bd590a95 ! 11:  9655bae9e21 CI: consistently use "export" in ci/lib.sh
    @@ ci/lib.sh: export TERM=${TERM:-dumb}
      if test "$GITHUB_ACTIONS" = "true"
      then
     @@ ci/lib.sh: then
    - 	CC="${CC:-gcc}"
    + 	CC="${CC_PACKAGE:-${CC:-gcc}}"
      
      	export GIT_PROVE_OPTS="--timer --jobs $NPROC"
     -	export GIT_TEST_OPTS="--verbose-log -x"
12:  ea35327e24c ! 12:  2da0aa1d40e CI: export variables via a wrapper
    @@ ci/lib.sh: check_unignored_build_artifacts ()
      NPROC=10
     @@ ci/lib.sh: then
      	CI_TYPE=github-actions
    - 	CC="${CC:-gcc}"
    + 	CC="${CC_PACKAGE:-${CC:-gcc}}"
      
     -	export GIT_PROVE_OPTS="--timer --jobs $NPROC"
     +	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
13:  f6ff81f48ab ! 13:  6e97633652e CI: remove "run-build-and-tests.sh", run "make [test]" directly
    @@ ci/install-dependencies.sh: ubuntu-latest)
     +		echo "$PATH" >>"$GITHUB_PATH"
     +	fi
      
    - 	P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
    + 	P4WHENCE=https://cdist2.perforce.com/perforce/r$LINUX_P4_VERSION
      	LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
     
      ## ci/lib.sh ##
14:  533dbc50c4f = 14:  0488e29859b CI: make ci/{lib,install-dependencies}.sh POSIX-compatible
15:  6044c2b383f = 15:  f7ac6c33044 CI: check ignored unignored build artifacts in "win[+VS] build" too
16:  4d1a9f88c32 = 16:  f89346f11eb CI: invoke "make artifacts-tar" directly in windows-build
17:  5c8f464a60f = 17:  7fa9c69e3ca CI: split up and reduce "ci/test-documentation.sh"
18:  a218be76819 ! 18:  7925b2610fc CI: combine ci/install{,-docker}-dependencies.sh
    @@ ci/install-dependencies.sh: linux-gcc-default)
     +	;;
      esac
      
    - if type p4d >/dev/null && type p4 >/dev/null
    + if type p4d >/dev/null 2>&1 && type p4 >/dev/null 2>&1
     
      ## ci/install-docker-dependencies.sh (deleted) ##
     @@
19:  83e21b4f501 = 19:  b00abc07637 CI: move "env" definitions into ci/lib.sh
20:  fd0e1610577 = 20:  bb5960355fe ci/run-test-slice.sh: replace shelling out with "echo"
21:  0c7cd9d64ba = 21:  1eeb2e8e7f0 CI: pre-select test slice in Windows & VS tests
22:  1b077665352 ! 22:  39979d9887d CI: only invoke ci/lib.sh as "steps" in main.yml
    @@ ci/lib.sh: setenv () {
     -	CI_TYPE=github-actions
     +case "$CI_TYPE" in
     +github-actions)
    - 	CC="${CC:-gcc}"
    + 	CC="${CC_PACKAGE:-${CC:-gcc}}"
      
      	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
     @@ ci/lib.sh: then
23:  1867da7eddb = 23:  e2b3bf032d6 CI: narrow down variable definitions in --build and --test
24:  a4e3a9d2c28 ! 24:  c6f47d52823 CI: add more variables to MAKEFLAGS, except under vs-build
    @@ ci/lib.sh: COMMON_MAKEFLAGS=--jobs=$NPROC
     +
      case "$CI_TYPE" in
      github-actions)
    - 	CC="${CC:-gcc}"
    + 	CC="${CC_PACKAGE:-${CC:-gcc}}"
     @@ ci/lib.sh: github-actions)
      	;;
      esac
25:  48d99cee809 ! 25:  377460e8024 CI: set CC in MAKEFLAGS directly, don't add it to the environment
    @@ ci/install-dependencies.sh: UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl
      ubuntu-latest)
      	# The Linux build installs the defined dependency versions below.
     @@ ci/install-dependencies.sh: macos-latest)
    - 	} ||
    - 	brew install homebrew/cask/perforce
    + 	PATH="$PATH:${HOME}/bin"
    + 	export PATH
      
     -	if test -n "$CC_PACKAGE"
     +	if test -n "$BREW_CC_PACKAGE"
    @@ ci/lib.sh: MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
      
      case "$CI_TYPE" in
      github-actions)
    --	CC="${CC:-gcc}"
    +-	CC="${CC_PACKAGE:-${CC:-gcc}}"
     -
      	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
      	GIT_TEST_OPTS="--verbose-log -x"
26:  07a6c087297 = 26:  98e320d5e67 CI: set SANITIZE=leak in MAKEFLAGS directly
27:  50f21ffdffe = 27:  282b7c89a4f CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case
28:  e35dff2bef3 = 28:  7a208927819 CI: don't use "set -x" in "ci/lib.sh" output
29:  13d8c6997c3 = 29:  4d8a8f3be07 CI: make it easy to use ci/*.sh outside of CI
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 01/29] CI: run "set -ex" early in ci/lib.sh

[Ævar Arnfjörð Bjarmason]

Change ci/lib.sh to run "set -ex" before it does anything else, before
this we were omitting a few "export" commands in our trace output that
happened before we enabled tracing.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 86e37da9bc5..57121dd664a 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,5 +1,11 @@
 # Library of functions shared by all CI scripts
 
+# Set 'exit on error' for all CI scripts to let the caller know that
+# something went wrong.
+# Set tracing executed commands, primarily setting environment variables
+# and installing dependencies.
+set -ex
+
 skip_branch_tip_with_tag () {
 	# Sometimes, a branch is pushed at the same time the tag that points
 	# at the same commit as the tip of the branch is pushed, and building
@@ -85,12 +91,6 @@ export TERM=${TERM:-dumb}
 # Clear MAKEFLAGS that may come from the outside world.
 export MAKEFLAGS=
 
-# Set 'exit on error' for all CI scripts to let the caller know that
-# something went wrong.
-# Set tracing executed commands, primarily setting environment variables
-# and installing dependencies.
-set -ex
-
 if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
 then
 	CI_TYPE=azure-pipelines
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 02/29] CI: make "$jobname" explicit, remove fallback

[Ævar Arnfjörð Bjarmason]

Remove the $jobname fallback last adjusted in b011fabd6e0 (ci/lib.sh:
encapsulate Travis-specific things, 2019-01-27).

This makes all of our CI jobs have an explicit $jobname, and removes
the special-case of the Windows jobs implicitly setting one (that they
don't currently use). In subsequent commits we'll use this new
$jobname explicitly.

This is a follow-up to my 25715419bf4 (CI: don't run "make test" twice
in one job, 2021-11-23), since then all the jobs except the Windows
jobs have had an explicit $jobname.

In the case of the Windows jobs we were using a jobname of windows-gcc
for both, due to picking "gcc" as a default fallback for "$CC" in
ci/lib.sh.

Since we'll need this to run ci/lib.sh it makes sense to move the
check to the top of the file. Subsequent commits will add more such
assertions to it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml |  8 ++++++++
 ci/lib.sh                  | 12 +++++++-----
 2 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index c35200defb9..309b9141249 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -79,6 +79,8 @@ jobs:
 
   windows-build:
     name: win build
+    env:
+      jobname: windows-build
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
     runs-on: windows-latest
@@ -100,6 +102,8 @@ jobs:
         path: artifacts
   windows-test:
     name: win test
+    env:
+      jobname: windows-test
     runs-on: windows-latest
     needs: [windows-build]
     strategy:
@@ -131,6 +135,8 @@ jobs:
         path: ${{env.FAILED_TEST_ARTIFACTS}}
   vs-build:
     name: win+VS build
+    env:
+      jobname: vs-build
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
     env:
@@ -183,6 +189,8 @@ jobs:
         path: artifacts
   vs-test:
     name: win+VS test
+    env:
+      jobname: vs-test
     runs-on: windows-latest
     needs: vs-build
     strategy:
diff --git a/ci/lib.sh b/ci/lib.sh
index 57121dd664a..3061de96d8f 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -6,6 +6,13 @@
 # and installing dependencies.
 set -ex
 
+# Starting assertions
+if test -z "$jobname"
+then
+	echo "error: must set a CI jobname in the environment" >&2
+	exit 1
+fi
+
 skip_branch_tip_with_tag () {
 	# Sometimes, a branch is pushed at the same time the tag that points
 	# at the same commit as the tip of the branch is pushed, and building
@@ -146,11 +153,6 @@ test -n "${DONT_SKIP_TAGS-}" ||
 skip_branch_tip_with_tag
 skip_good_tree
 
-if test -z "$jobname"
-then
-	jobname="$CI_OS_NAME-$CC"
-fi
-
 export DEVELOPER=1
 export DEFAULT_TEST_TARGET=prove
 export GIT_TEST_CLONE_2GB=true
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 03/29] CI: remove more dead Travis CI support

[Ævar Arnfjörð Bjarmason]

Remove code that I missed in 4a6e4b96026 (CI: remove Travis CI
support, 2021-11-23). This code was only called from or used by the
now-removed .travis.yml, or needed by the Travis CI environment.

For the symlinking in ci/run-build-and-tests.sh: Back when
3c93b829205 (travis-ci: build Git during the 'script' phase,
2018-01-08) and 4b060a4d973 (ci: use a junction on Windows instead of
a symlink, 2019-01-27) adjusted this "ln" command, the Windows build
would use ci/run-build-and-tests.sh.

As seen in 889cacb6897 (ci: configure GitHub Actions for CI/PR,
2020-04-11) the current windows build uses a different entry point
under the GitHub CI, which doesn't use this .prove caching. Namely
"ci/run-test-slice.sh".

We can be certain that it's never used in "ci/run-test-slice.sh"
because to have a ".prove" file we'd need to use "--state=save", which
we only do in the dead Azure codepath in ci/lib.sh. If it were used it
would do the wrong thing, because the different test slices would each
try to clobber the same "t/.prove" file.

If a subsequent run then used the "--state=failed,slow,save" it would
defeat the purpose of "ci/run-test-slice.sh", since all slices would
then run all tests. I.e. behavior of prove's "--state" options is to
select tests to run from the provided "--state" file, in addition to
those specified on the command-line.

For ci/run-docker{,-build}.sh: It was likewise last referenced in the
.travis.yml removed in my 4a6e4b96026. The current "dockerized" run in
".github/workflows/main.yml" calls the same entry points as the main
"regular" job.

For "ci/print-test-failures.sh" the creation of the tarball added in
aea8879a6ac (travis-ci: include the trash directories of failed tests
in the trace log, 2018-08-01) has been dead since my 4a6e4b96026,
which removed the fall-through case from the "case/esac" statement.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                     | 10 ++----
 ci/print-test-failures.sh     | 39 ---------------------
 ci/run-build-and-tests.sh     |  5 ---
 ci/run-docker-build.sh        | 66 -----------------------------------
 ci/run-docker.sh              | 47 -------------------------
 ci/run-test-slice.sh          |  5 ---
 ci/util/extract-trash-dirs.sh | 50 --------------------------
 7 files changed, 2 insertions(+), 220 deletions(-)
 delete mode 100755 ci/run-docker-build.sh
 delete mode 100755 ci/run-docker.sh
 delete mode 100755 ci/util/extract-trash-dirs.sh

diff --git a/ci/lib.sh b/ci/lib.sh
index 3061de96d8f..2e60c0a71b5 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -106,9 +106,6 @@ then
 	CI_COMMIT="$BUILD_SOURCEVERSION"
 	CI_JOB_ID="$BUILD_BUILDID"
 	CI_JOB_NUMBER="$BUILD_BUILDNUMBER"
-	CI_OS_NAME="$(echo "$AGENT_OS" | tr A-Z a-z)"
-	test darwin != "$CI_OS_NAME" || CI_OS_NAME=osx
-	CI_REPO_SLUG="$(expr "$BUILD_REPOSITORY_URI" : '.*/\([^/]*/[^/]*\)$')"
 	CC="${CC:-gcc}"
 
 	# use a subdirectory of the cache dir (because the file share is shared
@@ -118,16 +115,13 @@ then
 	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
-	test windows_nt != "$CI_OS_NAME" ||
+	test Windows_NT != "$AGENT_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 elif test true = "$GITHUB_ACTIONS"
 then
 	CI_TYPE=github-actions
 	CI_BRANCH="$GITHUB_REF"
 	CI_COMMIT="$GITHUB_SHA"
-	CI_OS_NAME="$(echo "$RUNNER_OS" | tr A-Z a-z)"
-	test macos != "$CI_OS_NAME" || CI_OS_NAME=osx
-	CI_REPO_SLUG="$GITHUB_REPOSITORY"
 	CI_JOB_ID="$GITHUB_RUN_ID"
 	CC="${CC_PACKAGE:-${CC:-gcc}}"
 	DONT_SKIP_TAGS=t
@@ -137,7 +131,7 @@ then
 	export GIT_PROVE_OPTS="--timer --jobs 10"
 	export GIT_TEST_OPTS="--verbose-log -x"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
-	test windows != "$CI_OS_NAME" ||
+	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 else
 	echo "Could not identify CI type" >&2
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 57277eefcd0..740ebb7c529 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -16,15 +16,6 @@ then
 	exit
 fi
 
-case "$jobname" in
-osx-clang|osx-gcc)
-	# base64 in OSX doesn't wrap its output at 76 columns by
-	# default, but prints a single, very long line.
-	base64_opts="-b 76"
-	;;
-esac
-
-combined_trash_size=0
 for TEST_EXIT in test-results/*.exit
 do
 	if [ "$(cat "$TEST_EXIT")" != "0" ]
@@ -42,47 +33,17 @@ do
 		azure-pipelines)
 			mkdir -p failed-test-artifacts
 			mv "$trash_dir" failed-test-artifacts
-			continue
 			;;
 		github-actions)
 			mkdir -p failed-test-artifacts
 			echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
 			cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
 			tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
-			continue
 			;;
 		*)
 			echo "Unhandled CI type: $CI_TYPE" >&2
 			exit 1
 			;;
 		esac
-		trash_tgz_b64="trash.$test_name.base64"
-		if [ -d "$trash_dir" ]
-		then
-			tar czp "$trash_dir" |base64 $base64_opts >"$trash_tgz_b64"
-
-			trash_size=$(wc -c <"$trash_tgz_b64")
-			if [ $trash_size -gt 1048576 ]
-			then
-				# larger than 1MB
-				echo "$(tput setaf 1)Didn't include the trash directory of '$test_name' in the trace log, it's too big$(tput sgr0)"
-				continue
-			fi
-
-			new_combined_trash_size=$(($combined_trash_size + $trash_size))
-			if [ $new_combined_trash_size -gt 1048576 ]
-			then
-				echo "$(tput setaf 1)Didn't include the trash directory of '$test_name' in the trace log, there is plenty of trash in there already.$(tput sgr0)"
-				continue
-			fi
-			combined_trash_size=$new_combined_trash_size
-
-			# DO NOT modify these two 'echo'-ed strings below
-			# without updating 'ci/util/extract-trash-dirs.sh'
-			# as well.
-			echo "$(tput setaf 1)Start of trash directory of '$test_name':$(tput sgr0)"
-			cat "$trash_tgz_b64"
-			echo "$(tput setaf 1)End of trash directory of '$test_name'$(tput sgr0)"
-		fi
 	fi
 done
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 280dda7d285..0a1ec8c2bae 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -5,11 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-case "$CI_OS_NAME" in
-windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
-*) ln -s "$cache_dir/.prove" t/.prove;;
-esac
-
 export MAKE_TARGETS="all test"
 
 case "$jobname" in
diff --git a/ci/run-docker-build.sh b/ci/run-docker-build.sh
deleted file mode 100755
index 6cd832efb9c..00000000000
--- a/ci/run-docker-build.sh
+++ /dev/null
@@ -1,66 +0,0 @@
-#!/bin/sh
-#
-# Build and test Git inside container
-#
-# Usage:
-#   run-docker-build.sh <host-user-id>
-#
-
-set -ex
-
-if test $# -ne 1 || test -z "$1"
-then
-	echo >&2 "usage: run-docker-build.sh <host-user-id>"
-	exit 1
-fi
-
-case "$jobname" in
-linux32)
-	switch_cmd="linux32 --32bit i386"
-	;;
-linux-musl)
-	switch_cmd=
-	useradd () { adduser -D "$@"; }
-	;;
-*)
-	exit 1
-	;;
-esac
-
-"${0%/*}/install-docker-dependencies.sh"
-
-# If this script runs inside a docker container, then all commands are
-# usually executed as root. Consequently, the host user might not be
-# able to access the test output files.
-# If a non 0 host user id is given, then create a user "ci" with that
-# user id to make everything accessible to the host user.
-HOST_UID=$1
-if test $HOST_UID -eq 0
-then
-	# Just in case someone does want to run the test suite as root.
-	CI_USER=root
-else
-	CI_USER=ci
-	if test "$(id -u $CI_USER 2>/dev/null)" = $HOST_UID
-	then
-		echo "user '$CI_USER' already exists with the requested ID $HOST_UID"
-	else
-		useradd -u $HOST_UID $CI_USER
-	fi
-fi
-
-# Build and test
-command $switch_cmd su -m -l $CI_USER -c "
-	set -ex
-	export DEVELOPER='$DEVELOPER'
-	export DEFAULT_TEST_TARGET='$DEFAULT_TEST_TARGET'
-	export GIT_PROVE_OPTS='$GIT_PROVE_OPTS'
-	export GIT_TEST_OPTS='$GIT_TEST_OPTS'
-	export GIT_TEST_CLONE_2GB='$GIT_TEST_CLONE_2GB'
-	export MAKEFLAGS='$MAKEFLAGS'
-	export cache_dir='$cache_dir'
-	cd /usr/src/git
-	test -n '$cache_dir' && ln -s '$cache_dir/.prove' t/.prove
-	make
-	make test
-"
diff --git a/ci/run-docker.sh b/ci/run-docker.sh
deleted file mode 100755
index af89d1624a4..00000000000
--- a/ci/run-docker.sh
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/bin/sh
-#
-# Download and run Docker image to build and test Git
-#
-
-. ${0%/*}/lib.sh
-
-case "$jobname" in
-linux32)
-	CI_CONTAINER="daald/ubuntu32:xenial"
-	;;
-linux-musl)
-	CI_CONTAINER=alpine
-	;;
-*)
-	exit 1
-	;;
-esac
-
-docker pull "$CI_CONTAINER"
-
-# Use the following command to debug the docker build locally:
-# <host-user-id> must be 0 if podman is used as drop-in replacement for docker
-# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/sh "$CI_CONTAINER"
-# root@container:/# export jobname=<jobname>
-# root@container:/# /usr/src/git/ci/run-docker-build.sh <host-user-id>
-
-container_cache_dir=/tmp/container-cache
-
-docker run \
-	--interactive \
-	--env DEVELOPER \
-	--env DEFAULT_TEST_TARGET \
-	--env GIT_PROVE_OPTS \
-	--env GIT_TEST_OPTS \
-	--env GIT_TEST_CLONE_2GB \
-	--env MAKEFLAGS \
-	--env jobname \
-	--env cache_dir="$container_cache_dir" \
-	--volume "${PWD}:/usr/src/git" \
-	--volume "$cache_dir:$container_cache_dir" \
-	"$CI_CONTAINER" \
-	/usr/src/git/ci/run-docker-build.sh $(id -u $USER)
-
-check_unignored_build_artifacts
-
-save_good_tree
diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index f8c2c3106a2..b9a682b4bcd 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -5,11 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-case "$CI_OS_NAME" in
-windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
-*) ln -s "$cache_dir/.prove" t/.prove;;
-esac
-
 make --quiet -C t T="$(cd t &&
 	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
 	tr '\n' ' ')"
diff --git a/ci/util/extract-trash-dirs.sh b/ci/util/extract-trash-dirs.sh
deleted file mode 100755
index 8e67bec21a2..00000000000
--- a/ci/util/extract-trash-dirs.sh
+++ /dev/null
@@ -1,50 +0,0 @@
-#!/bin/sh
-
-error () {
-	echo >&2 "error: $@"
-	exit 1
-}
-
-find_embedded_trash () {
-	while read -r line
-	do
-		case "$line" in
-		*Start\ of\ trash\ directory\ of\ \'t[0-9][0-9][0-9][0-9]-*\':*)
-			test_name="${line#*\'}"
-			test_name="${test_name%\'*}"
-
-			return 0
-		esac
-	done
-
-	return 1
-}
-
-extract_embedded_trash () {
-	while read -r line
-	do
-		case "$line" in
-		*End\ of\ trash\ directory\ of\ \'$test_name\'*)
-			return
-			;;
-		*)
-			printf '%s\n' "$line"
-			;;
-		esac
-	done
-
-	error "unexpected end of input"
-}
-
-# Raw logs from Linux build jobs have CRLF line endings, while OSX
-# build jobs mostly have CRCRLF, except an odd line every now and
-# then that has CRCRCRLF.  'base64 -d' from 'coreutils' doesn't like
-# CRs and complains about "invalid input", so remove all CRs at the
-# end of lines.
-sed -e 's/\r*$//' | \
-while find_embedded_trash
-do
-	echo "Extracting trash directory of '$test_name'"
-
-	extract_embedded_trash |base64 -d |tar xzp
-done
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 04/29] CI: remove dead "tree skipping" code

[Ævar Arnfjörð Bjarmason]

Remove the code related to skipping CI runs if the tree was
successfully tested before. With the GitHub CI this is done with the
"ci-config" step, see e76eec35540 (ci: allow per-branch config for
GitHub Actions, 2020-05-07) and 7d78d5fc1a9 (ci: skip GitHub workflow
runs for already-tested commits/trees, 2020-10-08).

This code hasn't been used since 4a6e4b96026 (CI: remove Travis CI
support, 2021-11-23), and before that for the retired Azure pipeline
support removed in 6081d3898fe (ci: retire the Azure Pipelines
definition, 2020-04-11).

This change is needed because this and subsequent commits are turning
"ci/lib.sh" into a dumber library that'll only be tasked with setting
variables for CI jobs, or for specific steps within those jobs. Now we
don't need to worry about it potentially skipping the run on its own.

This change also removes a subtle potential logic error introduced in
0e7696c64db (ci: disallow directional formatting, 2021-11-04). The
"ci/check-directional-formatting.bash" script would have been made to
run after the "save_good_tree" in invoked in
"ci/run-static-analysis.sh". So if e.g. the "make coccicheck" failed
we'd still mark the tree as good.

I.e. the addition of "ci/check-directional-formatting.bash" didn't
take into account that the various users of the "save_good_tree"
function in ci/*.sh made a hard assumption that they're the only
ci/*.sh script that's being run (and they use "set -e").

This is entirely academic since we weren't actually running this code,
but is something to be careful of if anyone ever needs to resurrect
parts of this.

Finally, the cache_dir="$HOME/none" added in a3f2eec862b (ci/lib:
allow running in GitHub Actions, 2020-04-08) for the GitHub CI can be
removed. It appears to have been added to appease the subsequent
'mkdir -p "$cache_dir"', which wasn't needed in GitHub CI.

1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                 | 91 ---------------------------------------
 ci/run-build-and-tests.sh |  2 -
 ci/run-static-analysis.sh |  2 -
 ci/test-documentation.sh  |  2 -
 4 files changed, 97 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 2e60c0a71b5..e04b53b428d 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -13,75 +13,6 @@ then
 	exit 1
 fi
 
-skip_branch_tip_with_tag () {
-	# Sometimes, a branch is pushed at the same time the tag that points
-	# at the same commit as the tip of the branch is pushed, and building
-	# both at the same time is a waste.
-	#
-	# When the build is triggered by a push to a tag, $CI_BRANCH will
-	# have that tagname, e.g. v2.14.0.  Let's see if $CI_BRANCH is
-	# exactly at a tag, and if so, if it is different from $CI_BRANCH.
-	# That way, we can tell if we are building the tip of a branch that
-	# is tagged and we can skip the build because we won't be skipping a
-	# build of a tag.
-
-	if TAG=$(git describe --exact-match "$CI_BRANCH" 2>/dev/null) &&
-		test "$TAG" != "$CI_BRANCH"
-	then
-		echo "$(tput setaf 2)Tip of $CI_BRANCH is exactly at $TAG$(tput sgr0)"
-		exit 0
-	fi
-}
-
-# Save some info about the current commit's tree, so we can skip the build
-# job if we encounter the same tree again and can provide a useful info
-# message.
-save_good_tree () {
-	echo "$(git rev-parse $CI_COMMIT^{tree}) $CI_COMMIT $CI_JOB_NUMBER $CI_JOB_ID" >>"$good_trees_file"
-	# limit the file size
-	tail -1000 "$good_trees_file" >"$good_trees_file".tmp
-	mv "$good_trees_file".tmp "$good_trees_file"
-}
-
-# Skip the build job if the same tree has already been built and tested
-# successfully before (e.g. because the branch got rebased, changing only
-# the commit messages).
-skip_good_tree () {
-	if test true = "$GITHUB_ACTIONS"
-	then
-		return
-	fi
-
-	if ! good_tree_info="$(grep "^$(git rev-parse $CI_COMMIT^{tree}) " "$good_trees_file")"
-	then
-		# Haven't seen this tree yet, or no cached good trees file yet.
-		# Continue the build job.
-		return
-	fi
-
-	echo "$good_tree_info" | {
-		read tree prev_good_commit prev_good_job_number prev_good_job_id
-
-		if test "$CI_JOB_ID" = "$prev_good_job_id"
-		then
-			cat <<-EOF
-			$(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0)
-			This commit has already been built and tested successfully by this build job.
-			To force a re-build delete the branch's cache and then hit 'Restart job'.
-			EOF
-		else
-			cat <<-EOF
-			$(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0)
-			This commit's tree has already been built and tested successfully in build job $prev_good_job_number for commit $prev_good_commit.
-			The log of that build job is available at $SYSTEM_TASKDEFINITIONSURI$SYSTEM_TEAMPROJECT/_build/results?buildId=$prev_good_job_id
-			To force a re-build delete the branch's cache and then hit 'Restart job'.
-			EOF
-		fi
-	}
-
-	exit 0
-}
-
 check_unignored_build_artifacts ()
 {
 	! git ls-files --other --exclude-standard --error-unmatch \
@@ -102,16 +33,8 @@ if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
 then
 	CI_TYPE=azure-pipelines
 	# We are running in Azure Pipelines
-	CI_BRANCH="$BUILD_SOURCEBRANCH"
-	CI_COMMIT="$BUILD_SOURCEVERSION"
-	CI_JOB_ID="$BUILD_BUILDID"
-	CI_JOB_NUMBER="$BUILD_BUILDNUMBER"
 	CC="${CC:-gcc}"
 
-	# use a subdirectory of the cache dir (because the file share is shared
-	# among *all* phases)
-	cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME"
-
 	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
 	MAKEFLAGS="$MAKEFLAGS --jobs=10"
@@ -120,13 +43,7 @@ then
 elif test true = "$GITHUB_ACTIONS"
 then
 	CI_TYPE=github-actions
-	CI_BRANCH="$GITHUB_REF"
-	CI_COMMIT="$GITHUB_SHA"
-	CI_JOB_ID="$GITHUB_RUN_ID"
 	CC="${CC_PACKAGE:-${CC:-gcc}}"
-	DONT_SKIP_TAGS=t
-
-	cache_dir="$HOME/none"
 
 	export GIT_PROVE_OPTS="--timer --jobs 10"
 	export GIT_TEST_OPTS="--verbose-log -x"
@@ -139,14 +56,6 @@ else
 	exit 1
 fi
 
-good_trees_file="$cache_dir/good-trees"
-
-mkdir -p "$cache_dir"
-
-test -n "${DONT_SKIP_TAGS-}" ||
-skip_branch_tip_with_tag
-skip_good_tree
-
 export DEVELOPER=1
 export DEFAULT_TEST_TARGET=prove
 export GIT_TEST_CLONE_2GB=true
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 0a1ec8c2bae..e5a21985b44 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -45,5 +45,3 @@ esac
 # start running tests.
 make $MAKE_TARGETS
 check_unignored_build_artifacts
-
-save_good_tree
diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh
index 65bcebda41a..5a87b1cac96 100755
--- a/ci/run-static-analysis.sh
+++ b/ci/run-static-analysis.sh
@@ -28,5 +28,3 @@ fi
 
 make hdr-check ||
 exit 1
-
-save_good_tree
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index de41888430a..9e0652c30dd 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -41,5 +41,3 @@ grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
 
 rm -f stdout.log stderr.log stderr.raw
 check_unignored_build_artifacts
-
-save_good_tree
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 05/29] CI: remove unused Azure ci/* code

[Ævar Arnfjörð Bjarmason]

Remove Azure-specific code that's been unused since 6081d3898fe (ci:
retire the Azure Pipelines definition, 2020-04-11). As noted in a
larger removal of all of the Azure-supporting code in [1] (although
that missed some of this) there is more of it in-tree, but let's focus
on only the ci/* code for now.

This is needed because in subsequent commits this unused code would
either need to be changed to accommodate changes in ci/*, or be
removed.

As we'll see in those subsequent commits the end-state we're heading
towards will actually make it easier to add new CI types in the
future, even though the only one we're left with now is the GitHub
CI. I.e. the "ci/*" framework will be made to do less, not more. We'll
be offloading more of what it does to our generic build and test
system.

While I'm at it (since the line needs to be touched anyway) change an
odd 'if test true == "$GITHUB_ACTIONS"' to the more usual style used
in other places of 'if test "$GITHUB_ACTIONS" = "true"'.

1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/install-dependencies.sh |  3 ---
 ci/lib.sh                  | 13 +------------
 ci/mount-fileshare.sh      | 25 -------------------------
 ci/print-test-failures.sh  |  4 ----
 4 files changed, 1 insertion(+), 44 deletions(-)
 delete mode 100755 ci/mount-fileshare.sh

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 107757a1fea..4fa3e282dda 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -34,8 +34,6 @@ macos-latest)
 	export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
 	# Uncomment this if you want to run perf tests:
 	# brew install gnu-time
-	test -z "$BREW_INSTALL_PACKAGES" ||
-	brew install $BREW_INSTALL_PACKAGES
 	brew link --force gettext
 	mkdir -p $HOME/bin
 	(
@@ -71,7 +69,6 @@ Documentation)
 	sudo apt-get -q update
 	sudo apt-get -q -y install asciidoc xmlto docbook-xsl-ns make
 
-	test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
 	sudo gem install --version 1.5.8 asciidoctor
 	;;
 linux-gcc-default)
diff --git a/ci/lib.sh b/ci/lib.sh
index e04b53b428d..ebb9461ff42 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -29,18 +29,7 @@ export TERM=${TERM:-dumb}
 # Clear MAKEFLAGS that may come from the outside world.
 export MAKEFLAGS=
 
-if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
-then
-	CI_TYPE=azure-pipelines
-	# We are running in Azure Pipelines
-	CC="${CC:-gcc}"
-
-	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
-	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
-	MAKEFLAGS="$MAKEFLAGS --jobs=10"
-	test Windows_NT != "$AGENT_OS" ||
-	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
-elif test true = "$GITHUB_ACTIONS"
+if test "$GITHUB_ACTIONS" = "true"
 then
 	CI_TYPE=github-actions
 	CC="${CC_PACKAGE:-${CC:-gcc}}"
diff --git a/ci/mount-fileshare.sh b/ci/mount-fileshare.sh
deleted file mode 100755
index 26b58a80960..00000000000
--- a/ci/mount-fileshare.sh
+++ /dev/null
@@ -1,25 +0,0 @@
-#!/bin/sh
-
-die () {
-	echo "$*" >&2
-	exit 1
-}
-
-test $# = 4 ||
-die "Usage: $0 <share> <username> <password> <mountpoint>"
-
-mkdir -p "$4" || die "Could not create $4"
-
-case "$(uname -s)" in
-Linux)
-	sudo mount -t cifs -o vers=3.0,username="$2",password="$3",dir_mode=0777,file_mode=0777,serverino "$1" "$4"
-	;;
-Darwin)
-	pass="$(echo "$3" | sed -e 's/\//%2F/g' -e 's/+/%2B/g')" &&
-	mount -t smbfs,soft "smb://$2:$pass@${1#//}" "$4"
-	;;
-*)
-	die "No support for $(uname -s)"
-	;;
-esac ||
-die "Could not mount $4"
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 740ebb7c529..47e2ea1ad10 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -30,10 +30,6 @@ do
 		test_name="${test_name##*/}"
 		trash_dir="trash directory.$test_name"
 		case "$CI_TYPE" in
-		azure-pipelines)
-			mkdir -p failed-test-artifacts
-			mv "$trash_dir" failed-test-artifacts
-			;;
 		github-actions)
 			mkdir -p failed-test-artifacts
 			echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 06/29] CI/lib.sh: stop adding leading whitespace to $MAKEFLAGS

[Ævar Arnfjörð Bjarmason]

In a8c51f77d16 (ci: clear and mark MAKEFLAGS exported just once,
2019-02-07) the assignment of MAKEFLAGS was done up-front to
future-proof this code for the bug fixed in 406f93ae488 (ci: make sure
we build Git parallel, 2019-02-07).

But since 4a6e4b96026 (CI: remove Travis CI support, 2021-11-23) the
--jobs=10 has been an invariant here, so let's assign --jobs=$NPROC to
the MAKEFLAGS right away, this gets rid of the leading whitespace from
the $MAKEFLAGS variable, as it would previously be empty on the first
assignment.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index ebb9461ff42..5f95f55a147 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -26,17 +26,19 @@ check_unignored_build_artifacts ()
 # GitHub Action doesn't set TERM, which is required by tput
 export TERM=${TERM:-dumb}
 
+# How many jobs to run in parallel?
+NPROC=10
+
 # Clear MAKEFLAGS that may come from the outside world.
-export MAKEFLAGS=
+export MAKEFLAGS=--jobs=$NPROC
 
 if test "$GITHUB_ACTIONS" = "true"
 then
 	CI_TYPE=github-actions
 	CC="${CC_PACKAGE:-${CC:-gcc}}"
 
-	export GIT_PROVE_OPTS="--timer --jobs 10"
+	export GIT_PROVE_OPTS="--timer --jobs $NPROC"
 	export GIT_TEST_OPTS="--verbose-log -x"
-	MAKEFLAGS="$MAKEFLAGS --jobs=10"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 else
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 07/29] CI: don't have "git grep" invoke a pager in tree content check

[Ævar Arnfjörð Bjarmason]

Fix the omission of not adding "-P" (a.k.a. "--no-pager") to the "git"
invocation that spaws "git grep", added in 0e7696c64db (ci: disallow
directional formatting, 2021-11-04).

This didn't matter for the GitHub CI, but does when running the check
locally. A subsequent commit will start invoking it via a new check
from the top-level Makefile, and without this we'd visibly spawn a
pager when doing so.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/check-directional-formatting.bash | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ci/check-directional-formatting.bash b/ci/check-directional-formatting.bash
index e6211b141aa..de8e6e486f0 100755
--- a/ci/check-directional-formatting.bash
+++ b/ci/check-directional-formatting.bash
@@ -23,5 +23,5 @@
 # U+2066..U+2069: LRI, RLI, FSI and PDI
 regex='(\u202a|\u202b|\u202c|\u202d|\u202e|\u2066|\u2067|\u2068|\u2069)'
 
-! LC_CTYPE=C git grep -El "$(LC_CTYPE=C.UTF-8 printf "$regex")" \
+! LC_CTYPE=C git -P grep -El "$(LC_CTYPE=C.UTF-8 printf "$regex")" \
 	-- ':(exclude,attr:binary)' ':(exclude)*.po'
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 08/29] CI: have "static-analysis" run a "make ci-static-analysis" target

[Ævar Arnfjörð Bjarmason]

Make it easier to run the CI logic locally by creating
"ci-static-analysis" target.

Now the "check-coccicheck" logic added in 0860a7641ba (travis-ci: fail
if Coccinelle static analysis found something to transform,
2018-07-23) can be easily tested outside of CI, and likewise the
sanity check added in 0e7696c64db (ci: disallow directional
formatting, 2021-11-04).

These targets could be improved, the "hdr-check" target will re-do all
of its work every time it's run, and could with a change similar to my
c234e8a0ecf (Makefile: make the "sparse" target non-.PHONY,
2021-09-23). The same goes for the new
"ci-check-directional-formatting" target.

But without those improvements being able to easily run a 1=1
equivalent of the checks we do in CI during a local build is already
an improvement. Subsequent commits will expand on this ability.

This change will also make the CI check faster, since we can take
advantage of parallelism. The "make coccicheck" takes a very long
time, and at the tail end we might only have one job pegging 100% of
one CPU core for a long time. Now any extra cores will be free to run
the rest of the targets under "ci-static-analysis".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml |  3 +--
 Makefile                   | 28 ++++++++++++++++++++++++++++
 ci/run-static-analysis.sh  | 30 ------------------------------
 shared.mak                 |  1 +
 4 files changed, 30 insertions(+), 32 deletions(-)
 delete mode 100755 ci/run-static-analysis.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 309b9141249..ad752010102 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -317,8 +317,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/run-static-analysis.sh
-    - run: ci/check-directional-formatting.bash
+    - run: make ci-static-analysis
   sparse:
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
diff --git a/Makefile b/Makefile
index 61aadf3ce88..9ae7e1a373e 100644
--- a/Makefile
+++ b/Makefile
@@ -3034,6 +3034,20 @@ coccicheck: $(addsuffix .patch,$(filter-out %.pending.cocci,$(wildcard contrib/c
 # See contrib/coccinelle/README
 coccicheck-pending: $(addsuffix .patch,$(wildcard contrib/coccinelle/*.pending.cocci))
 
+.PHONY: check-coccicheck
+check-coccicheck: coccicheck
+	$(QUIET_CHECK)for cocci_patch in contrib/coccinelle/*.patch; do \
+		if test -s "$$cocci_patch"; then \
+			echo "Coccinelle suggests the following changes in '$$cocci_patch':"; \
+			cat "$$cocci_patch"; \
+			fail=UnfortunatelyYes; \
+		fi \
+	done; \
+	if test -n "$$fail"; then \
+		echo "error: Coccinelle suggested some changes"; \
+		exit 1; \
+	fi
+
 .PHONY: coccicheck coccicheck-pending
 
 ### Installation rules
@@ -3469,3 +3483,17 @@ $(FUZZ_PROGRAMS): all
 		$(XDIFF_OBJS) $(EXTLIBS) git.o $@.o $(LIB_FUZZING_ENGINE) -o $@
 
 fuzz-all: $(FUZZ_PROGRAMS)
+
+### CI "check" targets
+#
+# These targets are run from the CI, see .github/workflows/main.yml,
+# but can also be run manually to run the same assertions locally.
+
+.PHONY: ci-check-directional-formatting
+ci-check-directional-formatting:
+	$(QUIET_CHECK)ci/check-directional-formatting.bash
+
+.PHONY: ci-static-analysis
+ci-static-analysis: ci-check-directional-formatting
+ci-static-analysis: check-coccicheck
+ci-static-analysis: hdr-check
diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh
deleted file mode 100755
index 5a87b1cac96..00000000000
--- a/ci/run-static-analysis.sh
+++ /dev/null
@@ -1,30 +0,0 @@
-#!/bin/sh
-#
-# Perform various static code analysis checks
-#
-
-. ${0%/*}/lib.sh
-
-make coccicheck
-
-set +x
-
-fail=
-for cocci_patch in contrib/coccinelle/*.patch
-do
-	if test -s "$cocci_patch"
-	then
-		echo "$(tput setaf 1)Coccinelle suggests the following changes in '$cocci_patch':$(tput sgr0)"
-		cat "$cocci_patch"
-		fail=UnfortunatelyYes
-	fi
-done
-
-if test -n "$fail"
-then
-	echo "$(tput setaf 1)error: Coccinelle suggested some changes$(tput sgr0)"
-	exit 1
-fi
-
-make hdr-check ||
-exit 1
diff --git a/shared.mak b/shared.mak
index 50d4596f0d6..83e0aa16e7d 100644
--- a/shared.mak
+++ b/shared.mak
@@ -68,6 +68,7 @@ ifndef V
 	QUIET_HDR      = @echo '   ' HDR $(<:hcc=h);
 	QUIET_RC       = @echo '   ' RC $@;
 	QUIET_SPATCH   = @echo '   ' SPATCH $<;
+	QUIET_CHECK    = @echo '   ' CHECK $@;
 
 ## Used in "Documentation/Makefile"
 	QUIET_ASCIIDOC	= @echo '   ' ASCIIDOC $@;
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 09/29] CI: have "static-analysis" run "check-builtins", not "documentation"

[Ævar Arnfjörð Bjarmason]

Move the "make check-builtins" check from the "documentation" job to
the "static-analysis" job.

The "check-builtins" target added in c74390e4a1d (cherry is built-in,
do not ship git-cherry.sh, 2006-11-05) is unrelated to the
documentation, so it's odd that b98712b9aa9 (travis-ci: build
documentation, 2016-05-04) added it to the "documentation" job.

Let's just move it to the "ci-static-analysis" target, and while we're
at it improve the output with $(QUIET_CHECK).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile                 | 3 ++-
 ci/test-documentation.sh | 1 -
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 9ae7e1a373e..3398d3f2e56 100644
--- a/Makefile
+++ b/Makefile
@@ -3405,7 +3405,7 @@ check-docs::
 ### Make sure built-ins do not have dups and listed in git.c
 #
 check-builtins::
-	./check-builtins.sh
+	$(QUIET_CHECK)./check-builtins.sh
 
 ### Test suite coverage testing
 #
@@ -3495,5 +3495,6 @@ ci-check-directional-formatting:
 
 .PHONY: ci-static-analysis
 ci-static-analysis: ci-check-directional-formatting
+ci-static-analysis: check-builtins
 ci-static-analysis: check-coccicheck
 ci-static-analysis: hdr-check
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 9e0652c30dd..41e2b126311 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -14,7 +14,6 @@ filter_log () {
 	    "$1"
 }
 
-make check-builtins
 make check-docs
 
 # Build docs with AsciiDoc
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 10/29] CI: move p4 and git-lfs variables to ci/install-dependencies.sh

[Ævar Arnfjörð Bjarmason]

Move the declaration of variables that are only used by the
"ubuntu-latest" block in "ci/install-dependencies.sh" there from
"ci/lib.sh".

This makes the code easier to follow, and changes "ci/lib.sh" to a
library that only exports CI variables for general use, we didn't need
to export these $P4_PATH and $GIT_LFS_PATH variables.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/install-dependencies.sh | 17 +++++++++++++++--
 ci/lib.sh                  | 12 ------------
 2 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 4fa3e282dda..41a9185bb1e 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -5,14 +5,27 @@
 
 . ${0%/*}/lib.sh
 
-P4WHENCE=https://cdist2.perforce.com/perforce/r$LINUX_P4_VERSION
-LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
 UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
  tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
  libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl"
 
 case "$runs_on_pool" in
 ubuntu-latest)
+	# The Linux build installs the defined dependency versions below.
+	# The OS X build installs much more recent versions, whichever
+	# were recorded in the Homebrew database upon creating the OS X
+	# image.
+	# Keep that in mind when you encounter a broken OS X build!
+	LINUX_P4_VERSION="16.2"
+	LINUX_GIT_LFS_VERSION="1.5.2"
+
+	P4_PATH="$HOME/custom/p4"
+	GIT_LFS_PATH="$HOME/custom/git-lfs"
+	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
+
+	P4WHENCE=https://cdist2.perforce.com/perforce/r$LINUX_P4_VERSION
+	LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
+
 	sudo apt-get -q update
 	sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \
 		$UBUNTU_COMMON_PKGS $CC_PACKAGE
diff --git a/ci/lib.sh b/ci/lib.sh
index 5f95f55a147..e789cd2a1cd 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -67,18 +67,6 @@ ubuntu-latest)
 	fi
 
 	export GIT_TEST_HTTPD=true
-
-	# The Linux build installs the defined dependency versions below.
-	# The OS X build installs much more recent versions, whichever
-	# were recorded in the Homebrew database upon creating the OS X
-	# image.
-	# Keep that in mind when you encounter a broken OS X build!
-	export LINUX_P4_VERSION="16.2"
-	export LINUX_GIT_LFS_VERSION="1.5.2"
-
-	P4_PATH="$HOME/custom/p4"
-	GIT_LFS_PATH="$HOME/custom/git-lfs"
-	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
 	;;
 macos-latest)
 	if [ "$jobname" = osx-gcc ]
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 11/29] CI: consistently use "export" in ci/lib.sh

[Ævar Arnfjörð Bjarmason]

Change the "ci/lib.sh" script co consistently use "export", for
e.g. MAKEFLAGS we were exporting it, and then assigning to it, let's
do it the other way around.

Right now this doesn't matter, since we in
e.g. "ci/install-dependencies.sh" source this file, and don't use
something like "env(1)" to retrieve these variables. But in a
subsequent commit we'll "export" these variables through a wrapper (to
additionally write them to a GitHub CI-specific $GITHUB_ENV
file). This change makes that subsequent change easier to read, as it
won't need to do any control flow refactoring.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index e789cd2a1cd..1b5b805c5ca 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -30,7 +30,7 @@ export TERM=${TERM:-dumb}
 NPROC=10
 
 # Clear MAKEFLAGS that may come from the outside world.
-export MAKEFLAGS=--jobs=$NPROC
+MAKEFLAGS=--jobs=$NPROC
 
 if test "$GITHUB_ACTIONS" = "true"
 then
@@ -38,9 +38,10 @@ then
 	CC="${CC_PACKAGE:-${CC:-gcc}}"
 
 	export GIT_PROVE_OPTS="--timer --jobs $NPROC"
-	export GIT_TEST_OPTS="--verbose-log -x"
+	GIT_TEST_OPTS="--verbose-log -x"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
+	export GIT_TEST_OPTS
 else
 	echo "Could not identify CI type" >&2
 	env >&2
@@ -94,4 +95,4 @@ linux-leaks)
 	;;
 esac
 
-MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
+export MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 12/29] CI: export variables via a wrapper

[Ævar Arnfjörð Bjarmason]

Add and use a "setenv" wrapper to export the variables set in
ci/lib.sh.

For now this merely adds debugging information about what variables
are set, but in a subsequent commit we'll expand this to set variables
in $GITHUB_ENV instead of the environment, and thus carry variable
settings between steps in a job.

We'll also eventually be interested in what type of variable we're
setting, and when it needs to be used, because we'll prominently
expose what variables we set in what CI "step", so having only the
ones we actually use will improve readability.

To that end call the "setenv" function with optional "--build",
"--test" and "--all" parameters to indicate whether they're needed for
building, testing or both.

We currently ignore the "varname" parameter, but will make use of it in a
subsequent commit. Adding it here reduces the needless churn, as we
won't need to change all these lines again for that subsequent change.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh                 | 51 ++++++++++++++++++++++++++++++---------
 ci/run-build-and-tests.sh | 34 +++++++++++++-------------
 2 files changed, 57 insertions(+), 28 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 1b5b805c5ca..71e6d6e08e7 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -13,6 +13,35 @@ then
 	exit 1
 fi
 
+# Helper functions
+setenv () {
+	while test $# != 0
+	do
+		case "$1" in
+		--build)
+			;;
+		--test)
+			;;
+		--all)
+			;;
+		-*)
+			echo "BUG: bad setenv() option '$1'" >&2
+			exit 1
+			;;
+		*)
+			break
+			;;
+		esac
+		shift
+	done
+
+	key=$1
+	val=$2
+	shift 2
+
+	eval "export $key=\"$val\""
+}
+
 check_unignored_build_artifacts ()
 {
 	! git ls-files --other --exclude-standard --error-unmatch \
@@ -24,7 +53,7 @@ check_unignored_build_artifacts ()
 }
 
 # GitHub Action doesn't set TERM, which is required by tput
-export TERM=${TERM:-dumb}
+setenv TERM ${TERM:-dumb}
 
 # How many jobs to run in parallel?
 NPROC=10
@@ -37,21 +66,21 @@ then
 	CI_TYPE=github-actions
 	CC="${CC_PACKAGE:-${CC:-gcc}}"
 
-	export GIT_PROVE_OPTS="--timer --jobs $NPROC"
+	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
 	GIT_TEST_OPTS="--verbose-log -x"
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
-	export GIT_TEST_OPTS
+	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
 else
 	echo "Could not identify CI type" >&2
 	env >&2
 	exit 1
 fi
 
-export DEVELOPER=1
-export DEFAULT_TEST_TARGET=prove
-export GIT_TEST_CLONE_2GB=true
-export SKIP_DASHED_BUILT_INS=YesPlease
+setenv --build DEVELOPER 1
+setenv --test DEFAULT_TEST_TARGET prove
+setenv --test GIT_TEST_CLONE_2GB true
+setenv --build SKIP_DASHED_BUILT_INS YesPlease
 
 case "$runs_on_pool" in
 ubuntu-latest)
@@ -67,7 +96,7 @@ ubuntu-latest)
 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python2"
 	fi
 
-	export GIT_TEST_HTTPD=true
+	setenv --test GIT_TEST_HTTPD true
 	;;
 macos-latest)
 	if [ "$jobname" = osx-gcc ]
@@ -90,9 +119,9 @@ linux-musl)
 	MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8"
 	;;
 linux-leaks)
-	export SANITIZE=leak
-	export GIT_TEST_PASSING_SANITIZE_LEAK=true
+	setenv --build SANITIZE leak
+	setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true
 	;;
 esac
 
-export MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
+setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index e5a21985b44..35d45a9373e 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -9,33 +9,33 @@ export MAKE_TARGETS="all test"
 
 case "$jobname" in
 linux-gcc)
-	export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
 	;;
 linux-TEST-vars)
-	export GIT_TEST_SPLIT_INDEX=yes
-	export GIT_TEST_MERGE_ALGORITHM=recursive
-	export GIT_TEST_FULL_IN_PACK_ARRAY=true
-	export GIT_TEST_OE_SIZE=10
-	export GIT_TEST_OE_DELTA_SIZE=5
-	export GIT_TEST_COMMIT_GRAPH=1
-	export GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS=1
-	export GIT_TEST_MULTI_PACK_INDEX=1
-	export GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP=1
-	export GIT_TEST_ADD_I_USE_BUILTIN=1
-	export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master
-	export GIT_TEST_WRITE_REV_INDEX=1
-	export GIT_TEST_CHECKOUT_WORKERS=2
+	setenv --test GIT_TEST_SPLIT_INDEX yes
+	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
+	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
+	setenv --test GIT_TEST_OE_SIZE 10
+	setenv --test GIT_TEST_OE_DELTA_SIZE 5
+	setenv --test GIT_TEST_COMMIT_GRAPH 1
+	setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
+	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
+	setenv --test GIT_TEST_WRITE_REV_INDEX 1
+	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
 	;;
 linux-clang)
-	export GIT_TEST_DEFAULT_HASH=sha1
+	setenv --test GIT_TEST_DEFAULT_HASH sha1
 	;;
 linux-sha256)
-	export GIT_TEST_DEFAULT_HASH=sha256
+	setenv --test GIT_TEST_DEFAULT_HASH sha256
 	;;
 pedantic)
 	# Don't run the tests; we only care about whether Git can be
 	# built.
-	export DEVOPTS=pedantic
+	setenv --build DEVOPTS pedantic
 	export MAKE_TARGETS=all
 	;;
 esac
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 13/29] CI: remove "run-build-and-tests.sh", run "make [test]" directly

[Ævar Arnfjörð Bjarmason]

Remove the already thin "ci/run-build-and-tests.sh" wrapper and
instead make the CI run "make" or "make test" directly. By doing this
we'll be able to easily see at a glance whether our failure was in the
compilation or testing, whether that's via human eyes or improve
machine readability.

We also need to run our new "ci/check-unignored-build-artifacts.sh" on
success() in the CI now, just like we already had a step conditional
on failure() running ci/print-test-failures.sh.

The reason we used a "ci/run-build-and-tests.sh" wrapper in the first
place had to do with Travis CI-specific constraints that no longer
apply to us, as the Travis CI support has been removed.

Instead we can configure the CI in an earlier step by running
"ci/lib.sh", which under GitHub CI will write the environment
variables we need to the "$GITHUB_ENV" file.

We'll then have access to them in subsequent steps, and crucially
those variables will be prominently visible at the start of each step
via an expandable drop-down in the UI.drop-do.

I.e. this changes the CI run from a top-down flow like (pseudocode):

 - job:
   - step1:
     - use ci/lib.sh to set env vars
     - run a script like ci/run-build-and-tests.sh
   - step2:
     - if: failure()
     - use ci/lib.sh to set env vars
     - run ci/print-test-failures.sh

To:

 - job:
   - step1:
     - set variables in $GITHUB_ENV using ci/lib.sh
   - step2:
     - make
   - step3:
     - make test
   - step4:
     - if: failure()
     - run ci/print-test-failures.sh
   - step5:
     - if: success()
     - run ci/check-unignored-build-artifacts.sh

There is a proposal[2] to get some of the benefits of this approach by
not re-arranging our variable setup in this way, but to instead use
the GitHub CI grouping syntax to focus on the relevant parts of "make"
or "make test" when we have failures.

Doing it this way makes for better looking GitHub CI UI, and lays much
better ground work for our CI going forward. Because:

 * The CI logic will be more portable to a future CI system, since a
   common feature of them is to run various commands in sequence, but
   a future system won't necessarily support the GitHub-specifics
   syntax of "grouping" output within a "step".

   Even if those systems don't support a "$GITHUB_ENV" emulating will
   be much easier than to deal with some CI-specific grouping syntax.

 * At the start of every step the GitHub CI presents an expandable
   list of environment variables from "$GITHUB_ENV". We'll now see
   exactly what variables affected that step (although we currently
   overshoot that a bit, and always define all variables).

 * CI failures will be easier to reproduce locally, as this makes the
   relevant ci/* scripts something that sets up our environment, but
   leaves "make" and "make test" working as they do locally.

   To reproduce a run the user only needs to set the variables
   discussed in the drop-down above, either manually or by running
   "ci/lib.sh".

 * The output will be less verbose. The "ci/lib.sh" script uses "set
   -x", and before this e.g. "ci/static-analysis.sh" would start with
   40 lines of trace output, culminating in using "export" to export
   the relevant environment variables.

   Now that verbosity is in the earlier "ci/lib.sh" step, and not in
   any subsequent one. The "make" targets then start out with the
   relevant non-trace output right away.

 * If we do want to use the grouping syntax within a "step" it'll now
   be easier to do so. It doesn't support nesting, so we'd have to
   make a choice between using it for e.g. "make" v.s. "make test", or
   individual test failures. See "sadly" in [3].

1. https://lore.kernel.org/git/211120.86k0h30zuw.gmgdl@evledraar.gmail.com/
2. https://lore.kernel.org/git/pull.1117.git.1643050574.gitgitgadget@gmail.com/
3. https://lore.kernel.org/git/9333ba781b8240f704e739b00d274f8c3d887e39.1643050574.git.gitgitgadget@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml            | 14 ++++++--
 ci/check-unignored-build-artifacts.sh |  8 +++++
 ci/install-dependencies.sh            |  4 +++
 ci/lib.sh                             | 36 +++++++++++++++++++-
 ci/make-test-artifacts.sh             |  2 --
 ci/run-build-and-tests.sh             | 47 ---------------------------
 ci/run-test-slice.sh                  |  2 --
 ci/test-documentation.sh              |  1 -
 8 files changed, 58 insertions(+), 56 deletions(-)
 create mode 100755 ci/check-unignored-build-artifacts.sh
 delete mode 100755 ci/run-build-and-tests.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index ad752010102..57b2db06923 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -268,7 +268,10 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/run-build-and-tests.sh
+    - run: ci/lib.sh
+    - run: make
+    - run: make test
+      if: success()
     - run: ci/print-test-failures.sh
       if: failure()
     - name: Upload failed tests' directories
@@ -292,6 +295,7 @@ jobs:
           image: daald/ubuntu32:xenial
         - jobname: pedantic
           image: fedora
+          skip-tests: yes
     env:
       jobname: ${{matrix.vector.jobname}}
     runs-on: ubuntu-latest
@@ -299,9 +303,12 @@ jobs:
     steps:
     - uses: actions/checkout@v1
     - run: ci/install-docker-dependencies.sh
-    - run: ci/run-build-and-tests.sh
+    - run: ci/lib.sh
+    - run: make
+    - run: make test
+      if: success() && matrix.vector.skip-tests != 'yes'
     - run: ci/print-test-failures.sh
-      if: failure()
+      if: failure() && matrix.vector.skip-tests != 'yes'
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v1
@@ -317,6 +324,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
+    - run: ci/lib.sh
     - run: make ci-static-analysis
   sparse:
     needs: ci-config
diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh
new file mode 100755
index 00000000000..56d04b0db9a
--- /dev/null
+++ b/ci/check-unignored-build-artifacts.sh
@@ -0,0 +1,8 @@
+#!/bin/sh
+#
+# Check whether the build created anything not in our .gitignore
+#
+
+. ${0%/*}/lib.sh
+
+check_unignored_build_artifacts
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 41a9185bb1e..5d50604e724 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -22,6 +22,10 @@ ubuntu-latest)
 	P4_PATH="$HOME/custom/p4"
 	GIT_LFS_PATH="$HOME/custom/git-lfs"
 	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
+	if test -n "$GITHUB_PATH"
+	then
+		echo "$PATH" >>"$GITHUB_PATH"
+	fi
 
 	P4WHENCE=https://cdist2.perforce.com/perforce/r$LINUX_P4_VERSION
 	LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
diff --git a/ci/lib.sh b/ci/lib.sh
index 71e6d6e08e7..d18ee1a91e7 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,3 +1,5 @@
+#!/bin/sh
+
 # Library of functions shared by all CI scripts
 
 # Set 'exit on error' for all CI scripts to let the caller know that
@@ -39,7 +41,10 @@ setenv () {
 	val=$2
 	shift 2
 
-	eval "export $key=\"$val\""
+	if test -n "$GITHUB_ENV"
+	then
+		echo "$key=$val" >>"$GITHUB_ENV"
+	fi
 }
 
 check_unignored_build_artifacts ()
@@ -109,6 +114,35 @@ macos-latest)
 esac
 
 case "$jobname" in
+linux-gcc)
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
+	;;
+linux-TEST-vars)
+	setenv --test GIT_TEST_SPLIT_INDEX yes
+	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
+	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
+	setenv --test GIT_TEST_OE_SIZE 10
+	setenv --test GIT_TEST_OE_DELTA_SIZE 5
+	setenv --test GIT_TEST_COMMIT_GRAPH 1
+	setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX 1
+	setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
+	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
+	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
+	setenv --test GIT_TEST_WRITE_REV_INDEX 1
+	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
+	;;
+linux-clang)
+	setenv --test GIT_TEST_DEFAULT_HASH sha1
+	;;
+linux-sha256)
+	setenv --test GIT_TEST_DEFAULT_HASH sha256
+	;;
+pedantic)
+	# Don't run the tests; we only care about whether Git can be
+	# built.
+	setenv --build DEVOPTS pedantic
+	;;
 linux32)
 	CC=gcc
 	;;
diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh
index 646967481f6..45298562982 100755
--- a/ci/make-test-artifacts.sh
+++ b/ci/make-test-artifacts.sh
@@ -8,5 +8,3 @@ mkdir -p "$1" # in case ci/lib.sh decides to quit early
 . ${0%/*}/lib.sh
 
 make artifacts-tar ARTIFACTS_DIRECTORY="$1"
-
-check_unignored_build_artifacts
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
deleted file mode 100755
index 35d45a9373e..00000000000
--- a/ci/run-build-and-tests.sh
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/bin/sh
-#
-# Build and test Git
-#
-
-. ${0%/*}/lib.sh
-
-export MAKE_TARGETS="all test"
-
-case "$jobname" in
-linux-gcc)
-	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
-	;;
-linux-TEST-vars)
-	setenv --test GIT_TEST_SPLIT_INDEX yes
-	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
-	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
-	setenv --test GIT_TEST_OE_SIZE 10
-	setenv --test GIT_TEST_OE_DELTA_SIZE 5
-	setenv --test GIT_TEST_COMMIT_GRAPH 1
-	setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
-	setenv --test GIT_TEST_MULTI_PACK_INDEX 1
-	setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
-	setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
-	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
-	setenv --test GIT_TEST_WRITE_REV_INDEX 1
-	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
-	;;
-linux-clang)
-	setenv --test GIT_TEST_DEFAULT_HASH sha1
-	;;
-linux-sha256)
-	setenv --test GIT_TEST_DEFAULT_HASH sha256
-	;;
-pedantic)
-	# Don't run the tests; we only care about whether Git can be
-	# built.
-	setenv --build DEVOPTS pedantic
-	export MAKE_TARGETS=all
-	;;
-esac
-
-# Any new "test" targets should not go after this "make", but should
-# adjust $MAKE_TARGETS. Otherwise compilation-only targets above will
-# start running tests.
-make $MAKE_TARGETS
-check_unignored_build_artifacts
diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index b9a682b4bcd..70326961454 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -8,5 +8,3 @@
 make --quiet -C t T="$(cd t &&
 	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
 	tr '\n' ' ')"
-
-check_unignored_build_artifacts
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 41e2b126311..6b5cce03bd7 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -39,4 +39,3 @@ test -s Documentation/git.html
 grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
 
 rm -f stdout.log stderr.log stderr.raw
-check_unignored_build_artifacts
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 14/29] CI: make ci/{lib,install-dependencies}.sh POSIX-compatible

[Ævar Arnfjörð Bjarmason]

Change the unportable constructs in the "ci/install-dependencies.sh"
scripts to use POSIX replacements, and have it run under "/bin/sh"
instead of "/usr/bin/env bash". We needed to:

* Replace pushd/popd with a sub-shell.
* Change "mkdir --parents" to "mkdir -p".
* Use POSIX "export" syntax.

This is in preparation for combining this script and
"ci/install-docker-dependencies.sh", which doesn't have access to
"bash".

While we're at it change "if [" to "if test" in ci/lib.sh. This isn't
strictly speaking a case of POSIX-incompatibility, but that syntax was
being used there because this code was using idiomatic bash
constructs. Let's have it use our usual style of "if test" instead.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/install-dependencies.sh | 27 +++++++++++++++++----------
 ci/lib.sh                  |  4 ++--
 2 files changed, 19 insertions(+), 12 deletions(-)

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 5d50604e724..3e05d38d4ab 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -1,4 +1,4 @@
-#!/usr/bin/env bash
+#!/bin/sh
 #
 # Install dependencies required to build and test Git on Linux and macOS
 #
@@ -21,7 +21,8 @@ ubuntu-latest)
 
 	P4_PATH="$HOME/custom/p4"
 	GIT_LFS_PATH="$HOME/custom/git-lfs"
-	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
+	PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
+	export PATH
 	if test -n "$GITHUB_PATH"
 	then
 		echo "$PATH" >>"$GITHUB_PATH"
@@ -33,22 +34,28 @@ ubuntu-latest)
 	sudo apt-get -q update
 	sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \
 		$UBUNTU_COMMON_PKGS $CC_PACKAGE
-	mkdir --parents "$P4_PATH"
-	pushd "$P4_PATH"
+	mkdir -p "$P4_PATH"
+	(
+		cd "$P4_PATH"
 		wget --quiet "$P4WHENCE/bin.linux26x86_64/p4d"
 		wget --quiet "$P4WHENCE/bin.linux26x86_64/p4"
 		chmod u+x p4d
 		chmod u+x p4
-	popd
-	mkdir --parents "$GIT_LFS_PATH"
-	pushd "$GIT_LFS_PATH"
+	)
+	mkdir -p "$GIT_LFS_PATH"
+	(
+		cd "$GIT_LFS_PATH"
 		wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
 		tar --extract --gunzip --file "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
 		cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs .
-	popd
+	)
 	;;
 macos-latest)
-	export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
+	HOMEBREW_NO_AUTO_UPDATE=1
+	export HOMEBREW_NO_AUTO_UPDATE
+	HOMEBREW_NO_INSTALL_CLEANUP=1
+	export HOMEBREW_NO_INSTALL_CLEANUP
+
 	# Uncomment this if you want to run perf tests:
 	# brew install gnu-time
 	brew link --force gettext
@@ -64,7 +71,7 @@ macos-latest)
 
 	if test -n "$CC_PACKAGE"
 	then
-		BREW_PACKAGE=${CC_PACKAGE/-/@}
+		BREW_PACKAGE=$(echo $CC_PACKAGE | tr '-' '@')
 		brew install "$BREW_PACKAGE"
 		brew link "$BREW_PACKAGE"
 	fi
diff --git a/ci/lib.sh b/ci/lib.sh
index d18ee1a91e7..8df133b840b 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -94,7 +94,7 @@ ubuntu-latest)
 		break
 	fi
 
-	if [ "$jobname" = linux-gcc ]
+	if test "$jobname" = linux-gcc
 	then
 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python3"
 	else
@@ -104,7 +104,7 @@ ubuntu-latest)
 	setenv --test GIT_TEST_HTTPD true
 	;;
 macos-latest)
-	if [ "$jobname" = osx-gcc ]
+	if test "$jobname" = osx-gcc
 	then
 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
 	else
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 15/29] CI: check ignored unignored build artifacts in "win[+VS] build" too

[Ævar Arnfjörð Bjarmason]

Use the newly created "ci/check-unignored-build-artifacts.sh" script
in the "win build" and "win+VS build" steps too. These steps weren't
doing this sanity check of untracked files against .gitignore before,
but as the check passes for them let's keep doing that extra sanity
check.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 57b2db06923..c97fbd0f0b3 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -93,6 +93,9 @@ jobs:
         HOME: ${{runner.workspace}}
         NO_PERL: 1
       run: . /etc/profile && ci/make-test-artifacts.sh artifacts
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
+      shell: bash
     - name: zip up tracked files
       run: git archive -o artifacts/tracked.tar.gz HEAD
     - name: upload tracked files and build artifacts
@@ -180,6 +183,9 @@ jobs:
       run: |
         mkdir -p artifacts &&
         eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts NO_GETTEXT=YesPlease 2>&1 | grep ^tar)"
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
+      shell: bash
     - name: zip up tracked files
       run: git archive -o artifacts/tracked.tar.gz HEAD
     - name: upload tracked files and build artifacts
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 16/29] CI: invoke "make artifacts-tar" directly in windows-build

[Ævar Arnfjörð Bjarmason]

Change the windows-build job to invoke the "make artifacts-tar" step
directly, instead of calling a "ci/make-test-artifacts.sh" script.

The script was needed because "ci/lib.sh" would set up various
environment variables for us, but now we can instead use the
"ci/lib.sh" in its script mode.

The "mkdir -p" added in b819f1d2cec (ci: parallelize testing on
Windows, 2019-01-29) isn't needed, the same commit added that "mkdir
-p" to the "artifacts-tar" rule itself, so we can have "make" create
the directory for us.

This also has the benefit of making the "build" step less chatty,
since it won't start with the verbose "set -x" output, that's now
contained in the "ci/lib.sh" step.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml |  5 ++++-
 ci/make-test-artifacts.sh  | 10 ----------
 2 files changed, 4 insertions(+), 11 deletions(-)
 delete mode 100755 ci/make-test-artifacts.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index c97fbd0f0b3..ca993afd098 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -87,12 +87,15 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
+    - run: ci/lib.sh
+      shell: bash
     - name: build
       shell: bash
       env:
         HOME: ${{runner.workspace}}
         NO_PERL: 1
-      run: . /etc/profile && ci/make-test-artifacts.sh artifacts
+      run: . /etc/profile && make artifacts-tar ARTIFACTS_DIRECTORY=artifacts
+      if: success()
     - run: ci/check-unignored-build-artifacts.sh
       if: success()
       shell: bash
diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh
deleted file mode 100755
index 45298562982..00000000000
--- a/ci/make-test-artifacts.sh
+++ /dev/null
@@ -1,10 +0,0 @@
-#!/bin/sh
-#
-# Build Git and store artifacts for testing
-#
-
-mkdir -p "$1" # in case ci/lib.sh decides to quit early
-
-. ${0%/*}/lib.sh
-
-make artifacts-tar ARTIFACTS_DIRECTORY="$1"
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 17/29] CI: split up and reduce "ci/test-documentation.sh"

[Ævar Arnfjörð Bjarmason]

Change the "ci/test-documentation.sh" script to run the bash-specific
parts in as one command in the CI job itself, and to run the two "make
doc" commands at the top-level.

It'll now be obvious from the title of the step if we failed in the
asciidoc or asciidoctor step.

Since the "check_unignored_build_artifacts()" function is now only
used in "ci/check-unignored-build-artifacts.sh" move that function
there.

The recipe for the job in ".github/workflows/main.yml" is now a bit
verbose because it's effectively the same job twice, with a "make
clean" in-between. It would be better for the verbosity to run it via
a matrix as done in the alternate approach in [1] does, but then we'd
sacrifice overall CPU time for the brevity. It might still be worth
doing, but let's go for this simpler approach for now.

1. https://lore.kernel.org/git/patch-v2-6.6-7c423c8283d-20211120T030848Z-avarab@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml            | 16 +++++++++++++-
 ci/check-unignored-build-artifacts.sh | 10 +++++++++
 ci/lib.sh                             | 10 ---------
 ci/test-documentation.sh              | 31 +++++++++------------------
 4 files changed, 35 insertions(+), 32 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index ca993afd098..88eb2ef2dad 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -365,4 +365,18 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/test-documentation.sh
+    - run: ci/lib.sh
+    - run: make check-docs
+    - run: "make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)"
+      shell: bash
+    - run: ci/test-documentation.sh AsciiDoc
+      if: success()
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
+    - run: make clean
+    - run: "make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)"
+      shell: bash
+    - run: ci/test-documentation.sh Asciidoctor
+      if: success()
+    - run: ci/check-unignored-build-artifacts.sh
+      if: success()
diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh
index 56d04b0db9a..0bc04f32804 100755
--- a/ci/check-unignored-build-artifacts.sh
+++ b/ci/check-unignored-build-artifacts.sh
@@ -5,4 +5,14 @@
 
 . ${0%/*}/lib.sh
 
+check_unignored_build_artifacts ()
+{
+	! git ls-files --other --exclude-standard --error-unmatch \
+		-- ':/*' 2>/dev/null ||
+	{
+		echo "$(tput setaf 1)error: found unignored build artifacts$(tput sgr0)"
+		false
+	}
+}
+
 check_unignored_build_artifacts
diff --git a/ci/lib.sh b/ci/lib.sh
index 8df133b840b..5141de85024 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -47,16 +47,6 @@ setenv () {
 	fi
 }
 
-check_unignored_build_artifacts ()
-{
-	! git ls-files --other --exclude-standard --error-unmatch \
-		-- ':/*' 2>/dev/null ||
-	{
-		echo "$(tput setaf 1)error: found unignored build artifacts$(tput sgr0)"
-		false
-	}
-}
-
 # GitHub Action doesn't set TERM, which is required by tput
 setenv TERM ${TERM:-dumb}
 
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 6b5cce03bd7..b8a6a6f664e 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -1,10 +1,12 @@
-#!/usr/bin/env bash
+#!/bin/sh
 #
-# Perform sanity checks on documentation and build it.
+# Perform sanity checks on "make doc" output and built documentation
 #
 
 . ${0%/*}/lib.sh
 
+generator=$1
+
 filter_log () {
 	sed -e '/^GIT_VERSION = /d' \
 	    -e "/constant Gem::ConfigMap is deprecated/d" \
@@ -14,28 +16,15 @@ filter_log () {
 	    "$1"
 }
 
-make check-docs
-
-# Build docs with AsciiDoc
-make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
-cat stderr.raw
-filter_log stderr.raw >stderr.log
-test ! -s stderr.log
-test -s Documentation/git.html
-test -s Documentation/git.xml
-test -s Documentation/git.1
-grep '<meta name="generator" content="AsciiDoc ' Documentation/git.html
-
-rm -f stdout.log stderr.log stderr.raw
-check_unignored_build_artifacts
-
-# Build docs with AsciiDoctor
-make clean
-make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
 cat stderr.raw
 filter_log stderr.raw >stderr.log
 test ! -s stderr.log
 test -s Documentation/git.html
-grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
+if test "$generator" = "Asciidoctor"
+then
+	test -s Documentation/git.xml
+	test -s Documentation/git.1
+fi
+grep "<meta name=\"generator\" content=\"$generator " Documentation/git.html
 
 rm -f stdout.log stderr.log stderr.raw
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 18/29] CI: combine ci/install{,-docker}-dependencies.sh

[Ævar Arnfjörð Bjarmason]

Combine the "ci/install-dependencies.sh" and
"ci/install-docker-dependencies.sh" into one script. The "case"
statement in the latter only cared about "$jobname", and can be folded
into the same "case" statement in the former.

The reason they split up is historical, until a preceding commit
"ci/lib.sh" required "bash", which might not have been available in
"docker".

This also fixes issue in "ci/install-docker-dependencies.sh" where
we'd hide errors due to not using "set -e". Now that we include
"ci/lib.sh" we won't have that potential issue.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml        |  2 +-
 ci/install-dependencies.sh        | 15 +++++++++++++++
 ci/install-docker-dependencies.sh | 22 ----------------------
 3 files changed, 16 insertions(+), 23 deletions(-)
 delete mode 100755 ci/install-docker-dependencies.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 88eb2ef2dad..42944020436 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -311,7 +311,7 @@ jobs:
     container: ${{matrix.vector.image}}
     steps:
     - uses: actions/checkout@v1
-    - run: ci/install-docker-dependencies.sh
+    - run: ci/install-dependencies.sh
     - run: ci/lib.sh
     - run: make
     - run: make test
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 3e05d38d4ab..cc7fd3bd7f8 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -99,6 +99,21 @@ linux-gcc-default)
 	sudo apt-get -q update
 	sudo apt-get -q -y install $UBUNTU_COMMON_PKGS
 	;;
+linux32)
+	linux32 --32bit i386 sh -c '
+		apt update >/dev/null &&
+		apt install -y build-essential libcurl4-openssl-dev \
+			libssl-dev libexpat-dev gettext python >/dev/null
+	'
+	;;
+linux-musl)
+	apk add --update build-base curl-dev openssl-dev expat-dev gettext \
+		pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null
+	;;
+pedantic)
+	dnf -yq update >/dev/null &&
+	dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null
+	;;
 esac
 
 if type p4d >/dev/null 2>&1 && type p4 >/dev/null 2>&1
diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh
deleted file mode 100755
index 78b7e326da6..00000000000
--- a/ci/install-docker-dependencies.sh
+++ /dev/null
@@ -1,22 +0,0 @@
-#!/bin/sh
-#
-# Install dependencies required to build and test Git inside container
-#
-
-case "$jobname" in
-linux32)
-	linux32 --32bit i386 sh -c '
-		apt update >/dev/null &&
-		apt install -y build-essential libcurl4-openssl-dev \
-			libssl-dev libexpat-dev gettext python >/dev/null
-	'
-	;;
-linux-musl)
-	apk add --update build-base curl-dev openssl-dev expat-dev gettext \
-		pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null
-	;;
-pedantic)
-	dnf -yq update >/dev/null &&
-	dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null
-	;;
-esac
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 19/29] CI: move "env" definitions into ci/lib.sh

[Ævar Arnfjörð Bjarmason]

Have the GitHub CI jobs use the "ci/lib".sh as a source of truth for
environment variables that affect their builds and tests. This brings
them in line with the rest of the jobs, and make it easier to see the
entire CI configuration per jobname at a glance.

To do this we need to add a ci/lib.sh "step" to the remaining jobs
that didn't have it added in preceding commits.

The Makefile parameters are luckily all accepted via the environment,
so we can export these instead of adding them to MAKEFLAGS.

Let's also use the documented GIT_CONIFG_* mechanism for setting
config instead of the internal GIT_CONFIG_PARAMETERS variable. This
adjusts code added in 889cacb6897 (ci: configure GitHub Actions for
CI/PR, 2020-04-11), we could probably use the GIT_AUTHOR_NAME etc, but
let's keep setting this via config, just with the documented
mechanism.

By setting "ARTIFACTS_DIRECTORY=artifacts" once we don't need to
repeat it in various places, let's get it from the environment
instead.

Finally, the "DEVELOPER: 1" was only needed by the "sparse" job, which
wasn't using "ci/lib.sh" (and which sets DEVELOPER=1). Let's instead
have the "sparse" job use the "ci/lib.sh" and remove DEVELOPER=1 from
".github/workflows/main.yml".

This substantially speeds up the "sparse" job, since it'll now pick up
the "--jobs" setting in MAKEFLAGS that we use everywhere else. Before
it ran in around 4m30s, now in around 2m30s.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 29 +++++++++++------------------
 ci/lib.sh                  | 21 +++++++++++++++++++++
 2 files changed, 32 insertions(+), 18 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 42944020436..5f167ac5be6 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -2,9 +2,6 @@ name: CI
 
 on: [push, pull_request]
 
-env:
-  DEVELOPER: 1
-
 jobs:
   ci-config:
     name: config
@@ -93,8 +90,7 @@ jobs:
       shell: bash
       env:
         HOME: ${{runner.workspace}}
-        NO_PERL: 1
-      run: . /etc/profile && make artifacts-tar ARTIFACTS_DIRECTORY=artifacts
+      run: . /etc/profile && make artifacts-tar
       if: success()
     - run: ci/check-unignored-build-artifacts.sh
       if: success()
@@ -105,7 +101,7 @@ jobs:
       uses: actions/upload-artifact@v2
       with:
         name: windows-artifacts
-        path: artifacts
+        path: ${{env.ARTIFACTS_DIRECTORY}}
   windows-test:
     name: win test
     env:
@@ -126,6 +122,8 @@ jobs:
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
+    - run: ci/lib.sh
+      shell: bash
     - name: test
       shell: bash
       run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
@@ -145,9 +143,6 @@ jobs:
       jobname: vs-build
     needs: ci-config
     if: needs.ci-config.outputs.enabled == 'yes'
-    env:
-      NO_PERL: 1
-      GIT_CONFIG_PARAMETERS: "'user.name=CI' 'user.email=ci@git'"
     runs-on: windows-latest
     steps:
     - uses: actions/checkout@v2
@@ -171,6 +166,8 @@ jobs:
     - name: copy dlls to root
       shell: cmd
       run: compat\vcbuild\vcpkg_copy_dlls.bat release
+    - run: ci/lib.sh
+      shell: bash
     - name: generate Visual Studio solution
       shell: bash
       run: |
@@ -181,21 +178,16 @@ jobs:
     - name: bundle artifact tar
       shell: bash
       env:
-        MSVC: 1
         VCPKG_ROOT: ${{github.workspace}}\compat\vcbuild\vcpkg
       run: |
-        mkdir -p artifacts &&
-        eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts NO_GETTEXT=YesPlease 2>&1 | grep ^tar)"
-    - run: ci/check-unignored-build-artifacts.sh
-      if: success()
-      shell: bash
+        eval "$(make -n artifacts-tar 2>&1 | grep -e ^mkdir -e ^tar)"
     - name: zip up tracked files
       run: git archive -o artifacts/tracked.tar.gz HEAD
     - name: upload tracked files and build artifacts
       uses: actions/upload-artifact@v2
       with:
         name: vs-artifacts
-        path: artifacts
+        path: ${{env.ARTIFACTS_DIRECTORY}}
   vs-test:
     name: win+VS test
     env:
@@ -216,10 +208,10 @@ jobs:
     - name: extract tracked files and build artifacts
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
+    - run: ci/lib.sh
+      shell: bash
     - name: test
       shell: bash
-      env:
-        NO_SVN_TESTS: 1
       run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
     - name: ci/print-test-failures.sh
       if: failure()
@@ -354,6 +346,7 @@ jobs:
     - uses: actions/checkout@v2
     - name: Install other dependencies
       run: ci/install-dependencies.sh
+    - run: ci/lib.sh
     - run: make sparse
   documentation:
     name: documentation
diff --git a/ci/lib.sh b/ci/lib.sh
index 5141de85024..de61914134e 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -104,6 +104,27 @@ macos-latest)
 esac
 
 case "$jobname" in
+windows-build)
+	setenv --build NO_PERL NoThanks
+	setenv --build ARTIFACTS_DIRECTORY artifacts
+	;;
+vs-build)
+	setenv --build NO_PERL NoThanks
+	setenv --build NO_GETTEXT NoThanks
+	setenv --build ARTIFACTS_DIRECTORY artifacts
+	setenv --build INCLUDE_DLLS_IN_ARTIFACTS YesPlease
+	setenv --build MSVC YesPlease
+
+	setenv --build GIT_CONFIG_COUNT 2
+	setenv --build GIT_CONFIG_KEY_0 user.name
+	setenv --build GIT_CONFIG_VALUE_0 CI
+	setenv --build GIT_CONFIG_KEY_1 user.emailname
+	setenv --build GIT_CONFIG_VALUE_1 ci@git
+	setenv --build GIT_CONFIG_VALUE_1 ci@git
+	;;
+vs-test)
+	setenv --test NO_SVN_TESTS YesPlease
+	;;
 linux-gcc)
 	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
 	;;
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 20/29] ci/run-test-slice.sh: replace shelling out with "echo"

[Ævar Arnfjörð Bjarmason]

Amend this code added in b819f1d2cec (ci: parallelize testing on
Windows, 2019-01-29) to stop shelling out to "tr", all we're trying to
do here is to turn '\n'-delimited output into a ' '-delimited list. To
do that we can just use "echo".

None of these files will contain the odd characters using "echo" might
choke on in the general case (i.e. '\'-escapes and the like).

This has the added benefit of not adding a redundant ' ' to the end of
the T-parameter, which didn't matter in practice, but as we'll see in
a subsequent commit makes for slightly nicer output, as we'll show
this variable using a GitHub CI-native feature.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/run-test-slice.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index 70326961454..1de6a18ca47 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -5,6 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-make --quiet -C t T="$(cd t &&
-	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
-	tr '\n' ' ')"
+tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \
+	t[0-9]*.sh))
+make --quiet -C t T="$tests"
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 21/29] CI: pre-select test slice in Windows & VS tests

[Ævar Arnfjörð Bjarmason]

In preceding commits the tests have been changed to do their setup via
$GITHUB_ENV in one step, and to then have subsequent steps that re-use
that environment.

Let's change the "test slice" tests added in b819f1d2cec (ci:
parallelize testing on Windows, 2019-01-29) to do the same. These
tests select 10% of the tests to run in 10 "test slices". Now we'll
select those in a step that immediately precedes the testing step, and
then simply invoke "make -C t -e".

This has the advantage that the tests to be run are now listed in the
standard "Run" drop-down at the start of the "test" step.

Since the "T" variable in "t/Makefile" doesn't normally accept
overrides from the environment we need to invoke "make" with the "-e"
option (a.k.a. "--environment-overrides"). We could also make $(T) in
t/Makefile be a "?=" assigned variable, but this way works, and is
arguably clearer as it's more obvious that we're injecting a special
list of tests that override the normal behavior of that Makefile.

Note that we cannot run the top-level "make test" here, because of how
the Windows CI builds git, i.e. either via CMake or some option that
would cause "make test" to recompile git itself. Instead we run "make
-C t [...]".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml                     | 10 ++++++++--
 ci/{run-test-slice.sh => select-test-slice.sh} |  7 +++++--
 2 files changed, 13 insertions(+), 4 deletions(-)
 rename ci/{run-test-slice.sh => select-test-slice.sh} (50%)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 5f167ac5be6..83e0aa1f469 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -124,9 +124,12 @@ jobs:
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
     - run: ci/lib.sh
       shell: bash
+    - name: select tests
+      run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
+      shell: bash
     - name: test
       shell: bash
-      run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
+      run: . /etc/profile && make -C t -e
     - name: ci/print-test-failures.sh
       if: failure()
       shell: bash
@@ -210,9 +213,12 @@ jobs:
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
     - run: ci/lib.sh
       shell: bash
+    - name: select tests
+      run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
+      shell: bash
     - name: test
       shell: bash
-      run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
+      run: . /etc/profile && make -C t -e
     - name: ci/print-test-failures.sh
       if: failure()
       shell: bash
diff --git a/ci/run-test-slice.sh b/ci/select-test-slice.sh
similarity index 50%
rename from ci/run-test-slice.sh
rename to ci/select-test-slice.sh
index 1de6a18ca47..f59d8cadda5 100755
--- a/ci/run-test-slice.sh
+++ b/ci/select-test-slice.sh
@@ -1,10 +1,13 @@
 #!/bin/sh
 #
-# Test Git in parallel
+# Select a portion of the tests for testing Git in parallel
 #
 
 . ${0%/*}/lib.sh
 
 tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \
 	t[0-9]*.sh))
-make --quiet -C t T="$tests"
+if test -n "$GITHUB_ENV"
+then
+	echo T="$tests" >>$GITHUB_ENV
+fi
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 22/29] CI: only invoke ci/lib.sh as "steps" in main.yml

[Ævar Arnfjörð Bjarmason]

Change the scripts in ci/ to stop using ci/lib.sh as a library, now
that the only thing it did for them was to "set -ex" and possibly set
TERM=dumb.

Let's create a ci/lib-tput.sh for those that need to use "tput"
instead, and have these scripts invoke "set -ex" themselves.

This makes their invocation a lot less verbose, since they'll be
relying on an earlier step in the CI job to have set the variables in
$GITHUB_ENV, and won't be spewing their own trace output to set those
variables again.

Let's also create a ci/lib-ci-type.sh, and have ci/lib.sh and
ci/print-test-failures.sh share the logic to discover the CI type. We
could have set the CI_TYPE in the environment with "setenv", but let's
avoid that verbosity for this purely internal variable.

The "ci/lib.sh" is now no longer a "Library of functions shared by all
CI scripts", so let's remove that commentary, and the misleading
comment about "set -ex" being for "installing dependencies", we're now
no longer using it in "ci/install-dependencies.sh" (but it does its
own "set -ex").

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/check-unignored-build-artifacts.sh |  4 +++-
 ci/install-dependencies.sh            |  2 +-
 ci/lib-ci-type.sh                     |  9 +++++++++
 ci/lib-tput.sh                        |  5 +++++
 ci/lib.sh                             | 27 ++++++++++-----------------
 ci/print-test-failures.sh             |  6 +++---
 ci/select-test-slice.sh               |  2 +-
 ci/test-documentation.sh              |  2 +-
 8 files changed, 33 insertions(+), 24 deletions(-)
 create mode 100644 ci/lib-ci-type.sh
 create mode 100644 ci/lib-tput.sh

diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh
index 0bc04f32804..c27d6a97f45 100755
--- a/ci/check-unignored-build-artifacts.sh
+++ b/ci/check-unignored-build-artifacts.sh
@@ -3,7 +3,9 @@
 # Check whether the build created anything not in our .gitignore
 #
 
-. ${0%/*}/lib.sh
+set -ex
+
+. ${0%/*}/lib-tput.sh
 
 check_unignored_build_artifacts ()
 {
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index cc7fd3bd7f8..9ee52769cbe 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -3,7 +3,7 @@
 # Install dependencies required to build and test Git on Linux and macOS
 #
 
-. ${0%/*}/lib.sh
+set -ex
 
 UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
  tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
diff --git a/ci/lib-ci-type.sh b/ci/lib-ci-type.sh
new file mode 100644
index 00000000000..6f01fd9e5d9
--- /dev/null
+++ b/ci/lib-ci-type.sh
@@ -0,0 +1,9 @@
+#!/bin/sh
+
+if test "$GITHUB_ACTIONS" = "true"
+then
+	CI_TYPE=github-actions
+else
+	echo "Could not identify CI type" >&2
+	exit 1
+fi
diff --git a/ci/lib-tput.sh b/ci/lib-tput.sh
new file mode 100644
index 00000000000..b62ef13c44c
--- /dev/null
+++ b/ci/lib-tput.sh
@@ -0,0 +1,5 @@
+#!/bin/sh
+
+# GitHub Action doesn't set TERM, which is required by tput
+TERM=${TERM:-dumb}
+export TERM
diff --git a/ci/lib.sh b/ci/lib.sh
index de61914134e..15a0011f6aa 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,13 +1,9 @@
 #!/bin/sh
-
-# Library of functions shared by all CI scripts
-
-# Set 'exit on error' for all CI scripts to let the caller know that
-# something went wrong.
-# Set tracing executed commands, primarily setting environment variables
-# and installing dependencies.
 set -ex
 
+# Helper libraries
+. ${0%/*}/lib-ci-type.sh
+
 # Starting assertions
 if test -z "$jobname"
 then
@@ -47,18 +43,14 @@ setenv () {
 	fi
 }
 
-# GitHub Action doesn't set TERM, which is required by tput
-setenv TERM ${TERM:-dumb}
-
 # How many jobs to run in parallel?
 NPROC=10
 
 # Clear MAKEFLAGS that may come from the outside world.
 MAKEFLAGS=--jobs=$NPROC
 
-if test "$GITHUB_ACTIONS" = "true"
-then
-	CI_TYPE=github-actions
+case "$CI_TYPE" in
+github-actions)
 	CC="${CC_PACKAGE:-${CC:-gcc}}"
 
 	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
@@ -66,11 +58,12 @@ then
 	test Windows != "$RUNNER_OS" ||
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
-else
-	echo "Could not identify CI type" >&2
-	env >&2
+	;;
+*)
+	echo "Unhandled CI type: $CI_TYPE" >&2
 	exit 1
-fi
+	;;
+esac
 
 setenv --build DEVELOPER 1
 setenv --test DEFAULT_TEST_TARGET prove
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 47e2ea1ad10..29f8c332eca 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -3,10 +3,10 @@
 # Print output of failing tests
 #
 
-. ${0%/*}/lib.sh
+set -e
 
-# Tracing executed commands would produce too much noise in the loop below.
-set +x
+. ${0%/*}/lib-ci-type.sh
+. ${0%/*}/lib-tput.sh
 
 cd t/
 
diff --git a/ci/select-test-slice.sh b/ci/select-test-slice.sh
index f59d8cadda5..eccf9a48104 100755
--- a/ci/select-test-slice.sh
+++ b/ci/select-test-slice.sh
@@ -3,7 +3,7 @@
 # Select a portion of the tests for testing Git in parallel
 #
 
-. ${0%/*}/lib.sh
+set -ex
 
 tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \
 	t[0-9]*.sh))
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index b8a6a6f664e..64ff212cdaa 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -3,7 +3,7 @@
 # Perform sanity checks on "make doc" output and built documentation
 #
 
-. ${0%/*}/lib.sh
+set -ex
 
 generator=$1
 
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 23/29] CI: narrow down variable definitions in --build and --test

[Ævar Arnfjörð Bjarmason]

In a preceding step the "setvar" function was made to take a
"--build", "--test" or "--all" argument to indicate where the
variables it sets were used.

Let's make use of that by having the relevant parts of
".github/workflows/main.yml" invoke "ci/lib.sh" with those options.

By doing this the set of variables shown in build-only steps will be
fewer, which makes diagnosing anything going on there easier, as we
won't have to look at a deluge of e.g. GIT_TEST_* variables.

For the "pedantic" job (which has no test phase) we won't run the
"ci/lib.sh --test" step, which will be clearly visible as a skipped
step in the UX.

Since we'll now always run "--build" for "make" and "--test" for "make
test" we can stop setting the "fat" MAKEFLAGS entirely on the "test"
steps, i.e. the one with PYTHON_PATH, CC etc. These will all be
carried over from the earlier --build step.

That won't be true in the case of the "windows-test" and "vs-test"
jobs, since they run separately from the corresponding "build"
step. So we'll need to make sure that we have the --jobs=N argument
for those.

This doesn't matter that much, as we'll still have --jobs=N in
GIT_PROVE_OPTS. So the only thing we'll use it for is parallelism in
the t/Makefile before we get to running "prove". Still, it's good to
be consistent for good measure, and to run the t/Makefile itself in
parallel.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 21 ++++++++-------
 ci/lib.sh                  | 52 +++++++++++++++++++++++++++++++++++---
 2 files changed, 60 insertions(+), 13 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 83e0aa1f469..bb62b4ff725 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -84,7 +84,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
       shell: bash
     - name: build
       shell: bash
@@ -122,7 +122,7 @@ jobs:
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
-    - run: ci/lib.sh
+    - run: ci/lib.sh --test
       shell: bash
     - name: select tests
       run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
@@ -169,7 +169,7 @@ jobs:
     - name: copy dlls to root
       shell: cmd
       run: compat\vcbuild\vcpkg_copy_dlls.bat release
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
       shell: bash
     - name: generate Visual Studio solution
       shell: bash
@@ -211,7 +211,7 @@ jobs:
     - name: extract tracked files and build artifacts
       shell: bash
       run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
-    - run: ci/lib.sh
+    - run: ci/lib.sh --test
       shell: bash
     - name: select tests
       run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
@@ -275,8 +275,9 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make
+    - run: ci/lib.sh --test
     - run: make test
       if: success()
     - run: ci/print-test-failures.sh
@@ -310,8 +311,10 @@ jobs:
     steps:
     - uses: actions/checkout@v1
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make
+    - run: ci/lib.sh --test
+      if: success() && matrix.vector.skip-tests != 'yes'
     - run: make test
       if: success() && matrix.vector.skip-tests != 'yes'
     - run: ci/print-test-failures.sh
@@ -331,7 +334,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make ci-static-analysis
   sparse:
     needs: ci-config
@@ -352,7 +355,7 @@ jobs:
     - uses: actions/checkout@v2
     - name: Install other dependencies
       run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make sparse
   documentation:
     name: documentation
@@ -364,7 +367,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2
     - run: ci/install-dependencies.sh
-    - run: ci/lib.sh
+    - run: ci/lib.sh --build
     - run: make check-docs
     - run: "make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)"
       shell: bash
diff --git a/ci/lib.sh b/ci/lib.sh
index 15a0011f6aa..96919882c11 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -4,6 +4,30 @@ set -ex
 # Helper libraries
 . ${0%/*}/lib-ci-type.sh
 
+# Parse options
+mode_build=
+mode_test=
+while test $# != 0
+do
+	case "$1" in
+	--build)
+		mode_build=t
+		;;
+	--test)
+		mode_test=t
+		;;
+	-*)
+		echo "error: invalid option: $1" >&2
+		exit 1
+		;;
+	*)
+		echo "error: invalid argument: $1" >&2
+		exit 1
+		;;
+	esac
+	shift
+done
+
 # Starting assertions
 if test -z "$jobname"
 then
@@ -11,16 +35,28 @@ then
 	exit 1
 fi
 
+if test "$mode_test$mode_build" != "t"
+then
+	echo "error: need one mode, e.g. --build or --test" >&2
+	exit 1
+fi
+
 # Helper functions
 setenv () {
 	while test $# != 0
 	do
 		case "$1" in
 		--build)
+			if test -z "$mode_build"
+			then
+				return 0
+			fi
 			;;
 		--test)
-			;;
-		--all)
+			if test -z "$mode_test"
+			then
+				return 0
+			fi
 			;;
 		-*)
 			echo "BUG: bad setenv() option '$1'" >&2
@@ -46,8 +82,12 @@ setenv () {
 # How many jobs to run in parallel?
 NPROC=10
 
+# For "--test" we carry the MAKEFLAGS over from earlier steps, except
+# in stand-alone jobs which will use $COMMON_MAKEFLAGS.
+COMMON_MAKEFLAGS=--jobs=$NPROC
+
 # Clear MAKEFLAGS that may come from the outside world.
-MAKEFLAGS=--jobs=$NPROC
+MAKEFLAGS=$COMMON_MAKEFLAGS
 
 case "$CI_TYPE" in
 github-actions)
@@ -101,6 +141,9 @@ windows-build)
 	setenv --build NO_PERL NoThanks
 	setenv --build ARTIFACTS_DIRECTORY artifacts
 	;;
+windows-test)
+	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
+	;;
 vs-build)
 	setenv --build NO_PERL NoThanks
 	setenv --build NO_GETTEXT NoThanks
@@ -117,6 +160,7 @@ vs-build)
 	;;
 vs-test)
 	setenv --test NO_SVN_TESTS YesPlease
+	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
 	;;
 linux-gcc)
 	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
@@ -162,4 +206,4 @@ linux-leaks)
 	;;
 esac
 
-setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
+setenv --build MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 24/29] CI: add more variables to MAKEFLAGS, except under vs-build

[Ævar Arnfjörð Bjarmason]

It's clearer that "DEVELOPER" is a flag that affects the Makefile
itself in particular if it's put into "MAKEFLAGS than" if it
generically sits in the environment. Let's move both it and
"SKIP_DASHED_BUILT_INS" to "MAKEFLAGS".

We can't do this under vs-build, since that invokes cmake. Let's have
only that job set these in the environment.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 96919882c11..16bac966d76 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -89,6 +89,13 @@ COMMON_MAKEFLAGS=--jobs=$NPROC
 # Clear MAKEFLAGS that may come from the outside world.
 MAKEFLAGS=$COMMON_MAKEFLAGS
 
+# Use common options for "make" (cmake in "vs-build" below uses the
+# intermediate variables directly)
+DEVELOPER=1
+MAKEFLAGS="$MAKEFLAGS DEVELOPER=$DEVELOPER"
+SKIP_DASHED_BUILT_INS=YesPlease
+MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
+
 case "$CI_TYPE" in
 github-actions)
 	CC="${CC_PACKAGE:-${CC:-gcc}}"
@@ -105,10 +112,8 @@ github-actions)
 	;;
 esac
 
-setenv --build DEVELOPER 1
 setenv --test DEFAULT_TEST_TARGET prove
 setenv --test GIT_TEST_CLONE_2GB true
-setenv --build SKIP_DASHED_BUILT_INS YesPlease
 
 case "$runs_on_pool" in
 ubuntu-latest)
@@ -145,6 +150,9 @@ windows-test)
 	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
 	;;
 vs-build)
+	setenv --build DEVELOPER $DEVELOPER
+	setenv --build SKIP_DASHED_BUILT_INS $SKIP_DASHED_BUILT_INS
+
 	setenv --build NO_PERL NoThanks
 	setenv --build NO_GETTEXT NoThanks
 	setenv --build ARTIFACTS_DIRECTORY artifacts
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 25/29] CI: set CC in MAKEFLAGS directly, don't add it to the environment

[Ævar Arnfjörð Bjarmason]

Rather than pass a "$CC" in the environment to be picked up in
ci/lib.sh let's instead have ci/lib.sh itself add it directly to
MAKEFLAGS. For "$CC_PACKAGE" its setting and use can stay within
ci/install-dependencies.sh.

Setting CC=gcc by default made for confusing trace output, and since a
preceding change to carry it and others over across "steps" in the
GitHub CI it's been even more misleading.  E.g. the "win+VS build" job
confusingly has CC=gcc set, even though it builds with MSVC.

Let's instead rely on the Makefile default of CC=cc, and only override
it for those jobs where it's needed. This does mean that we'll need to
set it for the "pedantic" job, which previously relied on the default
CC=gcc in case "clang" become the default on that platform.

This partially reverts my 707d2f2fe86 (CI: use "$runs_on_pool", not
"$jobname" to select packages & config, 2021-11-23), i.e. we're now
aiming to only set those variables specific jobs need.

Since we don't need to make this generic enough to handle "gcc-9"
turning into "gcc@9" for "brew install" let's remove what was
originally a bash-specific replacement in 707d2f2fe86 (CI: use
"$runs_on_pool", not "$jobname" to select packages & config,
2021-11-23), and which a preceding commit changed to a "tr"
invocation.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml | 13 -------------
 ci/install-dependencies.sh | 18 ++++++++++++++----
 ci/lib.sh                  | 23 ++++++++++++++++++++---
 3 files changed, 34 insertions(+), 20 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index bb62b4ff725..6835e942280 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -238,37 +238,24 @@ jobs:
       matrix:
         vector:
           - jobname: linux-clang
-            cc: clang
             pool: ubuntu-latest
           - jobname: linux-sha256
-            cc: clang
             os: ubuntu
             pool: ubuntu-latest
           - jobname: linux-gcc
-            cc: gcc
-            cc_package: gcc-8
             pool: ubuntu-latest
           - jobname: linux-TEST-vars
-            cc: gcc
             os: ubuntu
-            cc_package: gcc-8
             pool: ubuntu-latest
           - jobname: osx-clang
-            cc: clang
             pool: macos-latest
           - jobname: osx-gcc
-            cc: gcc
-            cc_package: gcc-9
             pool: macos-latest
           - jobname: linux-gcc-default
-            cc: gcc
             pool: ubuntu-latest
           - jobname: linux-leaks
-            cc: gcc
             pool: ubuntu-latest
     env:
-      CC: ${{matrix.vector.cc}}
-      CC_PACKAGE: ${{matrix.vector.cc_package}}
       jobname: ${{matrix.vector.jobname}}
       runs_on_pool: ${{matrix.vector.pool}}
     runs-on: ${{matrix.vector.pool}}
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 9ee52769cbe..b2be92f4370 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -9,6 +9,17 @@ UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
  tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
  libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl"
 
+CC_PACKAGE=
+BREW_CC_PACKAGE=
+case "$jobname" in
+linux-gcc | linux-TEST-vars)
+	CC_PACKAGE=gcc-8
+	;;
+osx-gcc)
+	BREW_CC_PACKAGE=gcc@9
+	;;
+esac
+
 case "$runs_on_pool" in
 ubuntu-latest)
 	# The Linux build installs the defined dependency versions below.
@@ -69,11 +80,10 @@ macos-latest)
 	PATH="$PATH:${HOME}/bin"
 	export PATH
 
-	if test -n "$CC_PACKAGE"
+	if test -n "$BREW_CC_PACKAGE"
 	then
-		BREW_PACKAGE=$(echo $CC_PACKAGE | tr '-' '@')
-		brew install "$BREW_PACKAGE"
-		brew link "$BREW_PACKAGE"
+		brew install "$BREW_CC_PACKAGE"
+		brew link "$BREW_CC_PACKAGE"
 	fi
 	;;
 esac
diff --git a/ci/lib.sh b/ci/lib.sh
index 16bac966d76..8d19ca5ced8 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -79,6 +79,9 @@ setenv () {
 	fi
 }
 
+# Clear variables that may come from the outside world.
+CC=
+
 # How many jobs to run in parallel?
 NPROC=10
 
@@ -98,8 +101,6 @@ MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
 
 case "$CI_TYPE" in
 github-actions)
-	CC="${CC_PACKAGE:-${CC:-gcc}}"
-
 	setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
 	GIT_TEST_OPTS="--verbose-log -x"
 	test Windows != "$RUNNER_OS" ||
@@ -171,9 +172,14 @@ vs-test)
 	setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS"
 	;;
 linux-gcc)
+	CC=gcc
 	setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main
 	;;
+linux-gcc-default)
+	CC=gcc
+	;;
 linux-TEST-vars)
+	CC=gcc
 	setenv --test GIT_TEST_SPLIT_INDEX yes
 	setenv --test GIT_TEST_MERGE_ALGORITHM recursive
 	setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true
@@ -188,13 +194,22 @@ linux-TEST-vars)
 	setenv --test GIT_TEST_WRITE_REV_INDEX 1
 	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
 	;;
+osx-gcc)
+	CC=gcc
+	;;
+osx-clang)
+	CC=clang
+	;;
 linux-clang)
+	CC=clang
 	setenv --test GIT_TEST_DEFAULT_HASH sha1
 	;;
 linux-sha256)
+	CC=clang
 	setenv --test GIT_TEST_DEFAULT_HASH sha256
 	;;
 pedantic)
+	CC=gcc
 	# Don't run the tests; we only care about whether Git can be
 	# built.
 	setenv --build DEVOPTS pedantic
@@ -209,9 +224,11 @@ linux-musl)
 	MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8"
 	;;
 linux-leaks)
+	CC=gcc
 	setenv --build SANITIZE leak
 	setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true
 	;;
 esac
 
-setenv --build MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}"
+MAKEFLAGS="$MAKEFLAGS${CC:+ CC=$CC}"
+setenv --build MAKEFLAGS "$MAKEFLAGS"
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 26/29] CI: set SANITIZE=leak in MAKEFLAGS directly

[Ævar Arnfjörð Bjarmason]

As in the preceding commit's change to do likewise for CC, let's not
set SANITIZE in the environment. When I added the "linux-leaks" job in
in 956d2e4639b (tests: add a test mode for SANITIZE=leak, run it in
CI, 2021-09-23) I missed that I could do that.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 8d19ca5ced8..b6e7a6647bb 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -225,7 +225,7 @@ linux-musl)
 	;;
 linux-leaks)
 	CC=gcc
-	setenv --build SANITIZE leak
+	MAKEFLAGS="$MAKEFLAGS SANITIZE=leak"
 	setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true
 	;;
 esac
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 27/29] CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case

[Ævar Arnfjörð Bjarmason]

Move the setting of "PYTHON_PATH=[...]" to the "$jobname" case
statement. This partially backs out of my 707d2f2fe86 (CI: use
"$runs_on_pool", not "$jobname" to select packages & config,
2021-11-23), now that we have a "osx-{clang,gcc}" anyway for setting
"$CC" we might as well do away with this part of the "$runs_on_pool"
case.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index b6e7a6647bb..f2f75901a9c 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -132,14 +132,6 @@ ubuntu-latest)
 
 	setenv --test GIT_TEST_HTTPD true
 	;;
-macos-latest)
-	if test "$jobname" = osx-gcc
-	then
-		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
-	else
-		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
-	fi
-	;;
 esac
 
 case "$jobname" in
@@ -195,9 +187,11 @@ linux-TEST-vars)
 	setenv --test GIT_TEST_CHECKOUT_WORKERS 2
 	;;
 osx-gcc)
+	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
 	CC=gcc
 	;;
 osx-clang)
+	MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
 	CC=clang
 	;;
 linux-clang)
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 28/29] CI: don't use "set -x" in "ci/lib.sh" output

[Ævar Arnfjörð Bjarmason]

Remove the "set -x" from the "ci/lib.sh" output. Before preceding
commits the logic in that file was much more complex, and likely to
fail in some scenarios.

Now we only task "ci/lib.sh" with setting various variables for
subsequent steps in our jobs, so we can start emitting more tailored
debugging output, which makes what it's doing easier to read.

This change also changes the output of the "ci/print-test-failures.sh"
script, since it's the only other user of "ci/lib-ci-type.sh". In that
case it's also helpful to know what "$CI_TYPE" we're using, as that
script doesn't "set -x" and will act differently depending on the
$CI_TYPE.

It's useful when developing this script to see what variables
it *doesn't* set, but such output isn't useful to users, so let's hide
the "SKIP" output behind a --debug option.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib.sh | 35 ++++++++++++++++++++++++++++++++---
 1 file changed, 32 insertions(+), 3 deletions(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index f2f75901a9c..9e38c893890 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,5 +1,5 @@
 #!/bin/sh
-set -ex
+set -e
 
 # Helper libraries
 . ${0%/*}/lib-ci-type.sh
@@ -7,15 +7,22 @@ set -ex
 # Parse options
 mode_build=
 mode_test=
+mode_debug=
 while test $# != 0
 do
 	case "$1" in
 	--build)
+		echo "MODE: $1" >&2
 		mode_build=t
 		;;
 	--test)
+		echo "MODE: $1" >&2
 		mode_test=t
 		;;
+	--debug)
+		echo "DEBUG: $1" >&2
+		mode_debug=t
+		;;
 	-*)
 		echo "error: invalid option: $1" >&2
 		exit 1
@@ -41,21 +48,32 @@ then
 	exit 1
 fi
 
+# Show our configuration
+echo "CONFIG: CI_TYPE=$CI_TYPE" >&2
+echo "CONFIG: jobname=$jobname" >&2
+echo "CONFIG: runs_on_pool=$runs_on_pool" >&2
+if test -n "$GITHUB_ENV"
+then
+	echo "CONFIG: GITHUB_ENV=$GITHUB_ENV" >&2
+fi
+echo >&2
+
 # Helper functions
 setenv () {
+	skip=
 	while test $# != 0
 	do
 		case "$1" in
 		--build)
 			if test -z "$mode_build"
 			then
-				return 0
+				skip=t
 			fi
 			;;
 		--test)
 			if test -z "$mode_test"
 			then
-				return 0
+				skip=t
 			fi
 			;;
 		-*)
@@ -73,10 +91,21 @@ setenv () {
 	val=$2
 	shift 2
 
+	if test -n "$skip"
+	then
+		if test -n "$mode_debug"
+		then
+			echo "SKIP: '$key=$val'" >&2
+		fi
+		return 0
+	fi
+
 	if test -n "$GITHUB_ENV"
 	then
 		echo "$key=$val" >>"$GITHUB_ENV"
 	fi
+
+	echo "SET: '$key=$val'" >&2
 }
 
 # Clear variables that may come from the outside world.
-- 
2.36.1.1045.gf356b5617dd

[PATCH v6 29/29] CI: make it easy to use ci/*.sh outside of CI

[Ævar Arnfjörð Bjarmason]

In preceding commits the ci/.sh scripts have lost most of their
CI-specific assumptions. Let's go even further and explicitly support
running ci/lib.sh outside of CI.

This was possible before by faking up enough CI-specific variables,
but as shown in the new "help" output being added here using the
ci/lib.sh to provide "CI-like" has now become trivial.

The ci/print-test-failures.sh scripts can now be used outside of CI as
well, the only GitHub CI-specific part is now guarded by a check that
we'll pass if outside of GitHub CI.

There's also a special-case here to not clobber $MAKEFLAGS in the
environment if we're outside of CI, in case the user has e.g. "jN" or
other flags to "make" that they'd prefer configured already.

Using "ci/lib.sh" as a stand-alone script is much more useful if it
doesn't hardcode NPROC=10, let's provide a poor shellscript
replacement for the online_cpus() we have in thread-utils.c to cover
the most common OS's.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 ci/lib-ci-type.sh         |  3 --
 ci/lib-online_cpus.sh     | 26 +++++++++++++
 ci/lib.sh                 | 80 ++++++++++++++++++++++++++++++++++-----
 ci/print-test-failures.sh |  4 --
 4 files changed, 96 insertions(+), 17 deletions(-)
 create mode 100644 ci/lib-online_cpus.sh

diff --git a/ci/lib-ci-type.sh b/ci/lib-ci-type.sh
index 6f01fd9e5d9..09acab7aaec 100644
--- a/ci/lib-ci-type.sh
+++ b/ci/lib-ci-type.sh
@@ -3,7 +3,4 @@
 if test "$GITHUB_ACTIONS" = "true"
 then
 	CI_TYPE=github-actions
-else
-	echo "Could not identify CI type" >&2
-	exit 1
 fi
diff --git a/ci/lib-online_cpus.sh b/ci/lib-online_cpus.sh
new file mode 100644
index 00000000000..7cf0cd042fa
--- /dev/null
+++ b/ci/lib-online_cpus.sh
@@ -0,0 +1,26 @@
+#!/bin/sh
+
+# TODO: Ideally we'd compile t/helper/test-online-cpus.c early, but
+# that currently presents a chicken & egg problem. We need this before
+# we build (much of) anything.
+online_cpus() {
+	NPROC=
+
+	if command -v nproc >/dev/null
+	then
+		# GNU coreutils
+		NPROC=$(nproc)
+	elif command -v sysctl >/dev/null
+	then
+		# BSD & Mac OS X
+		NPROC=$(sysctl -n hw.ncpu)
+	elif test -n "$NUMBER_OF_PROCESSORS"
+	then
+		# Windows
+		NPROC="$NUMBER_OF_PROCESSORS"
+	else
+		NPROC=1
+	fi
+
+	echo $NPROC
+}
diff --git a/ci/lib.sh b/ci/lib.sh
index 9e38c893890..80e89f89b7f 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,16 +1,55 @@
 #!/bin/sh
 set -e
 
+# Usage
+CI_TYPE_HELP_COMMANDS='
+	# run "make all test" like the "linux-leaks" job
+	(eval $(jobname=linux-leaks ci/lib.sh --all) && make test)
+
+	# run "make all test" like the "linux-musl" job
+	(eval $(jobname=linux-musl ci/lib.sh --all) && make test)
+
+	# run "make test" like the "linux-TEST-vars" job (uses various GIT_TEST_* modes)
+	make && (eval $(jobname=linux-TEST-vars ci/lib.sh --test) && make test)
+
+	# run "make test" like the "linux-sha256" job
+	make && (eval $(jobname=linux-sha256 ci/lib.sh --test) && make test)
+'
+
+CI_TYPE_HELP="
+Running $0 outside of CI? You can use ci/lib.sh to set up your
+environment like a given CI job. E.g.:
+$CI_TYPE_HELP_COMMANDS
+Note that some of these (e.g. the linux-musl one) may not work as
+expected due to the CI job configuring a platform that may not match
+yours."
+
+usage() {
+	echo "usage: jobname=<job-name> [runs_on_pool=<pool-name>] $0 (--build | --test)" >&2
+	if test -z "$CI_TYPE"
+	then
+		echo "$CI_TYPE_HELP" >&2
+	fi
+	exit 1
+}
+
 # Helper libraries
 . ${0%/*}/lib-ci-type.sh
 
 # Parse options
+mode_all=
 mode_build=
 mode_test=
 mode_debug=
 while test $# != 0
 do
 	case "$1" in
+	--all)
+		echo "MODE: $1" >&2
+		mode_all=t
+		mode_build=t
+		mode_test=t
+		;;
 	--build)
 		echo "MODE: $1" >&2
 		mode_build=t
@@ -25,11 +64,11 @@ do
 		;;
 	-*)
 		echo "error: invalid option: $1" >&2
-		exit 1
+		usage
 		;;
 	*)
 		echo "error: invalid argument: $1" >&2
-		exit 1
+		usage
 		;;
 	esac
 	shift
@@ -39,13 +78,13 @@ done
 if test -z "$jobname"
 then
 	echo "error: must set a CI jobname in the environment" >&2
-	exit 1
+	usage
 fi
 
-if test "$mode_test$mode_build" != "t"
+if test "$mode_test$mode_build" != "t" && test -z "$mode_all"
 then
 	echo "error: need one mode, e.g. --build or --test" >&2
-	exit 1
+	usage
 fi
 
 # Show our configuration
@@ -65,13 +104,13 @@ setenv () {
 	do
 		case "$1" in
 		--build)
-			if test -z "$mode_build"
+			if test -z "$mode_build$mode_all"
 			then
 				skip=t
 			fi
 			;;
 		--test)
-			if test -z "$mode_test"
+			if test -z "$mode_test$mode_all"
 			then
 				skip=t
 			fi
@@ -103,6 +142,10 @@ setenv () {
 	if test -n "$GITHUB_ENV"
 	then
 		echo "$key=$val" >>"$GITHUB_ENV"
+	elif test -z "$CI_TYPE"
+	then
+		echo "$key=\"$val\""
+		echo "export $key"
 	fi
 
 	echo "SET: '$key=$val'" >&2
@@ -113,10 +156,25 @@ CC=
 
 # How many jobs to run in parallel?
 NPROC=10
+case "$CI_TYPE" in
+'')
+	. ${0%/*}/lib-online_cpus.sh
+	NPROC=$(online_cpus)
+
+	if test -n "$MAKEFLAGS"
+	then
+		COMMON_MAKEFLAGS="$MAKEFLAGS"
+	else
+		COMMON_MAKEFLAGS=--jobs=$NPROC
+	fi
+	;;
+*)
+	# For "--test" we carry the MAKEFLAGS over from earlier steps, except
+	# in stand-alone jobs which will use $COMMON_MAKEFLAGS.
+	COMMON_MAKEFLAGS=--jobs=$NPROC
+	;;
+esac
 
-# For "--test" we carry the MAKEFLAGS over from earlier steps, except
-# in stand-alone jobs which will use $COMMON_MAKEFLAGS.
-COMMON_MAKEFLAGS=--jobs=$NPROC
 
 # Clear MAKEFLAGS that may come from the outside world.
 MAKEFLAGS=$COMMON_MAKEFLAGS
@@ -136,6 +194,8 @@ github-actions)
 	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 	setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
 	;;
+'')
+	;;
 *)
 	echo "Unhandled CI type: $CI_TYPE" >&2
 	exit 1
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 29f8c332eca..ec395c79ccd 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -36,10 +36,6 @@ do
 			cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
 			tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
 			;;
-		*)
-			echo "Unhandled CI type: $CI_TYPE" >&2
-			exit 1
-			;;
 		esac
 	fi
 done
-- 
2.36.1.1045.gf356b5617dd