git@vger.kernel.org mailing list mirror (one of many)
 help / Atom feed
* [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services
@ 2018-09-03 21:10 Johannes Schindelin via GitGitGadget
  2018-09-03 21:10 ` [PATCH 1/9] ci: rename the library of common functions Johannes Schindelin via GitGitGadget
                   ` (11 more replies)
  0 siblings, 12 replies; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-09-03 21:10 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano

For a long time already, we have Git's source code continuously tested via
Travis CI, see e.g. https://travis-ci.org/git/git/builds/421738884. It has
served us well, and more and more developers actually pay attention and
benefit from the testing this gives us.

It is also an invaluable tool for contributors who can validate their code
contributions via PRs on GitHub, e.g. to verify that their tests do actually
run on macOS (i.e. with the BSD family of Unix tools instead of the GNU
one).

The one sad part about this is the Windows support. Travis lacks it, and we
work around that by using Visual Studio Team Services (VSTS) indirectly: one
phase in Travis would trigger a build, wait for its log, and then paste that
log.

As Git's Windows builds (and tests!) take quite a bit of time, Travis often
timed out, or somehow the trigger did not work, and for security reasons
(the Windows builds are performed in a private pool of containers), the
Windows builds are completely disabled for Pull Requests on GitHub.

One might ask why we did not use Visual Studio Team Services directly. There
were a couple of reasons for that:

 * most notably, VSTS's build logs could not be viewed anonymously,
 * while VSTS had Linux and Windows agents, it lacked macOS agents,
 * etc

The main two reasons no longer apply: macOS agents are available now
[https://docs.microsoft.com/en-us/vsts/release-notes/2018/jul-10-vsts], and
there is a limited preview of "public projects"
[https://blogs.msdn.microsoft.com/devops/2018/04/27/vsts-public-projects-limited-preview/]
, i.e. it is possible to configure a VSTS project so that anybody can view
the logs.

I had secured such a public project for Git for Windows already, and I
recently also got one for Git. For now, the latter is hooked up with my
personal git.git fork on GitHub, but it is my hope that I convince y'all
that these VSTS builds are a good idea, and then hook it up with 
https://github.com/git/git.

As a special treat, this patch series adds the ability to present the
outcome of Git's test suite as JUnit-style .xml files. This allows the VSTS
build to present fun diagrams, trends, and makes it a lot easier to drill
down to test failures than before. See for example 
https://git.visualstudio.com/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details
[https://git.visualstudio.com/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details] 
(you can click on the label of the failed test, and then see the detailed
output in the right pane).

This patch series took way more time than I had originally planned, but I
think that in particular the advanced display of the test results was worth
it. Please let me know what you think about this.

Johannes Schindelin (9):
  ci: rename the library of common functions
  ci/lib.sh: encapsulate Travis-specific things
  test-date: add a subcommand to measure times in shell scripts
  tests: optionally write results as JUnit-style .xml
  ci/lib.sh: add support for VSTS CI
  Add a build definition for VSTS
  tests: include detailed trace logs with --write-junit-xml upon failure
  tests: record more stderr with --write-junit-xml in case of failure
  README: add a build badge (status of the VSTS build)

 .vsts-ci.yml                   | 296 +++++++++++++++++++++++++++++++++
 README.md                      |   2 +
 ci/install-dependencies.sh     |   5 +-
 ci/{lib-travisci.sh => lib.sh} |  67 ++++++--
 ci/mount-fileshare.sh          |  26 +++
 ci/print-test-failures.sh      |   4 +-
 ci/run-build-and-tests.sh      |   2 +-
 ci/run-linux32-docker.sh       |   2 +-
 ci/run-static-analysis.sh      |   2 +-
 ci/run-windows-build.sh        |   2 +-
 ci/test-documentation.sh       |   3 +-
 t/.gitignore                   |   1 +
 t/helper/test-date.c           |  12 ++
 t/test-lib.sh                  | 142 ++++++++++++++++
 14 files changed, 544 insertions(+), 22 deletions(-)
 create mode 100644 .vsts-ci.yml
 rename ci/{lib-travisci.sh => lib.sh} (61%)
 create mode 100755 ci/mount-fileshare.sh


base-commit: 2f743933341f276111103550fbf383a34dfcfd38
Published-As: https://github.com/gitgitgadget/git/releases/tags/pr-31%2Fdscho%2Fvsts-ci-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-31/dscho/vsts-ci-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/31
-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH 1/9] ci: rename the library of common functions
  2018-09-03 21:10 [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Johannes Schindelin via GitGitGadget
@ 2018-09-03 21:10 ` Johannes Schindelin via GitGitGadget
  2018-09-03 21:10 ` [PATCH 2/9] ci/lib.sh: encapsulate Travis-specific things Johannes Schindelin via GitGitGadget
                   ` (10 subsequent siblings)
  11 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-09-03 21:10 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The name is hard-coded to reflect that we use Travis CI for continuous
testing.

In the next commits, we will extend this to be able use Visual Studio
Team Services, too.

So let's adjust the name to make it more generic.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 ci/install-dependencies.sh     | 2 +-
 ci/{lib-travisci.sh => lib.sh} | 0
 ci/print-test-failures.sh      | 2 +-
 ci/run-build-and-tests.sh      | 2 +-
 ci/run-linux32-docker.sh       | 2 +-
 ci/run-static-analysis.sh      | 2 +-
 ci/run-windows-build.sh        | 2 +-
 ci/test-documentation.sh       | 2 +-
 8 files changed, 7 insertions(+), 7 deletions(-)
 rename ci/{lib-travisci.sh => lib.sh} (100%)

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 75a9fd2475..961064658e 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -3,7 +3,7 @@
 # Install dependencies required to build and test Git on Linux and macOS
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
 LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
diff --git a/ci/lib-travisci.sh b/ci/lib.sh
similarity index 100%
rename from ci/lib-travisci.sh
rename to ci/lib.sh
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index d55460a212..7aef39a2fd 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -3,7 +3,7 @@
 # Print output of failing tests
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 # Tracing executed commands would produce too much noise in the loop below.
 set +x
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 2a5bff4a1c..e28ac2fb9a 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -3,7 +3,7 @@
 # Build and test Git
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 ln -s "$cache_dir/.prove" t/.prove
 
diff --git a/ci/run-linux32-docker.sh b/ci/run-linux32-docker.sh
index 21637903ce..751acfcf8a 100755
--- a/ci/run-linux32-docker.sh
+++ b/ci/run-linux32-docker.sh
@@ -3,7 +3,7 @@
 # Download and run Docker image to build and test 32-bit Git
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 docker pull daald/ubuntu32:xenial
 
diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh
index 5688f261d0..dc189c7456 100755
--- a/ci/run-static-analysis.sh
+++ b/ci/run-static-analysis.sh
@@ -3,7 +3,7 @@
 # Perform various static code analysis checks
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 make --jobs=2 coccicheck
 
diff --git a/ci/run-windows-build.sh b/ci/run-windows-build.sh
index d99a180e52..a73a4eca0a 100755
--- a/ci/run-windows-build.sh
+++ b/ci/run-windows-build.sh
@@ -6,7 +6,7 @@
 # supported) and a commit hash.
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 test $# -ne 2 && echo "Unexpected number of parameters" && exit 1
 test -z "$GFW_CI_TOKEN" && echo "GFW_CI_TOKEN not defined" && exit
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index a20de9ca12..d3cdbac73f 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -3,7 +3,7 @@
 # Perform sanity checks on documentation and build it.
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 gem install asciidoctor
 
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH 2/9] ci/lib.sh: encapsulate Travis-specific things
  2018-09-03 21:10 [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Johannes Schindelin via GitGitGadget
  2018-09-03 21:10 ` [PATCH 1/9] ci: rename the library of common functions Johannes Schindelin via GitGitGadget
@ 2018-09-03 21:10 ` Johannes Schindelin via GitGitGadget
  2018-09-03 23:43   ` Eric Sunshine
  2018-09-05 18:57   ` Sebastian Schuberth
  2018-09-03 21:10 ` [PATCH 3/9] test-date: add a subcommand to measure times in shell scripts Johannes Schindelin via GitGitGadget
                   ` (9 subsequent siblings)
  11 siblings, 2 replies; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-09-03 21:10 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The upcoming patches will allow building git.git via VSTS CI, where
variable names and URLs look a bit different than in Travis CI.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 ci/install-dependencies.sh |  3 ++-
 ci/lib.sh                  | 44 +++++++++++++++++++++++++++-----------
 ci/print-test-failures.sh  |  2 +-
 ci/test-documentation.sh   |  1 +
 4 files changed, 35 insertions(+), 15 deletions(-)

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 961064658e..6d92cc1cba 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -28,7 +28,8 @@ osx-clang|osx-gcc)
 	brew update --quiet
 	# Uncomment this if you want to run perf tests:
 	# brew install gnu-time
-	brew install git-lfs gettext
+	test -z "$BREW_INSTALL_PACKAGES" ||
+	eval brew install $BREW_INSTALL_PACKAGES
 	brew link --force gettext
 	brew install caskroom/cask/perforce
 	;;
diff --git a/ci/lib.sh b/ci/lib.sh
index 06970f7213..657ff88672 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,5 +1,26 @@
 # Library of functions shared by all CI scripts
 
+if test -n "$TRAVIS_COMMIT"
+then
+	# We are running within Travis CI
+	CI_BRANCH="$TRAVIS_BRANCH"
+	CI_COMMIT="$TRAVIS_COMMIT"
+	CI_JOB_ID="$TRAVIS_JOB_ID"
+	CI_JOB_NUMBER="$TRAVIS_JOB_NUMBER"
+	CI_OS_NAME="$TRAVIS_OS_NAME"
+	CI_REPO_SLUG="$TRAVIS_REPO_SLUG"
+
+	cache_dir="$HOME/travis-cache"
+
+	url_for_job_id () {
+		echo "https://travis-ci.org/$CI_REPO_SLUG/jobs/$1"
+	}
+
+	BREW_INSTALL_PACKAGES="git-lfs gettext"
+	export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
+	export GIT_TEST_OPTS="--verbose-log -x --immediate"
+fi
+
 skip_branch_tip_with_tag () {
 	# Sometimes, a branch is pushed at the same time the tag that points
 	# at the same commit as the tip of the branch is pushed, and building
@@ -13,10 +34,10 @@ skip_branch_tip_with_tag () {
 	# we can skip the build because we won't be skipping a build
 	# of a tag.
 
-	if TAG=$(git describe --exact-match "$TRAVIS_BRANCH" 2>/dev/null) &&
-		test "$TAG" != "$TRAVIS_BRANCH"
+	if TAG=$(git describe --exact-match "$CI_BRANCH" 2>/dev/null) &&
+		test "$TAG" != "$CI_BRANCH"
 	then
-		echo "$(tput setaf 2)Tip of $TRAVIS_BRANCH is exactly at $TAG$(tput sgr0)"
+		echo "$(tput setaf 2)Tip of $CI_BRANCH is exactly at $TAG$(tput sgr0)"
 		exit 0
 	fi
 }
@@ -25,7 +46,7 @@ skip_branch_tip_with_tag () {
 # job if we encounter the same tree again and can provide a useful info
 # message.
 save_good_tree () {
-	echo "$(git rev-parse $TRAVIS_COMMIT^{tree}) $TRAVIS_COMMIT $TRAVIS_JOB_NUMBER $TRAVIS_JOB_ID" >>"$good_trees_file"
+	echo "$(git rev-parse $CI_COMMIT^{tree}) $CI_COMMIT $CI_JOB_NUMBER $CI_JOB_ID" >>"$good_trees_file"
 	# limit the file size
 	tail -1000 "$good_trees_file" >"$good_trees_file".tmp
 	mv "$good_trees_file".tmp "$good_trees_file"
@@ -35,7 +56,7 @@ save_good_tree () {
 # successfully before (e.g. because the branch got rebased, changing only
 # the commit messages).
 skip_good_tree () {
-	if ! good_tree_info="$(grep "^$(git rev-parse $TRAVIS_COMMIT^{tree}) " "$good_trees_file")"
+	if ! good_tree_info="$(grep "^$(git rev-parse $CI_COMMIT^{tree}) " "$good_trees_file")"
 	then
 		# Haven't seen this tree yet, or no cached good trees file yet.
 		# Continue the build job.
@@ -45,18 +66,18 @@ skip_good_tree () {
 	echo "$good_tree_info" | {
 		read tree prev_good_commit prev_good_job_number prev_good_job_id
 
-		if test "$TRAVIS_JOB_ID" = "$prev_good_job_id"
+		if test "$CI_JOB_ID" = "$prev_good_job_id"
 		then
 			cat <<-EOF
-			$(tput setaf 2)Skipping build job for commit $TRAVIS_COMMIT.$(tput sgr0)
+			$(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0)
 			This commit has already been built and tested successfully by this build job.
 			To force a re-build delete the branch's cache and then hit 'Restart job'.
 			EOF
 		else
 			cat <<-EOF
-			$(tput setaf 2)Skipping build job for commit $TRAVIS_COMMIT.$(tput sgr0)
+			$(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0)
 			This commit's tree has already been built and tested successfully in build job $prev_good_job_number for commit $prev_good_commit.
-			The log of that build job is available at https://travis-ci.org/$TRAVIS_REPO_SLUG/jobs/$prev_good_job_id
+			The log of that build job is available at $(url_for_job_id $prev_good_job_id)
 			To force a re-build delete the branch's cache and then hit 'Restart job'.
 			EOF
 		fi
@@ -81,7 +102,6 @@ check_unignored_build_artifacts ()
 # and installing dependencies.
 set -ex
 
-cache_dir="$HOME/travis-cache"
 good_trees_file="$cache_dir/good-trees"
 
 mkdir -p "$cache_dir"
@@ -91,13 +111,11 @@ skip_good_tree
 
 if test -z "$jobname"
 then
-	jobname="$TRAVIS_OS_NAME-$CC"
+	jobname="$CI_OS_NAME-$CC"
 fi
 
 export DEVELOPER=1
 export DEFAULT_TEST_TARGET=prove
-export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
-export GIT_TEST_OPTS="--verbose-log -x --immediate"
 export GIT_TEST_CLONE_2GB=YesPlease
 if [ "$jobname" = linux-gcc ]; then
 	export CC=gcc-8
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 7aef39a2fd..d2045b63a6 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -69,7 +69,7 @@ do
 	fi
 done
 
-if [ $combined_trash_size -gt 0 ]
+if [ -n "$TRAVIS_JOB_ID" -a $combined_trash_size -gt 0 ]
 then
 	echo "------------------------------------------------------------------------"
 	echo "Trash directories embedded in this log can be extracted by running:"
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index d3cdbac73f..7d0beb2832 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -5,6 +5,7 @@
 
 . ${0%/*}/lib.sh
 
+test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
 gem install asciidoctor
 
 make check-builtins
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH 3/9] test-date: add a subcommand to measure times in shell scripts
  2018-09-03 21:10 [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Johannes Schindelin via GitGitGadget
  2018-09-03 21:10 ` [PATCH 1/9] ci: rename the library of common functions Johannes Schindelin via GitGitGadget
  2018-09-03 21:10 ` [PATCH 2/9] ci/lib.sh: encapsulate Travis-specific things Johannes Schindelin via GitGitGadget
@ 2018-09-03 21:10 ` Johannes Schindelin via GitGitGadget
  2018-09-03 21:10 ` [PATCH 4/9] tests: optionally write results as JUnit-style .xml Johannes Schindelin via GitGitGadget
                   ` (8 subsequent siblings)
  11 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-09-03 21:10 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

In the next commit, we want to teach Git's test suite to optionally
output test results in JUnit-style .xml files. These files contain
information about the time spent. So we need a way to measure time.

While we could use `date +%s` for that, this will give us only seconds,
i.e. very coarse-grained timings.

GNU `date` supports `date +%s.%N` (i.e. nanosecond-precision output),
but there is no equivalent in BSD `date` (read: on macOS, we would not
be able to obtain precise timings).

So let's introduce `test-tool date getnanos`, with an optional start
time, that outputs preciser values.

Granted, it is a bit pointless to try measuring times accurately in
shell scripts, certainly to nanosecond precision. But it is better than
second-granularity.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/helper/test-date.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/t/helper/test-date.c b/t/helper/test-date.c
index a0837371ab..792a805374 100644
--- a/t/helper/test-date.c
+++ b/t/helper/test-date.c
@@ -7,6 +7,7 @@ static const char *usage_msg = "\n"
 "  test-tool date parse [date]...\n"
 "  test-tool date approxidate [date]...\n"
 "  test-tool date timestamp [date]...\n"
+"  test-tool date getnanos [start-nanos]\n"
 "  test-tool date is64bit\n"
 "  test-tool date time_t-is64bit\n";
 
@@ -82,6 +83,15 @@ static void parse_approx_timestamp(const char **argv, struct timeval *now)
 	}
 }
 
+static void getnanos(const char **argv, struct timeval *now)
+{
+	double seconds = getnanotime() / 1.0e9;
+
+	if (*argv)
+		seconds -= strtod(*argv, NULL);
+	printf("%lf\n", seconds);
+}
+
 int cmd__date(int argc, const char **argv)
 {
 	struct timeval now;
@@ -108,6 +118,8 @@ int cmd__date(int argc, const char **argv)
 		parse_approxidate(argv+1, &now);
 	else if (!strcmp(*argv, "timestamp"))
 		parse_approx_timestamp(argv+1, &now);
+	else if (!strcmp(*argv, "getnanos"))
+		getnanos(argv+1, &now);
 	else if (!strcmp(*argv, "is64bit"))
 		return sizeof(timestamp_t) == 8 ? 0 : 1;
 	else if (!strcmp(*argv, "time_t-is64bit"))
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH 4/9] tests: optionally write results as JUnit-style .xml
  2018-09-03 21:10 [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Johannes Schindelin via GitGitGadget
                   ` (2 preceding siblings ...)
  2018-09-03 21:10 ` [PATCH 3/9] test-date: add a subcommand to measure times in shell scripts Johannes Schindelin via GitGitGadget
@ 2018-09-03 21:10 ` Johannes Schindelin via GitGitGadget
  2018-09-04  0:43   ` Eric Sunshine
  2018-09-03 21:10 ` [PATCH 5/9] ci/lib.sh: add support for VSTS CI Johannes Schindelin via GitGitGadget
                   ` (7 subsequent siblings)
  11 siblings, 1 reply; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-09-03 21:10 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

This will come in handy when publishing the results of Git's test suite
during an automated VSTS CI run.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/.gitignore  |  1 +
 t/test-lib.sh | 98 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 99 insertions(+)

diff --git a/t/.gitignore b/t/.gitignore
index 348715f0e4..91cf5772fe 100644
--- a/t/.gitignore
+++ b/t/.gitignore
@@ -2,3 +2,4 @@
 /test-results
 /.prove
 /chainlinttmp
+/out/
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 8bb0f4348e..50a65a600e 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -288,6 +288,9 @@ do
 	--verbose-log)
 		verbose_log=t
 		shift ;;
+	--write-junit-xml)
+		write_junit_xml=t
+		shift ;;
 	*)
 		echo "error: unknown test option '$1'" >&2; exit 1 ;;
 	esac
@@ -431,11 +434,24 @@ trap 'exit $?' INT
 # the test_expect_* functions instead.
 
 test_ok_ () {
+	if test -n "$write_junit_xml"
+	then
+		write_junit_xml_testcase "$*"
+	fi
 	test_success=$(($test_success + 1))
 	say_color "" "ok $test_count - $@"
 }
 
 test_failure_ () {
+	if test -n "$write_junit_xml"
+	then
+		junit_insert="<failure message=\"not ok $test_count -"
+		junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
+		junit_insert="$junit_insert $(xml_attr_encode \
+			"$(printf '%s\n' "$@" | sed 1d)")"
+		junit_insert="$junit_insert</failure>"
+		write_junit_xml_testcase "$1" "      $junit_insert"
+	fi
 	test_failure=$(($test_failure + 1))
 	say_color error "not ok $test_count - $1"
 	shift
@@ -444,11 +460,19 @@ test_failure_ () {
 }
 
 test_known_broken_ok_ () {
+	if test -n "$write_junit_xml"
+	then
+		write_junit_xml_testcase "$* (breakage fixed)"
+	fi
 	test_fixed=$(($test_fixed+1))
 	say_color error "ok $test_count - $@ # TODO known breakage vanished"
 }
 
 test_known_broken_failure_ () {
+	if test -n "$write_junit_xml"
+	then
+		write_junit_xml_testcase "$* (known breakage)"
+	fi
 	test_broken=$(($test_broken+1))
 	say_color warn "not ok $test_count - $@ # TODO known breakage"
 }
@@ -706,6 +730,10 @@ test_start_ () {
 	test_count=$(($test_count+1))
 	maybe_setup_verbose
 	maybe_setup_valgrind
+	if test -n "$write_junit_xml"
+	then
+		junit_start=$(test-tool date getnanos)
+	fi
 }
 
 test_finish_ () {
@@ -743,6 +771,13 @@ test_skip () {
 
 	case "$to_skip" in
 	t)
+		if test -n "$write_junit_xml"
+		then
+			message="$(xml_attr_encode "$skipped_reason")"
+			write_junit_xml_testcase "$1" \
+				"      <skipped message=\"$message\" />"
+		fi
+
 		say_color skip >&3 "skipping test: $@"
 		say_color skip "ok $test_count # skip $1 ($skipped_reason)"
 		: true
@@ -758,9 +793,58 @@ test_at_end_hook_ () {
 	:
 }
 
+write_junit_xml () {
+	case "$1" in
+	--truncate)
+		>"$junit_xml_path"
+		junit_have_testcase=
+		shift
+		;;
+	esac
+	printf '%s\n' "$@" >>"$junit_xml_path"
+}
+
+xml_attr_encode () {
+	# We do not translate CR to &#x0d; because BSD sed does not handle
+	# \r in the regex. In practice, the output should not even have any
+	# carriage returns.
+	printf '%s\n' "$@" |
+	sed -e 's/&/\&amp;/g' -e "s/'/\&apos;/g" -e 's/"/\&quot;/g' \
+		-e 's/</\&lt;/g' -e 's/>/\&gt;/g' \
+		-e 's/	/\&#x09;/g' -e 's/$/\&#x0a;/' -e '$s/&#x0a;$//' |
+	tr -d '\012\015'
+}
+
+write_junit_xml_testcase () {
+	junit_attrs="name=\"$(xml_attr_encode "$this_test.$test_count $1")\""
+	shift
+	junit_attrs="$junit_attrs classname=\"$this_test\""
+	junit_attrs="$junit_attrs time=\"$(test-tool \
+		date getnanos $junit_start)\""
+	write_junit_xml "$(printf '%s\n' \
+		"    <testcase $junit_attrs>" "$@" "    </testcase>")"
+	junit_have_testcase=t
+}
+
 test_done () {
 	GIT_EXIT_OK=t
 
+	if test -n "$write_junit_xml" && test -n "$junit_xml_path"
+	then
+		test -n "$junit_have_testcase" || {
+			junit_start=$(test-tool date getnanos)
+			write_junit_xml_testcase "all tests skipped"
+		}
+
+		# adjust the overall time
+		junit_time=$(test-tool date getnanos $junit_suite_start)
+		sed "s/<testsuite [^>]*/& time=\"$junit_time\"/" \
+			<"$junit_xml_path" >"$junit_xml_path.new"
+		mv "$junit_xml_path.new" "$junit_xml_path"
+
+		write_junit_xml "  </testsuite>" "</testsuites>"
+	fi
+
 	if test -z "$HARNESS_ACTIVE"
 	then
 		test_results_dir="$TEST_OUTPUT_DIRECTORY/test-results"
@@ -996,6 +1080,7 @@ then
 else
 	mkdir -p "$TRASH_DIRECTORY"
 fi
+
 # Use -P to resolve symlinks in our working directory so that the cwd
 # in subprocesses like git equals our $PWD (for pathname comparisons).
 cd -P "$TRASH_DIRECTORY" || exit 1
@@ -1009,6 +1094,19 @@ then
 	test_done
 fi
 
+if test -n "$write_junit_xml"
+then
+	junit_xml_dir="$TEST_OUTPUT_DIRECTORY/out"
+	mkdir -p "$junit_xml_dir"
+	junit_xml_base=${0##*/}
+	junit_xml_path="$junit_xml_dir/TEST-${junit_xml_base%.sh}.xml"
+	junit_attrs="name=\"${junit_xml_base%.sh}\""
+	junit_attrs="$junit_attrs timestamp=\"$(TZ=UTC \
+		date +%Y-%m-%dT%H:%M:%S)\""
+	write_junit_xml --truncate "<testsuites>" "  <testsuite $junit_attrs>"
+	junit_suite_start=$(test-tool date getnanos)
+fi
+
 # Provide an implementation of the 'yes' utility
 yes () {
 	if test $# = 0
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH 5/9] ci/lib.sh: add support for VSTS CI
  2018-09-03 21:10 [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Johannes Schindelin via GitGitGadget
                   ` (3 preceding siblings ...)
  2018-09-03 21:10 ` [PATCH 4/9] tests: optionally write results as JUnit-style .xml Johannes Schindelin via GitGitGadget
@ 2018-09-03 21:10 ` Johannes Schindelin via GitGitGadget
  2018-09-03 21:10 ` [PATCH 6/9] Add a build definition for VSTS Johannes Schindelin via GitGitGadget
                   ` (6 subsequent siblings)
  11 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-09-03 21:10 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

This patch introduces a conditional arm that defines some environment
variables and a function that displays the URL given the job id (to
identify previous runs for known-good trees).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 ci/lib.sh | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/ci/lib.sh b/ci/lib.sh
index 657ff88672..f2f6d70c72 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -19,6 +19,29 @@ then
 	BREW_INSTALL_PACKAGES="git-lfs gettext"
 	export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --immediate"
+elif test -n "$SYSTEM_TASKDEFINITIONSURI"
+then
+	# We are running in VSTS CI
+	CI_BRANCH="$BUILD_SOURCEBRANCH"
+	CI_COMMIT="$BUILD_SOURCEVERSION"
+	CI_JOB_ID="$BUILD_BUILDID"
+	CI_JOB_NUMBER="$BUILD_BUILDNUMBER"
+	CI_OS_NAME="$(echo "$AGENT_OS" | tr A-Z a-z)"
+	test darwin != "$CI_OS_NAME" || CI_OS_NAME=osx
+	CI_REPO_SLUG="$(expr "$BUILD_REPOSITORY_URI" : '.*/\([^/]*/[^/]*\)$')"
+	CC="${CC:-gcc}"
+
+	# use a subdirectory of the cache dir (because the file share is shared
+	# among *all* phases)
+	cache_dir="$HOME/vsts-cache/$SYSTEM_PHASENAME"
+
+	url_for_job_id () {
+		echo "$SYSTEM_TASKDEFINITIONSURI$SYSTEM_TEAMPROJECT/_build/results?buildId=$1"
+	}
+
+	BREW_INSTALL_PACKAGES=
+	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
+	export GIT_TEST_OPTS="--quiet --write-junit-xml"
 fi
 
 skip_branch_tip_with_tag () {
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH 6/9] Add a build definition for VSTS
  2018-09-03 21:10 [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Johannes Schindelin via GitGitGadget
                   ` (4 preceding siblings ...)
  2018-09-03 21:10 ` [PATCH 5/9] ci/lib.sh: add support for VSTS CI Johannes Schindelin via GitGitGadget
@ 2018-09-03 21:10 ` Johannes Schindelin via GitGitGadget
  2018-09-03 21:10 ` [PATCH 7/9] tests: include detailed trace logs with --write-junit-xml upon failure Johannes Schindelin via GitGitGadget
                   ` (5 subsequent siblings)
  11 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-09-03 21:10 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

This commit adds a .vsts-ci.yml which is Visual Studio Team Services'
equivalent to Travis CI's .travis.yml.

To make things a bit easier to understand, we refrain from using the
`matrix` feature here because (while it is powerful) it can be a bit
confusing to users who are not familiar with CI setups. Therefore, we
use a separate phase even for similar configurations (such as GCC vs
Clang on Linux, GCC vs Clang on macOS).

Also, we make use of the shiny new feature we just introduced where the
test suite can output JUnit-style .xml files. This information is made
available in a nice UI that allows the viewer to filter by phase and/or
test number, and to see trends such as: number of (failing) tests, time
spent running the test suite, etc.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 .vsts-ci.yml          | 296 ++++++++++++++++++++++++++++++++++++++++++
 ci/mount-fileshare.sh |  26 ++++
 2 files changed, 322 insertions(+)
 create mode 100644 .vsts-ci.yml
 create mode 100755 ci/mount-fileshare.sh

diff --git a/.vsts-ci.yml b/.vsts-ci.yml
new file mode 100644
index 0000000000..ff1d952df9
--- /dev/null
+++ b/.vsts-ci.yml
@@ -0,0 +1,296 @@
+resources:
+- repo: self
+  fetchDepth: 1
+
+phases:
+- phase: linux_clang
+  displayName: linux-clang
+  condition: succeeded()
+  queue:
+    name: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
+
+       sudo apt-get update
+       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin
+
+       export CC=clang
+
+       ci/install-dependencies.sh
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       sudo umount "$HOME/vsts-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'linux-clang'
+      platform: Linux
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+
+- phase: linux_gcc
+  displayName: linux-gcc
+  condition: succeeded()
+  queue:
+    name: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
+
+       sudo apt-get update
+       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin
+
+       ci/install-dependencies.sh
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       sudo umount "$HOME/vsts-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'linux-gcc'
+      platform: Linux
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+
+- phase: osx_clang
+  displayName: osx-clang
+  condition: succeeded()
+  queue:
+    name: Hosted macOS
+  steps:
+  - bash: |
+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
+
+       export CC=clang
+
+       ci/install-dependencies.sh
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       umount "$HOME/vsts-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'osx-clang'
+      platform: macOS
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+
+- phase: osx_gcc
+  displayName: osx-gcc
+  condition: succeeded()
+  queue:
+    name: Hosted macOS
+  steps:
+  - bash: |
+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
+
+       ci/install-dependencies.sh
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       umount "$HOME/vsts-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'osx-gcc'
+      platform: macOS
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+
+- phase: gettext_poison
+  displayName: GETTEXT_POISON
+  condition: succeeded()
+  queue:
+    name: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
+
+       sudo apt-get update
+       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev
+
+       export jobname=GETTEXT_POISON
+
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       sudo umount "$HOME/vsts-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'gettext-poison'
+      platform: Linux
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+
+- phase: windows
+  displayName: Windows
+  condition: succeeded()
+  queue:
+    name: Hosted VS2017
+    timeoutInMinutes: 240
+  steps:
+  - powershell: |
+       # Helper to check the error level of the latest command (exit with error when appropriate)
+       function c() { if (!$?) { exit(1) } }
+
+       net use s: \\gitfileshare.file.core.windows.net\vsts-cache "$(gitfileshare.pwd)" /user:AZURE\gitfileshare /persistent:no; c
+       cmd /c mklink /d "$(Build.SourcesDirectory)\vsts-cache" S:\; c
+
+       # Add build agent's MinGit to PATH
+       $env:PATH = $env:AGENT_HOMEDIRECTORY +"\externals\\git\cmd;" +$env:PATH
+
+       # Helper to initialize (or update) a Git worktree
+       function init ($path, $url, $set_origin) {
+           if (Test-Path $path) {
+               cd $path; c
+               if (Test-Path .git) {
+                   git init; c
+               } else {
+                   git status
+               }
+           } else {
+               git init $path; c
+               cd $path; c
+           }
+           git config core.autocrlf false; c
+           git config core.untrackedCache true; c
+           if (($set_origin -ne 0) -and !(git config remote.origin.url)) {
+               git remote add origin $url; c
+           }
+           git fetch --depth=1 $url master; c
+           git reset --hard FETCH_HEAD; c
+           git clean -df; c
+       }
+
+       # Initialize Git for Windows' SDK
+       $sdk_path = "$(Build.SourcesDirectory)\git-sdk-64"
+       init "$sdk_path" "https://git-for-windows.visualstudio.com/_git/git-sdk-64" 0
+       init usr\src\build-extra https://github.com/git-for-windows/build-extra 1
+
+       cd "$(Build.SourcesDirectory)"; c
+
+       $env:HOME = "$(Build.SourcesDirectory)"
+       $env:MSYSTEM = "MINGW64"
+       git-sdk-64\git-cmd --command=usr\\bin\\bash.exe -lc @"
+         . ci/lib.sh
+
+         make -j10 DEVELOPER=1 NO_PERL=1 || exit 1
+         NO_PERL=1 NO_SVN_TESTS=1 GIT_TEST_OPTS=\"--quiet --write-junit-xml\" time make -j15 -k DEVELOPER=1 test || {
+           NO_PERL=1 NO_SVN_TESTS=1 GIT_TEST_OPTS=\"-i -v -x\" make -k failed; exit 1
+         }
+
+         save_good_tree
+       "@
+
+       cmd /c rmdir "$(Build.SourcesDirectory)\vsts-cache"
+    displayName: 'build & test'
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'windows'
+      platform: Windows
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+
+- phase: linux32
+  displayName: Linux32
+  condition: succeeded()
+  queue:
+    name: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
+
+       sudo apt-get update
+       sudo apt-get -y install \
+           apt-transport-https \
+           ca-certificates \
+           curl \
+           software-properties-common
+       curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
+       sudo add-apt-repository \
+          "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
+          $(lsb_release -cs) \
+          stable"
+       sudo apt-get update
+       sudo apt-get -y install docker-ce
+
+       sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS=-j3 bash -lxc ci/run-linux32-docker.sh || exit 1
+
+       sudo chmod a+r t/out/TEST-*.xml
+
+       sudo umount "$HOME/vsts-cache" || exit 1
+    displayName: 'ci/run-linux32-docker.sh'
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'linux32'
+      platform: Linux
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+
+- phase: static_analysis
+  displayName: StaticAnalysis
+  condition: succeeded()
+  queue:
+    name: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
+
+       sudo apt-get update
+       sudo apt-get install -y coccinelle
+
+       export jobname=StaticAnalysis
+
+       ci/run-static-analysis.sh || exit 1
+
+       sudo umount "$HOME/vsts-cache" || exit 1
+    displayName: 'ci/run-static-analysis.sh'
+
+- phase: documentation
+  displayName: Documentation
+  condition: succeeded()
+  queue:
+    name: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
+
+       sudo apt-get update
+       sudo apt-get install -y asciidoc xmlto asciidoctor
+
+       export ALREADY_HAVE_ASCIIDOCTOR=yes.
+       export jobname=Documentation
+
+       ci/test-documentation.sh || exit 1
+
+       sudo umount "$HOME/vsts-cache" || exit 1
+    displayName: 'ci/test-documentation.sh'
diff --git a/ci/mount-fileshare.sh b/ci/mount-fileshare.sh
new file mode 100755
index 0000000000..5fb5f74b70
--- /dev/null
+++ b/ci/mount-fileshare.sh
@@ -0,0 +1,26 @@
+#!/bin/sh
+
+die () {
+	echo "$*" >&2
+	exit 1
+}
+
+test $# = 4 ||
+die "Usage: $0 <share> <username> <password> <mountpoint"
+
+mkdir -p "$4" || die "Could not create $4"
+
+case "$(uname -s)" in
+Linux)
+	sudo mount -t cifs -o vers=3.0,username="$2",password="$3",dir_mode=0777,file_mode=0777,serverino "$1" "$4"
+	;;
+Darwin)
+	pass="$(echo "$3" | sed -e 's/\//%2F/g' -e 's/+/%2B/g')" &&
+	mount -t smbfs,soft "smb://$2:$pass@${1#//}" "$4"
+	;;
+*)
+	die "No support for $(uname -s)"
+	;;
+esac ||
+die "Could not mount $4"
+
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH 7/9] tests: include detailed trace logs with --write-junit-xml upon failure
  2018-09-03 21:10 [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Johannes Schindelin via GitGitGadget
                   ` (5 preceding siblings ...)
  2018-09-03 21:10 ` [PATCH 6/9] Add a build definition for VSTS Johannes Schindelin via GitGitGadget
@ 2018-09-03 21:10 ` Johannes Schindelin via GitGitGadget
  2018-09-04  4:30   ` Eric Sunshine
  2018-09-03 21:10 ` [PATCH 8/9] tests: record more stderr with --write-junit-xml in case of failure Johannes Schindelin via GitGitGadget
                   ` (4 subsequent siblings)
  11 siblings, 1 reply; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-09-03 21:10 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The JUnit XML format lends itself to be presented in a powerful UI,
where you can drill down to the information you are interested in very
quickly.

For test failures, this usually means that you want to see the detailed
trace of the failing tests.

With Travis CI, we passed the `--verbose-log` option to get those
traces. However, that seems excessive, as we do not need/use the logs in
almost all of those cases: only when a test fails do we have a way to
include the trace.

So let's do something different in VSTS: let's run all the tests with
`--quiet` first, and only if a failure is encountered, try to trace the
commands as they are executed.

Of course, we cannot turn on `--verbose-log` after the fact. So let's
just re-run the test with all the same options, adding `--verbose-log`.
And then munging the output file into the JUnit XML on the fly.

Note: there is an off chance that re-running the test in verbose mode
"fixes" the failures (and this does happen from time to time!). That is
a possibility we should be able to live with. Ideally, we would label
this as "Passed upon rerun", and that outcome even exists on VSTS, but
it is not available when using the JUnit XML format for now:
https://github.com/Microsoft/vsts-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.cs

This patch contains a slightly inelegant workaround for the p4 and
git-daemon tests: when we try to re-run the p4/git-daemon tests after
the daemon has been started already, we need to kill said daemon. We do
this by detecting the presence of the `kill_p4d` and `stop_git_daemon`
shell functions and calling them if available.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/test-lib.sh | 40 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 39 insertions(+), 1 deletion(-)

diff --git a/t/test-lib.sh b/t/test-lib.sh
index 50a65a600e..ea4ed250cc 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -85,6 +85,13 @@ done,*)
 	test "$(cat "$BASE.exit")" = 0
 	exit
 	;;
+*' --write-junit-xml '*)
+	# record how to call this script *with* --verbose-log, in case
+	# we encounter a breakage
+	junit_rerun_options_sq="$(printf '%s\n' "$0" --verbose-log -x "$@" |
+		sed -e "s/'/'\\\\''/g" -e "s/^/'/" -e "s/\$/'/" |
+		tr '\012' ' ')"
+	;;
 esac
 
 # For repeatability, reset the environment to known value.
@@ -445,10 +452,37 @@ test_ok_ () {
 test_failure_ () {
 	if test -n "$write_junit_xml"
 	then
+		if test -z "$GIT_TEST_TEE_OUTPUT_FILE"
+		then
+			case "$(type kill_p4d 2>/dev/null | head -n 1)" in
+			*function*) kill_p4d;;
+			esac
+
+			case "$(type stop_git_daemon 2>/dev/null |
+				head -n 1)" in
+			*function*) stop_git_daemon;;
+			esac
+
+			# re-run with --verbose-log
+			echo "# Re-running: $junit_rerun_options_sq" >&2
+
+			cd "$TEST_DIRECTORY" &&
+			eval "${TEST_SHELL_PATH}" "$junit_rerun_options_sq" \
+				>/dev/null 2>&1
+			status=$?
+
+			say_color "" "$(test 0 = $status ||
+				echo "not ")ok $test_count - (re-ran with trace)"
+			say "1..$test_count"
+			GIT_EXIT_OK=t
+			exit $status
+		fi
+
 		junit_insert="<failure message=\"not ok $test_count -"
 		junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
 		junit_insert="$junit_insert $(xml_attr_encode \
-			"$(printf '%s\n' "$@" | sed 1d)")"
+			"$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")"
+		>"$GIT_TEST_TEE_OUTPUT_FILE"
 		junit_insert="$junit_insert</failure>"
 		write_junit_xml_testcase "$1" "      $junit_insert"
 	fi
@@ -733,6 +767,10 @@ test_start_ () {
 	if test -n "$write_junit_xml"
 	then
 		junit_start=$(test-tool date getnanos)
+
+		# truncate output
+		test -z "$GIT_TEST_TEE_OUTPUT_FILE" ||
+		>"$GIT_TEST_TEE_OUTPUT_FILE"
 	fi
 }
 
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH 8/9] tests: record more stderr with --write-junit-xml in case of failure
  2018-09-03 21:10 [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Johannes Schindelin via GitGitGadget
                   ` (6 preceding siblings ...)
  2018-09-03 21:10 ` [PATCH 7/9] tests: include detailed trace logs with --write-junit-xml upon failure Johannes Schindelin via GitGitGadget
@ 2018-09-03 21:10 ` Johannes Schindelin via GitGitGadget
  2018-09-03 21:10 ` [PATCH 9/9] README: add a build badge (status of the VSTS build) Johannes Schindelin via GitGitGadget
                   ` (3 subsequent siblings)
  11 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-09-03 21:10 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

Sometimes, failures in a test case are actually caused by issues in
earlier test cases.

To make it easier to see those issues, let's attach the output from
before the failing test case (i.e. stdout/stderr since the previous
failing test case, or the start of the test script). This will be
visible in the "Attachments" of the details of the failed test.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/test-lib.sh | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/t/test-lib.sh b/t/test-lib.sh
index ea4ed250cc..6fc03d5a3b 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -484,6 +484,9 @@ test_failure_ () {
 			"$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")"
 		>"$GIT_TEST_TEE_OUTPUT_FILE"
 		junit_insert="$junit_insert</failure>"
+		junit_insert="$junit_insert<system-err>$(xml_attr_encode \
+			"$(cat "$GIT_TEST_TEE_OUTPUT_FILE.err")")</system-err>"
+		>"$GIT_TEST_TEE_OUTPUT_FILE.err"
 		write_junit_xml_testcase "$1" "      $junit_insert"
 	fi
 	test_failure=$(($test_failure + 1))
@@ -768,9 +771,12 @@ test_start_ () {
 	then
 		junit_start=$(test-tool date getnanos)
 
-		# truncate output
-		test -z "$GIT_TEST_TEE_OUTPUT_FILE" ||
-		>"$GIT_TEST_TEE_OUTPUT_FILE"
+		# append to future <system-err>; truncate output
+		test -z "$GIT_TEST_TEE_OUTPUT_FILE" || {
+			cat "$GIT_TEST_TEE_OUTPUT_FILE" \
+				>>"$GIT_TEST_TEE_OUTPUT_FILE.err"
+			>"$GIT_TEST_TEE_OUTPUT_FILE"
+		}
 	fi
 }
 
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH 9/9] README: add a build badge (status of the VSTS build)
  2018-09-03 21:10 [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Johannes Schindelin via GitGitGadget
                   ` (7 preceding siblings ...)
  2018-09-03 21:10 ` [PATCH 8/9] tests: record more stderr with --write-junit-xml in case of failure Johannes Schindelin via GitGitGadget
@ 2018-09-03 21:10 ` Johannes Schindelin via GitGitGadget
  2018-09-05 19:01 ` [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Sebastian Schuberth
                   ` (2 subsequent siblings)
  11 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-09-03 21:10 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

Just like so many other OSS projects, we now also have a build badge.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 README.md | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/README.md b/README.md
index f920a42fad..f260e78042 100644
--- a/README.md
+++ b/README.md
@@ -1,3 +1,5 @@
+[![Build Status](https://git.visualstudio.com/git/_apis/build/status/test-git.git)](https://git.visualstudio.com/git/_build/latest?definitionId=2)
+
 Git - fast, scalable, distributed revision control system
 =========================================================
 
-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 2/9] ci/lib.sh: encapsulate Travis-specific things
  2018-09-03 21:10 ` [PATCH 2/9] ci/lib.sh: encapsulate Travis-specific things Johannes Schindelin via GitGitGadget
@ 2018-09-03 23:43   ` Eric Sunshine
  2018-09-04 11:04     ` Johannes Schindelin
  2018-09-05 18:57   ` Sebastian Schuberth
  1 sibling, 1 reply; 88+ messages in thread
From: Eric Sunshine @ 2018-09-03 23:43 UTC (permalink / raw)
  To: gitgitgadget; +Cc: Git List, Junio C Hamano, Johannes Schindelin

On Mon, Sep 3, 2018 at 5:10 PM Johannes Schindelin via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> The upcoming patches will allow building git.git via VSTS CI, where
> variable names and URLs look a bit different than in Travis CI.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
> @@ -28,7 +28,8 @@ osx-clang|osx-gcc)
>         # Uncomment this if you want to run perf tests:
>         # brew install gnu-time
> -       brew install git-lfs gettext
> +       test -z "$BREW_INSTALL_PACKAGES" ||
> +       eval brew install $BREW_INSTALL_PACKAGES

This 'eval' is unnecessary, isn't it?

    brew install $BREW_INSTALL_PACKAGES

should give the same result.

>         brew link --force gettext
>         brew install caskroom/cask/perforce
> diff --git a/ci/lib.sh b/ci/lib.sh
> @@ -1,5 +1,26 @@
> +       BREW_INSTALL_PACKAGES="git-lfs gettext"

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 4/9] tests: optionally write results as JUnit-style .xml
  2018-09-03 21:10 ` [PATCH 4/9] tests: optionally write results as JUnit-style .xml Johannes Schindelin via GitGitGadget
@ 2018-09-04  0:43   ` Eric Sunshine
  2018-09-04 10:59     ` Johannes Schindelin
  0 siblings, 1 reply; 88+ messages in thread
From: Eric Sunshine @ 2018-09-04  0:43 UTC (permalink / raw)
  To: gitgitgadget; +Cc: Git List, Junio C Hamano, Johannes Schindelin

On Mon, Sep 3, 2018 at 5:10 PM Johannes Schindelin via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> This will come in handy when publishing the results of Git's test suite
> during an automated VSTS CI run.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> @@ -431,11 +434,24 @@ trap 'exit $?' INT
>  test_failure_ () {
> +       if test -n "$write_junit_xml"
> +       then
> +               junit_insert="<failure message=\"not ok $test_count -"
> +               junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
> +               junit_insert="$junit_insert $(xml_attr_encode \
> +                       "$(printf '%s\n' "$@" | sed 1d)")"
> +               junit_insert="$junit_insert</failure>"

This is a genuine failure, so you're creating a <failure> node. Okay.

> +               write_junit_xml_testcase "$1" "      $junit_insert"
> +       fi
> @@ -444,11 +460,19 @@ test_failure_ () {
>  test_known_broken_ok_ () {
> +       if test -n "$write_junit_xml"
> +       then
> +               write_junit_xml_testcase "$* (breakage fixed)"
> +       fi
>         test_fixed=$(($test_fixed+1))
>         say_color error "ok $test_count - $@ # TODO known breakage vanished"
>  }

This was expected to fail but didn't, which means it probably needs
some sort of attention. test_known_broken_ok_() prints this result in
the 'error' color, and test_done() re-inforces that by printing a
message, also in 'error' color:

    42 known breakage(s) vanished; please update test(s)

So, should this emit a <failure> node also, perhaps with 'type'
attribute set to "warning" or something? (<failure type="WARNING"
message="...">)

> @@ -758,9 +793,58 @@ test_at_end_hook_ () {
> +xml_attr_encode () {
> +       # We do not translate CR to &#x0d; because BSD sed does not handle
> +       # \r in the regex. In practice, the output should not even have any
> +       # carriage returns.
> +       printf '%s\n' "$@" |
> +       sed -e 's/&/\&amp;/g' -e "s/'/\&apos;/g" -e 's/"/\&quot;/g' \
> +               -e 's/</\&lt;/g' -e 's/>/\&gt;/g' \
> +               -e 's/  /\&#x09;/g' -e 's/$/\&#x0a;/' -e '$s/&#x0a;$//' |
> +       tr -d '\012\015'
> +}

It's possible to insert a literal CR in the 'sed' expression, which
does match correctly on BSD (and MacOS). For instance:

    CR=$(printf "\r")
    sed -e "s/$CR/\&#x0d;/g"

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 7/9] tests: include detailed trace logs with --write-junit-xml upon failure
  2018-09-03 21:10 ` [PATCH 7/9] tests: include detailed trace logs with --write-junit-xml upon failure Johannes Schindelin via GitGitGadget
@ 2018-09-04  4:30   ` Eric Sunshine
  2018-09-04 11:09     ` Johannes Schindelin
  0 siblings, 1 reply; 88+ messages in thread
From: Eric Sunshine @ 2018-09-04  4:30 UTC (permalink / raw)
  To: gitgitgadget; +Cc: Git List, Junio C Hamano, Johannes Schindelin

On Mon, Sep 3, 2018 at 5:10 PM Johannes Schindelin via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> So let's do something different in VSTS: let's run all the tests with
> `--quiet` first, and only if a failure is encountered, try to trace the
> commands as they are executed. [...]
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> @@ -445,10 +452,37 @@ test_ok_ () {
>  test_failure_ () {
>         if test -n "$write_junit_xml"
>         then
> +               if test -z "$GIT_TEST_TEE_OUTPUT_FILE"
> +               then
> +                       case "$(type kill_p4d 2>/dev/null | head -n 1)" in
> +                       *function*) kill_p4d;;
> +                       esac
> +
> +                       case "$(type stop_git_daemon 2>/dev/null |
> +                               head -n 1)" in
> +                       *function*) stop_git_daemon;;
> +                       esac

In the long run, it might make more sense, and be more scalable, to
have those scripts define a "prepare_for_rerun" variable or function
which this code then runs generically rather than having special
knowledge of those facilities.

I could imagine, for instance, test-lib.sh defining a no-op:

    test_failure_prepare_rerun () {}

and then each of those scripts overriding the function:

    # in lib-git-p4.sh
    test_failure_prepare_rerun () {
        kill_p4d
    }

    # in lib-git-daemon.sh
    test_failure_prepare_rerun () {
        stop_git_daemon
    }

> +                       # re-run with --verbose-log
> +                       echo "# Re-running: $junit_rerun_options_sq" >&2
> +
> +                       cd "$TEST_DIRECTORY" &&
> +                       eval "${TEST_SHELL_PATH}" "$junit_rerun_options_sq" \
> +                               >/dev/null 2>&1
> +                       status=$?
> +
> +                       say_color "" "$(test 0 = $status ||
> +                               echo "not ")ok $test_count - (re-ran with trace)"
> +                       say "1..$test_count"
> +                       GIT_EXIT_OK=t
> +                       exit $status
> +               fi
> +
>                 junit_insert="<failure message=\"not ok $test_count -"
>                 junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
>                 junit_insert="$junit_insert $(xml_attr_encode \
> -                       "$(printf '%s\n' "$@" | sed 1d)")"
> +                       "$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")"
> +               >"$GIT_TEST_TEE_OUTPUT_FILE"
>                 junit_insert="$junit_insert</failure>"
>                 write_junit_xml_testcase "$1" "      $junit_insert"
>         fi

This junit-related stuff is getting pretty lengthy. I wonder if it
would make sense to pull it out to its own function at some point
(again, in the long run).

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 4/9] tests: optionally write results as JUnit-style .xml
  2018-09-04  0:43   ` Eric Sunshine
@ 2018-09-04 10:59     ` Johannes Schindelin
  0 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin @ 2018-09-04 10:59 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: gitgitgadget, Git List, Junio C Hamano

Hi Eric,

On Mon, 3 Sep 2018, Eric Sunshine wrote:

> On Mon, Sep 3, 2018 at 5:10 PM Johannes Schindelin via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
> > This will come in handy when publishing the results of Git's test suite
> > during an automated VSTS CI run.
> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> > diff --git a/t/test-lib.sh b/t/test-lib.sh
> > @@ -431,11 +434,24 @@ trap 'exit $?' INT
> >  test_failure_ () {
> > +       if test -n "$write_junit_xml"
> > +       then
> > +               junit_insert="<failure message=\"not ok $test_count -"
> > +               junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
> > +               junit_insert="$junit_insert $(xml_attr_encode \
> > +                       "$(printf '%s\n' "$@" | sed 1d)")"
> > +               junit_insert="$junit_insert</failure>"
> 
> This is a genuine failure, so you're creating a <failure> node. Okay.
> 
> > +               write_junit_xml_testcase "$1" "      $junit_insert"
> > +       fi
> > @@ -444,11 +460,19 @@ test_failure_ () {
> >  test_known_broken_ok_ () {
> > +       if test -n "$write_junit_xml"
> > +       then
> > +               write_junit_xml_testcase "$* (breakage fixed)"
> > +       fi
> >         test_fixed=$(($test_fixed+1))
> >         say_color error "ok $test_count - $@ # TODO known breakage vanished"
> >  }
> 
> This was expected to fail but didn't, which means it probably needs
> some sort of attention. test_known_broken_ok_() prints this result in
> the 'error' color, and test_done() re-inforces that by printing a
> message, also in 'error' color:
> 
>     42 known breakage(s) vanished; please update test(s)
> 
> So, should this emit a <failure> node also, perhaps with 'type'
> attribute set to "warning" or something? (<failure type="WARNING"
> message="...">)

My primary aim is to display the test results in the web interface, see
e.g.
https://git.visualstudio.com/git/_build/results?buildId=128&view=ms.vss-test-web.test-result-details

The parser for JUnit XML (and in fact, the JUnit XML schema itself) do not
allow for such a warning. If you add a `<failure>`, then the build fails.

And we do not want the build to fail. Historically, I saw quite a couple
of "vanished" breakages depending on the platform where I ran the tests.

> > @@ -758,9 +793,58 @@ test_at_end_hook_ () {
> > +xml_attr_encode () {
> > +       # We do not translate CR to &#x0d; because BSD sed does not handle
> > +       # \r in the regex. In practice, the output should not even have any
> > +       # carriage returns.
> > +       printf '%s\n' "$@" |
> > +       sed -e 's/&/\&amp;/g' -e "s/'/\&apos;/g" -e 's/"/\&quot;/g' \
> > +               -e 's/</\&lt;/g' -e 's/>/\&gt;/g' \
> > +               -e 's/  /\&#x09;/g' -e 's/$/\&#x0a;/' -e '$s/&#x0a;$//' |
> > +       tr -d '\012\015'
> > +}
> 
> It's possible to insert a literal CR in the 'sed' expression, which
> does match correctly on BSD (and MacOS). For instance:
> 
>     CR=$(printf "\r")
>     sed -e "s/$CR/\&#x0d;/g"

Okay. But since we are talking about displaying some chunk of text, I
would rather just delete the CR here anyway.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 2/9] ci/lib.sh: encapsulate Travis-specific things
  2018-09-03 23:43   ` Eric Sunshine
@ 2018-09-04 11:04     ` Johannes Schindelin
  0 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin @ 2018-09-04 11:04 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: gitgitgadget, Git List, Junio C Hamano

Hi Eric,

On Mon, 3 Sep 2018, Eric Sunshine wrote:

> On Mon, Sep 3, 2018 at 5:10 PM Johannes Schindelin via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
> > The upcoming patches will allow building git.git via VSTS CI, where
> > variable names and URLs look a bit different than in Travis CI.
> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> > diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
> > @@ -28,7 +28,8 @@ osx-clang|osx-gcc)
> >         # Uncomment this if you want to run perf tests:
> >         # brew install gnu-time
> > -       brew install git-lfs gettext
> > +       test -z "$BREW_INSTALL_PACKAGES" ||
> > +       eval brew install $BREW_INSTALL_PACKAGES
> 
> This 'eval' is unnecessary, isn't it?
> 
>     brew install $BREW_INSTALL_PACKAGES
> 
> should give the same result.

Oh right! Fixed in https://github.com/gitgitgadget/git/pull/31 (and I also
opened https://github.com/git/git/pull/531 to verify that Travis CI still
works).

Thanks,
Dscho

> >         brew link --force gettext
> >         brew install caskroom/cask/perforce
> > diff --git a/ci/lib.sh b/ci/lib.sh
> > @@ -1,5 +1,26 @@
> > +       BREW_INSTALL_PACKAGES="git-lfs gettext"
> 

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 7/9] tests: include detailed trace logs with --write-junit-xml upon failure
  2018-09-04  4:30   ` Eric Sunshine
@ 2018-09-04 11:09     ` Johannes Schindelin
  2018-09-05  5:32       ` Luke Diamand
  0 siblings, 1 reply; 88+ messages in thread
From: Johannes Schindelin @ 2018-09-04 11:09 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: gitgitgadget, Git List, Junio C Hamano

Hi Eric,

On Tue, 4 Sep 2018, Eric Sunshine wrote:

> On Mon, Sep 3, 2018 at 5:10 PM Johannes Schindelin via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
> > So let's do something different in VSTS: let's run all the tests with
> > `--quiet` first, and only if a failure is encountered, try to trace the
> > commands as they are executed. [...]
> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> > diff --git a/t/test-lib.sh b/t/test-lib.sh
> > @@ -445,10 +452,37 @@ test_ok_ () {
> >  test_failure_ () {
> >         if test -n "$write_junit_xml"
> >         then
> > +               if test -z "$GIT_TEST_TEE_OUTPUT_FILE"
> > +               then
> > +                       case "$(type kill_p4d 2>/dev/null | head -n 1)" in
> > +                       *function*) kill_p4d;;
> > +                       esac
> > +
> > +                       case "$(type stop_git_daemon 2>/dev/null |
> > +                               head -n 1)" in
> > +                       *function*) stop_git_daemon;;
> > +                       esac
> 
> In the long run, it might make more sense, and be more scalable, to
> have those scripts define a "prepare_for_rerun" variable or function
> which this code then runs generically rather than having special
> knowledge of those facilities.
> 
> I could imagine, for instance, test-lib.sh defining a no-op:
> 
>     test_failure_prepare_rerun () {}
> 
> and then each of those scripts overriding the function:
> 
>     # in lib-git-p4.sh
>     test_failure_prepare_rerun () {
>         kill_p4d
>     }
> 
>     # in lib-git-daemon.sh
>     test_failure_prepare_rerun () {
>         stop_git_daemon
>     }

Or we could implement `test_atexit` (similar to `test_when_finished`, but
to be executed at `test_done` time). I guess that's what the p4 and daemon
tests really needed to begin with (and probably also the apache2-using
tests).

> 
> > +                       # re-run with --verbose-log
> > +                       echo "# Re-running: $junit_rerun_options_sq" >&2
> > +
> > +                       cd "$TEST_DIRECTORY" &&
> > +                       eval "${TEST_SHELL_PATH}" "$junit_rerun_options_sq" \
> > +                               >/dev/null 2>&1
> > +                       status=$?
> > +
> > +                       say_color "" "$(test 0 = $status ||
> > +                               echo "not ")ok $test_count - (re-ran with trace)"
> > +                       say "1..$test_count"
> > +                       GIT_EXIT_OK=t
> > +                       exit $status
> > +               fi
> > +
> >                 junit_insert="<failure message=\"not ok $test_count -"
> >                 junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
> >                 junit_insert="$junit_insert $(xml_attr_encode \
> > -                       "$(printf '%s\n' "$@" | sed 1d)")"
> > +                       "$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")"
> > +               >"$GIT_TEST_TEE_OUTPUT_FILE"
> >                 junit_insert="$junit_insert</failure>"
> >                 write_junit_xml_testcase "$1" "      $junit_insert"
> >         fi
> 
> This junit-related stuff is getting pretty lengthy. I wonder if it
> would make sense to pull it out to its own function at some point
> (again, in the long run).

Now that you mention it... I agree. This is getting long.

In the short run, I have two things to consider, though: I want to make
this work first, then think about introducing a layer of abstraction, and
I want to go on vacation tomorrow.

So I agree that this is something to be considered in the long run, i.e.
not right now ;-)

Thanks,
Dscho

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 7/9] tests: include detailed trace logs with --write-junit-xml upon failure
  2018-09-04 11:09     ` Johannes Schindelin
@ 2018-09-05  5:32       ` Luke Diamand
  2018-09-05 12:39         ` Johannes Schindelin
  0 siblings, 1 reply; 88+ messages in thread
From: Luke Diamand @ 2018-09-05  5:32 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Eric Sunshine, gitgitgadget, Git List, Junio C Hamano

On 4 September 2018 at 12:09, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi Eric,
>
> On Tue, 4 Sep 2018, Eric Sunshine wrote:
>
>> On Mon, Sep 3, 2018 at 5:10 PM Johannes Schindelin via GitGitGadget
>> <gitgitgadget@gmail.com> wrote:
>> > So let's do something different in VSTS: let's run all the tests with
>> > `--quiet` first, and only if a failure is encountered, try to trace the
>> > commands as they are executed. [...]

Is this re-running just an individual test on its own or all the tests
within a single file?

The latter shouldn't need this at all. And the former, I'm not sure
will actually work - most of the tests assume some particular p4
state. But perhaps I'm missing something?

I also think it does look kind of ugly. And if there's one thing I've
learned, it's that the ugly hack you write today with the words "we'll
tidy this up later" goes on to live with you forever!

>> >
>> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
>> > ---
>> > diff --git a/t/test-lib.sh b/t/test-lib.sh
>> > @@ -445,10 +452,37 @@ test_ok_ () {
>> >  test_failure_ () {
>> >         if test -n "$write_junit_xml"
>> >         then
>> > +               if test -z "$GIT_TEST_TEE_OUTPUT_FILE"
>> > +               then
>> > +                       case "$(type kill_p4d 2>/dev/null | head -n 1)" in
>> > +                       *function*) kill_p4d;;
>> > +                       esac
>> > +
>> > +                       case "$(type stop_git_daemon 2>/dev/null |
>> > +                               head -n 1)" in
>> > +                       *function*) stop_git_daemon;;
>> > +                       esac
>>
>> In the long run, it might make more sense, and be more scalable, to
>> have those scripts define a "prepare_for_rerun" variable or function
>> which this code then runs generically rather than having special
>> knowledge of those facilities.
>>
>> I could imagine, for instance, test-lib.sh defining a no-op:
>>
>>     test_failure_prepare_rerun () {}
>>
>> and then each of those scripts overriding the function:
>>
>>     # in lib-git-p4.sh
>>     test_failure_prepare_rerun () {
>>         kill_p4d
>>     }
>>
>>     # in lib-git-daemon.sh
>>     test_failure_prepare_rerun () {
>>         stop_git_daemon
>>     }
>
> Or we could implement `test_atexit` (similar to `test_when_finished`, but
> to be executed at `test_done` time). I guess that's what the p4 and daemon
> tests really needed to begin with (and probably also the apache2-using
> tests).
>
>>
>> > +                       # re-run with --verbose-log
>> > +                       echo "# Re-running: $junit_rerun_options_sq" >&2
>> > +
>> > +                       cd "$TEST_DIRECTORY" &&
>> > +                       eval "${TEST_SHELL_PATH}" "$junit_rerun_options_sq" \
>> > +                               >/dev/null 2>&1
>> > +                       status=$?
>> > +
>> > +                       say_color "" "$(test 0 = $status ||
>> > +                               echo "not ")ok $test_count - (re-ran with trace)"
>> > +                       say "1..$test_count"
>> > +                       GIT_EXIT_OK=t
>> > +                       exit $status
>> > +               fi
>> > +
>> >                 junit_insert="<failure message=\"not ok $test_count -"
>> >                 junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
>> >                 junit_insert="$junit_insert $(xml_attr_encode \
>> > -                       "$(printf '%s\n' "$@" | sed 1d)")"
>> > +                       "$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")"
>> > +               >"$GIT_TEST_TEE_OUTPUT_FILE"
>> >                 junit_insert="$junit_insert</failure>"
>> >                 write_junit_xml_testcase "$1" "      $junit_insert"
>> >         fi
>>
>> This junit-related stuff is getting pretty lengthy. I wonder if it
>> would make sense to pull it out to its own function at some point
>> (again, in the long run).
>
> Now that you mention it... I agree. This is getting long.
>
> In the short run, I have two things to consider, though: I want to make
> this work first, then think about introducing a layer of abstraction, and
> I want to go on vacation tomorrow.
>
> So I agree that this is something to be considered in the long run, i.e.
> not right now ;-)
>
> Thanks,
> Dscho

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 7/9] tests: include detailed trace logs with --write-junit-xml upon failure
  2018-09-05  5:32       ` Luke Diamand
@ 2018-09-05 12:39         ` Johannes Schindelin
  2018-09-05 13:03           ` Luke Diamand
  2018-09-05 18:38           ` Eric Sunshine
  0 siblings, 2 replies; 88+ messages in thread
From: Johannes Schindelin @ 2018-09-05 12:39 UTC (permalink / raw)
  To: Luke Diamand; +Cc: Eric Sunshine, gitgitgadget, Git List, Junio C Hamano

Hi Luke,

On Wed, 5 Sep 2018, Luke Diamand wrote:

> On 4 September 2018 at 12:09, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > On Tue, 4 Sep 2018, Eric Sunshine wrote:
> >
> >> On Mon, Sep 3, 2018 at 5:10 PM Johannes Schindelin via GitGitGadget
> >> <gitgitgadget@gmail.com> wrote:
> >> > So let's do something different in VSTS: let's run all the tests with
> >> > `--quiet` first, and only if a failure is encountered, try to trace the
> >> > commands as they are executed. [...]
> 
> Is this re-running just an individual test on its own or all the tests
> within a single file?

Upon encountering a failed test, it is re-running the entire test script
afresh.

> The latter shouldn't need this at all.

Please do not let me die dumb. In other words, I would love for you to
explain what exactly you mean by that sentence.

> And the former, I'm not sure will actually work - most of the tests
> assume some particular p4 state. But perhaps I'm missing something?

No, the former would not work at all. Not only for the p4 tests: Git's
tests frequently commit the deadly sin of relying on output of one another
(wreaking havoc e.g. when test cases are skipped due to missing
prerequisites, and latter test cases relying on their output). It is not
the only thing that is wrong with the test suite, of course.

> I also think it does look kind of ugly. And if there's one thing I've
> learned, it's that the ugly hack you write today with the words "we'll
> tidy this up later" goes on to live with you forever!

Okay.

(And having read lib-git-p4.sh, I kind of see where you learned that.)

But maybe you also have some splendid idea what to do instead? Because
doing something about it, that we need. We can't just say "oh, the only
solution we found is ugly, so let's not do it at all".

I am even going so far as to say: unless you have a better idea, it is
pretty detrimental to criticize the current approach. It is the opposite
of constructive.

So let's hear some ideas how to improve the situation, m'kay?

Just as a reminder, this is the problem I want to solve: I want to run the
tests in a light-weight manner, with minimal output, and only in case of
an error do I want to crank up the verbosity. Instead of wasting most of the
effort to log everything and then throwing it away in most of the common
cases, I suggest to re-run the entire test.

What do you suggest to solve this?

Ciao,
Johannes

> >> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> >> > ---
> >> > diff --git a/t/test-lib.sh b/t/test-lib.sh
> >> > @@ -445,10 +452,37 @@ test_ok_ () {
> >> >  test_failure_ () {
> >> >         if test -n "$write_junit_xml"
> >> >         then
> >> > +               if test -z "$GIT_TEST_TEE_OUTPUT_FILE"
> >> > +               then
> >> > +                       case "$(type kill_p4d 2>/dev/null | head -n 1)" in
> >> > +                       *function*) kill_p4d;;
> >> > +                       esac
> >> > +
> >> > +                       case "$(type stop_git_daemon 2>/dev/null |
> >> > +                               head -n 1)" in
> >> > +                       *function*) stop_git_daemon;;
> >> > +                       esac
> >>
> >> In the long run, it might make more sense, and be more scalable, to
> >> have those scripts define a "prepare_for_rerun" variable or function
> >> which this code then runs generically rather than having special
> >> knowledge of those facilities.
> >>
> >> I could imagine, for instance, test-lib.sh defining a no-op:
> >>
> >>     test_failure_prepare_rerun () {}
> >>
> >> and then each of those scripts overriding the function:
> >>
> >>     # in lib-git-p4.sh
> >>     test_failure_prepare_rerun () {
> >>         kill_p4d
> >>     }
> >>
> >>     # in lib-git-daemon.sh
> >>     test_failure_prepare_rerun () {
> >>         stop_git_daemon
> >>     }
> >
> > Or we could implement `test_atexit` (similar to `test_when_finished`, but
> > to be executed at `test_done` time). I guess that's what the p4 and daemon
> > tests really needed to begin with (and probably also the apache2-using
> > tests).
> >
> >>
> >> > +                       # re-run with --verbose-log
> >> > +                       echo "# Re-running: $junit_rerun_options_sq" >&2
> >> > +
> >> > +                       cd "$TEST_DIRECTORY" &&
> >> > +                       eval "${TEST_SHELL_PATH}" "$junit_rerun_options_sq" \
> >> > +                               >/dev/null 2>&1
> >> > +                       status=$?
> >> > +
> >> > +                       say_color "" "$(test 0 = $status ||
> >> > +                               echo "not ")ok $test_count - (re-ran with trace)"
> >> > +                       say "1..$test_count"
> >> > +                       GIT_EXIT_OK=t
> >> > +                       exit $status
> >> > +               fi
> >> > +
> >> >                 junit_insert="<failure message=\"not ok $test_count -"
> >> >                 junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
> >> >                 junit_insert="$junit_insert $(xml_attr_encode \
> >> > -                       "$(printf '%s\n' "$@" | sed 1d)")"
> >> > +                       "$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")"
> >> > +               >"$GIT_TEST_TEE_OUTPUT_FILE"
> >> >                 junit_insert="$junit_insert</failure>"
> >> >                 write_junit_xml_testcase "$1" "      $junit_insert"
> >> >         fi
> >>
> >> This junit-related stuff is getting pretty lengthy. I wonder if it
> >> would make sense to pull it out to its own function at some point
> >> (again, in the long run).
> >
> > Now that you mention it... I agree. This is getting long.
> >
> > In the short run, I have two things to consider, though: I want to make
> > this work first, then think about introducing a layer of abstraction, and
> > I want to go on vacation tomorrow.
> >
> > So I agree that this is something to be considered in the long run, i.e.
> > not right now ;-)
> >
> > Thanks,
> > Dscho
> 

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 7/9] tests: include detailed trace logs with --write-junit-xml upon failure
  2018-09-05 12:39         ` Johannes Schindelin
@ 2018-09-05 13:03           ` Luke Diamand
  2018-09-14 18:46             ` Johannes Schindelin
  2018-09-05 18:38           ` Eric Sunshine
  1 sibling, 1 reply; 88+ messages in thread
From: Luke Diamand @ 2018-09-05 13:03 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Eric Sunshine, gitgitgadget, Git List, Junio C Hamano

On 5 September 2018 at 13:39, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi Luke,
>
> On Wed, 5 Sep 2018, Luke Diamand wrote:
>
>> On 4 September 2018 at 12:09, Johannes Schindelin
>> <Johannes.Schindelin@gmx.de> wrote:
>> >
>> > On Tue, 4 Sep 2018, Eric Sunshine wrote:
>> >
>> >> On Mon, Sep 3, 2018 at 5:10 PM Johannes Schindelin via GitGitGadget
>> >> <gitgitgadget@gmail.com> wrote:
>> >> > So let's do something different in VSTS: let's run all the tests with
>> >> > `--quiet` first, and only if a failure is encountered, try to trace the
>> >> > commands as they are executed. [...]
>>
>> Is this re-running just an individual test on its own or all the tests
>> within a single file?
>
> Upon encountering a failed test, it is re-running the entire test script
> afresh.
>
>> The latter shouldn't need this at all.
>
> Please do not let me die dumb. In other words, I would love for you to
> explain what exactly you mean by that sentence.

Just re-run the script. You shouldn't need to kill p4d, as each script
starts up its own instance of p4d, and shuts it down when it exits.

$ cd t
$ ./t9800-git-p4-basic.sh
Ctrl^C
$ ./t9800-git-p4-basic.sh -v

There's a cleanup() function in lib-git-p4.sh which kills the p4d
server, and that's invoked via:

  trap cleanup EXIT

That's the only cleanup that each of the scripts require AFAIK.

>
>> And the former, I'm not sure will actually work - most of the tests
>> assume some particular p4 state. But perhaps I'm missing something?
>
> No, the former would not work at all. Not only for the p4 tests: Git's
> tests frequently commit the deadly sin of relying on output of one another
> (wreaking havoc e.g. when test cases are skipped due to missing
> prerequisites, and latter test cases relying on their output). It is not
> the only thing that is wrong with the test suite, of course.
>
>> I also think it does look kind of ugly. And if there's one thing I've
>> learned, it's that the ugly hack you write today with the words "we'll
>> tidy this up later" goes on to live with you forever!
>
> Okay.
>
> (And having read lib-git-p4.sh, I kind of see where you learned that.)
>
> But maybe you also have some splendid idea what to do instead? Because
> doing something about it, that we need. We can't just say "oh, the only
> solution we found is ugly, so let's not do it at all".
>
> I am even going so far as to say: unless you have a better idea, it is
> pretty detrimental to criticize the current approach. It is the opposite
> of constructive.
>
> So let's hear some ideas how to improve the situation, m'kay?
>
> Just as a reminder, this is the problem I want to solve: I want to run the
> tests in a light-weight manner, with minimal output, and only in case of
> an error do I want to crank up the verbosity. Instead of wasting most of the
> effort to log everything and then throwing it away in most of the common
> cases, I suggest to re-run the entire test.
>
> What do you suggest to solve this?
>

I don't know about any other tests, but the git-p4 tests don't take
any longer in verbose mode. So one simple solution is to just run it
in verbose mode - unless there are other tests which have more
overhead.

The trap/exit/cleanup method that the git-p4 tests already use would
seem to be ideally suited to cleaning up everything on exit.

There might be some specific tests where this doesn't quite work out,
if you let me know what they are I can have a look at fixing them for
you.

Thanks,
Luke

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 7/9] tests: include detailed trace logs with --write-junit-xml upon failure
  2018-09-05 12:39         ` Johannes Schindelin
  2018-09-05 13:03           ` Luke Diamand
@ 2018-09-05 18:38           ` Eric Sunshine
  2018-09-05 20:24             ` Jeff King
  2018-09-14 18:51             ` Johannes Schindelin
  1 sibling, 2 replies; 88+ messages in thread
From: Eric Sunshine @ 2018-09-05 18:38 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Luke Diamand, gitgitgadget, Git List, Junio C Hamano, Jeff King,
	Jonathan Nieder

On Wed, Sep 5, 2018 at 8:39 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> So let's hear some ideas how to improve the situation, m'kay?
> Just as a reminder, this is the problem I want to solve: I want to run the
> tests in a light-weight manner, with minimal output, and only in case of
> an error do I want to crank up the verbosity. Instead of wasting most of the
> effort to log everything and then throwing it away in most of the common
> cases, I suggest to re-run the entire test.

What about the very different approach of capturing the full "verbose"
output the executed tests in addition to whatever is actually output
to the terminal? If a test fails, then (and only then) you can insert
the captured verbose output into the JUnit XML file. This way (if we
always have the full verbose output at hand), you don't need to re-run
the test at all.

I've cc:'d Peff and Jonathan since I believe they are more familiar
with how all the capturing / output-redirection works in the test
suite.

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 2/9] ci/lib.sh: encapsulate Travis-specific things
  2018-09-03 21:10 ` [PATCH 2/9] ci/lib.sh: encapsulate Travis-specific things Johannes Schindelin via GitGitGadget
  2018-09-03 23:43   ` Eric Sunshine
@ 2018-09-05 18:57   ` Sebastian Schuberth
  2018-09-14 19:07     ` Johannes Schindelin
  1 sibling, 1 reply; 88+ messages in thread
From: Sebastian Schuberth @ 2018-09-05 18:57 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

On 9/3/2018 11:10 PM, Johannes Schindelin via GitGitGadget wrote:

> +if test -n "$TRAVIS_COMMIT"
> +then
> +	# We are running within Travis CI

Personally, I'd find a check like

if test "$TRAVIS" = "true"

more speaking (also see [1]).

[1] https://docs.travis-ci.com/user/environment-variables/

-- 
Sebastian Schuberth


^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services
  2018-09-03 21:10 [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Johannes Schindelin via GitGitGadget
                   ` (8 preceding siblings ...)
  2018-09-03 21:10 ` [PATCH 9/9] README: add a build badge (status of the VSTS build) Johannes Schindelin via GitGitGadget
@ 2018-09-05 19:01 ` Sebastian Schuberth
  2018-09-05 19:08   ` Stefan Beller
  2018-10-15 10:11 ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Johannes Schindelin via GitGitGadget
  2018-10-15 15:06 ` [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Ævar Arnfjörð Bjarmason
  11 siblings, 1 reply; 88+ messages in thread
From: Sebastian Schuberth @ 2018-09-05 19:01 UTC (permalink / raw)
  To: Johannes Schindelin via GitGitGadget, git; +Cc: Junio C Hamano

On 9/3/2018 11:10 PM, Johannes Schindelin via GitGitGadget wrote:

> The one sad part about this is the Windows support. Travis lacks it, and we
> work around that by using Visual Studio Team Services (VSTS) indirectly: one
> phase in Travis would trigger a build, wait for its log, and then paste that
> log.

I'm sorry if this has been discussed before, but as this recap doesn't 
mention it: Has AppVeyor been considered as an option? It seems to be 
the defacto standard for Windows CI for projects on GitHub.

-- 
Sebastian Schuberth

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services
  2018-09-05 19:01 ` [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Sebastian Schuberth
@ 2018-09-05 19:08   ` Stefan Beller
  0 siblings, 0 replies; 88+ messages in thread
From: Stefan Beller @ 2018-09-05 19:08 UTC (permalink / raw)
  To: Sebastian Schuberth; +Cc: gitgitgadget, git, Junio C Hamano

On Wed, Sep 5, 2018 at 12:02 PM Sebastian Schuberth
<sschuberth@gmail.com> wrote:
>
> On 9/3/2018 11:10 PM, Johannes Schindelin via GitGitGadget wrote:
>
> > The one sad part about this is the Windows support. Travis lacks it, and we
> > work around that by using Visual Studio Team Services (VSTS) indirectly: one
> > phase in Travis would trigger a build, wait for its log, and then paste that
> > log.
>
> I'm sorry if this has been discussed before, but as this recap doesn't
> mention it: Has AppVeyor been considered as an option? It seems to be
> the defacto standard for Windows CI for projects on GitHub.

There was
https://public-inbox.org/git/BAY169-W30CD27F2F7606F4DF52944A78F0@phx.gbl/
and
https://public-inbox.org/git/alpine.DEB.2.20.1703241242210.17768@tvnag.unkk.fr/

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 7/9] tests: include detailed trace logs with --write-junit-xml upon failure
  2018-09-05 18:38           ` Eric Sunshine
@ 2018-09-05 20:24             ` Jeff King
  2018-09-14 19:04               ` Johannes Schindelin
  2018-09-14 18:51             ` Johannes Schindelin
  1 sibling, 1 reply; 88+ messages in thread
From: Jeff King @ 2018-09-05 20:24 UTC (permalink / raw)
  To: Eric Sunshine
  Cc: Johannes Schindelin, Luke Diamand, gitgitgadget, Git List,
	Junio C Hamano, Jonathan Nieder

On Wed, Sep 05, 2018 at 02:38:34PM -0400, Eric Sunshine wrote:

> On Wed, Sep 5, 2018 at 8:39 AM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> > So let's hear some ideas how to improve the situation, m'kay?
> > Just as a reminder, this is the problem I want to solve: I want to run the
> > tests in a light-weight manner, with minimal output, and only in case of
> > an error do I want to crank up the verbosity. Instead of wasting most of the
> > effort to log everything and then throwing it away in most of the common
> > cases, I suggest to re-run the entire test.
> 
> What about the very different approach of capturing the full "verbose"
> output the executed tests in addition to whatever is actually output
> to the terminal? If a test fails, then (and only then) you can insert
> the captured verbose output into the JUnit XML file. This way (if we
> always have the full verbose output at hand), you don't need to re-run
> the test at all.
> 
> I've cc:'d Peff and Jonathan since I believe they are more familiar
> with how all the capturing / output-redirection works in the test
> suite.

I don't think there's much to know beyond what you wrote. The
"--verbose" case does not really cost any more than the non-verbose one,
because the only difference is whether output goes to a file versus
/dev/null. But the commands all call write() regardless.

For --verbose-log, it does of course cost a little extra to run one
`tee` per script, and to write a few kb of logs. I doubt those are
measurable versus the rest of a script run, but I'm happy to be
disproven by numbers. There are some gymnastics done to re-exec the test
script with the same shell, but AFAIK those are robust and don't cost a
lot (again, one extra process per script run).

I'm not overly concerned about the cost of re-running a test, since the
idea is that failed tests should be rare. I would be a little worried
about flaky tests mysteriously righting themselves on the second run (so
you know a failure happened, but you have no good output to describe
it).

I do agree that a test_atexit() cleanup would make a lot more sense than
what's in the original patch. And that's nicer than the exit trap we're
using already, because you may note that each caller has to manually
restore the original 'die' handler that test-lib.sh installs.

That would also help with bitrot. If this funky cleanup case only causes
problems with junit output, then other people are likely to forget to do
it, and the mess falls onto the junit folks (i.e., Dscho). But if the
tests start _relying_ on test_atexit() being called (i.e., don't call
stop_git_daemon manually, but assume that the atexit handler does so),
then the responsible authors are a lot more likely to notice and fix it
early.

-Peff

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 7/9] tests: include detailed trace logs with --write-junit-xml upon failure
  2018-09-05 13:03           ` Luke Diamand
@ 2018-09-14 18:46             ` Johannes Schindelin
  0 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin @ 2018-09-14 18:46 UTC (permalink / raw)
  To: Luke Diamand; +Cc: Eric Sunshine, gitgitgadget, Git List, Junio C Hamano

Hi Luke,

On Wed, 5 Sep 2018, Luke Diamand wrote:

> On 5 September 2018 at 13:39, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > On Wed, 5 Sep 2018, Luke Diamand wrote:
> >
> >> On 4 September 2018 at 12:09, Johannes Schindelin
> >> <Johannes.Schindelin@gmx.de> wrote:
> >> >
> >> > On Tue, 4 Sep 2018, Eric Sunshine wrote:
> >> >
> >> >> On Mon, Sep 3, 2018 at 5:10 PM Johannes Schindelin via GitGitGadget
> >> >> <gitgitgadget@gmail.com> wrote:
> >> >> > So let's do something different in VSTS: let's run all the tests with
> >> >> > `--quiet` first, and only if a failure is encountered, try to trace the
> >> >> > commands as they are executed. [...]
> >>
> >> Is this re-running just an individual test on its own or all the tests
> >> within a single file?
> >
> > Upon encountering a failed test, it is re-running the entire test script
> > afresh.
> >
> >> The latter shouldn't need this at all.
> >
> > Please do not let me die dumb. In other words, I would love for you to
> > explain what exactly you mean by that sentence.
> 
> Just re-run the script. You shouldn't need to kill p4d, as each script
> starts up its own instance of p4d, and shuts it down when it exits.
> 
> $ cd t
> $ ./t9800-git-p4-basic.sh
> Ctrl^C

That Ctrl^C is not possible in the automated builds (which are the subject
of this patch series).

What *is* possible is to detect in the test script that there was a
breakage, and then re-run the test script.

Unfortunately, at this stage the first test script has not exited yet, so
your p4d has not exited yet, and we have to clean up in some way before
re-running (because the port in use is determined by the test script's
number, so we really need to kill the old daemon).

And this is exactly what my patches try to achieve here.

> $ ./t9800-git-p4-basic.sh -v
> 
> There's a cleanup() function in lib-git-p4.sh which kills the p4d
> server, and that's invoked via:
> 
>   trap cleanup EXIT
> 
> That's the only cleanup that each of the scripts require AFAIK.

Good.

However, we really will want to consider introducing something consistent
here, something that works for *all* test scripts. IOW if any test script
wants to start a process for the life-time of that script, and needs to
kill that process at the end, we will want to have a consistent,
documented way to register a function (or commands) to be called at the
end.

And that function (or commands) need to be run also when stopping
everything in preparation for a re-run.

> >> And the former, I'm not sure will actually work - most of the tests
> >> assume some particular p4 state. But perhaps I'm missing something?
> >
> > No, the former would not work at all. Not only for the p4 tests: Git's
> > tests frequently commit the deadly sin of relying on output of one another
> > (wreaking havoc e.g. when test cases are skipped due to missing
> > prerequisites, and latter test cases relying on their output). It is not
> > the only thing that is wrong with the test suite, of course.
> >
> >> I also think it does look kind of ugly. And if there's one thing I've
> >> learned, it's that the ugly hack you write today with the words "we'll
> >> tidy this up later" goes on to live with you forever!
> >
> > Okay.
> >
> > (And having read lib-git-p4.sh, I kind of see where you learned that.)
> >
> > But maybe you also have some splendid idea what to do instead? Because
> > doing something about it, that we need. We can't just say "oh, the only
> > solution we found is ugly, so let's not do it at all".
> >
> > I am even going so far as to say: unless you have a better idea, it is
> > pretty detrimental to criticize the current approach. It is the opposite
> > of constructive.
> >
> > So let's hear some ideas how to improve the situation, m'kay?
> >
> > Just as a reminder, this is the problem I want to solve: I want to run the
> > tests in a light-weight manner, with minimal output, and only in case of
> > an error do I want to crank up the verbosity. Instead of wasting most of the
> > effort to log everything and then throwing it away in most of the common
> > cases, I suggest to re-run the entire test.
> >
> > What do you suggest to solve this?
> >
> 
> I don't know about any other tests, but the git-p4 tests don't take
> any longer in verbose mode.

That statement is too general to be correct. It may be the case in your
setup, but I found that even a redirected, chatty stderr can slow down
things on Windows substantially, probably due to the POSIX emulation going
on in the background that needs to prepare for all kinds of eventualities
(that we do not even need, but the MSYS2 runtime has no way of knowing
that).

> So one simple solution is to just run it in verbose mode - unless there
> are other tests which have more overhead.

It is a bit of an overhead. As the entire test run on Windows takes
*quite* a while (due to said POSIX emulation issues), I was not able to
quantify it exactly.

> The trap/exit/cleanup method that the git-p4 tests already use would
> seem to be ideally suited to cleaning up everything on exit.

Yes. On exit. But I cannot exit here, not before re-running the script.

> There might be some specific tests where this doesn't quite work out,
> if you let me know what they are I can have a look at fixing them for
> you.

Thanks. I will let you know when I encounter other tests than the p4 and
git-daemon tests (which I fixed in this patch series).

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 7/9] tests: include detailed trace logs with --write-junit-xml upon failure
  2018-09-05 18:38           ` Eric Sunshine
  2018-09-05 20:24             ` Jeff King
@ 2018-09-14 18:51             ` Johannes Schindelin
  1 sibling, 0 replies; 88+ messages in thread
From: Johannes Schindelin @ 2018-09-14 18:51 UTC (permalink / raw)
  To: Eric Sunshine
  Cc: Luke Diamand, gitgitgadget, Git List, Junio C Hamano, Jeff King,
	Jonathan Nieder

Hi Eric,

On Wed, 5 Sep 2018, Eric Sunshine wrote:

> On Wed, Sep 5, 2018 at 8:39 AM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> > So let's hear some ideas how to improve the situation, m'kay?  Just as
> > a reminder, this is the problem I want to solve: I want to run the
> > tests in a light-weight manner, with minimal output, and only in case
> > of an error do I want to crank up the verbosity. Instead of wasting
> > most of the effort to log everything and then throwing it away in most
> > of the common cases, I suggest to re-run the entire test.
> 
> What about the very different approach of capturing the full "verbose"
> output the executed tests in addition to whatever is actually output
> to the terminal?

I fear it is not really possible to do a "verbose but not really" mode. I
want the console output to be quiet, there is no use in chatting up the
build log with these messages, as we have to run the tests in parallel, so
the output would be utterly hard to interpret anyway. At the same time, I
want verbose output for use in the test results. It is not really possible
to `tee` all output, then "quiet down" the part that makes it into the
log.

> If a test fails, then (and only then) you can insert the captured
> verbose output into the JUnit XML file.

Yep. That's what my patch series does.

If a test fails, then (and only then) I re-run the script with verbose
output that is immediately moved into that JUnit XML file.

> This way (if we always have the full verbose output at hand), you don't
> need to re-run the test at all.

But that way, if we always have the full verbose output, the build log
will be unnecessarily verbose and confusing.

> I've cc:'d Peff and Jonathan since I believe they are more familiar
> with how all the capturing / output-redirection works in the test
> suite.

Okay.

Ciao,
Dscho

P.S.: An unintended side effect of re-running the tests is to identify
flakey tests. I do not yet have a way to represent this outcome in the
test result, but I deem this an additional benefit in favor of keeping my
current strategy.

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 7/9] tests: include detailed trace logs with --write-junit-xml upon failure
  2018-09-05 20:24             ` Jeff King
@ 2018-09-14 19:04               ` Johannes Schindelin
  0 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin @ 2018-09-14 19:04 UTC (permalink / raw)
  To: Jeff King
  Cc: Eric Sunshine, Luke Diamand, gitgitgadget, Git List,
	Junio C Hamano, Jonathan Nieder

Hi Peff,

On Wed, 5 Sep 2018, Jeff King wrote:

> On Wed, Sep 05, 2018 at 02:38:34PM -0400, Eric Sunshine wrote:
> 
> > On Wed, Sep 5, 2018 at 8:39 AM Johannes Schindelin
> > <Johannes.Schindelin@gmx.de> wrote:
> > > So let's hear some ideas how to improve the situation, m'kay?  Just
> > > as a reminder, this is the problem I want to solve: I want to run
> > > the tests in a light-weight manner, with minimal output, and only in
> > > case of an error do I want to crank up the verbosity. Instead of
> > > wasting most of the effort to log everything and then throwing it
> > > away in most of the common cases, I suggest to re-run the entire
> > > test.
> > 
> > What about the very different approach of capturing the full "verbose"
> > output the executed tests in addition to whatever is actually output
> > to the terminal? If a test fails, then (and only then) you can insert
> > the captured verbose output into the JUnit XML file. This way (if we
> > always have the full verbose output at hand), you don't need to re-run
> > the test at all.
> > 
> > I've cc:'d Peff and Jonathan since I believe they are more familiar
> > with how all the capturing / output-redirection works in the test
> > suite.
> 
> I don't think there's much to know beyond what you wrote. The
> "--verbose" case does not really cost any more than the non-verbose one,
> because the only difference is whether output goes to a file versus
> /dev/null. But the commands all call write() regardless.

This really only holds true on systems where redirected output is
essentially a no-op.

For the MSYS2-emulated case, this is incorrect. And as that case is
already suffering, time-wise, I would be loathe to pile more onto it.

> For --verbose-log, it does of course cost a little extra to run one
> `tee` per script, and to write a few kb of logs. I doubt those are
> measurable versus the rest of a script run, but I'm happy to be
> disproven by numbers.

I tried, and failed, to quantify this properly. Essentially because a
single test run takes over 1h15m, so I tried to do it in the cloud, but
those timings are too variable for anything approaching an accurate
measurement.

> There are some gymnastics done to re-exec the test script with the same
> shell, but AFAIK those are robust and don't cost a lot (again, one extra
> process per script run).

It *is* actually a bit worse here (but only in the case of failures, which
should be the minority of the cases): imagine an expensive test like t0027
that takes something like 14 minutes to run, and then a test failure near
the end: re-running with verbose output will cost another 14 minutes.

However, the benefit of allowing to pass flakey tests outweighs the costs
here, if you ask me.

> I'm not overly concerned about the cost of re-running a test, since the
> idea is that failed tests should be rare. I would be a little worried
> about flaky tests mysteriously righting themselves on the second run (so
> you know a failure happened, but you have no good output to describe
> it).

This is actually a benefit.

Pretty much all the times I can remember a test being flakey (with the one
notable exception of the O_APPEND issue with GIT_TRACE), the *tests* were
buggy.

Plus, it is less concerning if a test fails occasionally vs a test that
fails all the time.

My plan is to indicate the outcome of "Passed upon rerun" in the test
output eventually, as soon as there is server-side support for it.

(Sidenote: there is technically already server-side support for it, but we
would have to generate Visual Studio-style test output, and I had the
hunch that some die-hard Linuxers among the core Git developers would take
objection to that.)

> I do agree that a test_atexit() cleanup would make a lot more sense than
> what's in the original patch. And that's nicer than the exit trap we're
> using already, because you may note that each caller has to manually
> restore the original 'die' handler that test-lib.sh installs.

Okay, then. I will work on this.

Ciao,
Dscho

> That would also help with bitrot. If this funky cleanup case only causes
> problems with junit output, then other people are likely to forget to do
> it, and the mess falls onto the junit folks (i.e., Dscho). But if the
> tests start _relying_ on test_atexit() being called (i.e., don't call
> stop_git_daemon manually, but assume that the atexit handler does so),
> then the responsible authors are a lot more likely to notice and fix it
> early.
> 
> -Peff
> 

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 2/9] ci/lib.sh: encapsulate Travis-specific things
  2018-09-05 18:57   ` Sebastian Schuberth
@ 2018-09-14 19:07     ` Johannes Schindelin
  0 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin @ 2018-09-14 19:07 UTC (permalink / raw)
  To: Sebastian Schuberth; +Cc: git, Junio C Hamano

Hi Sebastian,

On Wed, 5 Sep 2018, Sebastian Schuberth wrote:

> On 9/3/2018 11:10 PM, Johannes Schindelin via GitGitGadget wrote:
> 
> > +if test -n "$TRAVIS_COMMIT"
> > +then
> > +	# We are running within Travis CI
> 
> Personally, I'd find a check like
> 
> if test "$TRAVIS" = "true"
> 
> more speaking (also see [1]).

Good call.

Will fix,
Dscho

> 
> [1] https://docs.travis-ci.com/user/environment-variables/
> 
> -- 
> Sebastian Schuberth
> 
> 

^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines
  2018-09-03 21:10 [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Johannes Schindelin via GitGitGadget
                   ` (9 preceding siblings ...)
  2018-09-05 19:01 ` [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Sebastian Schuberth
@ 2018-10-15 10:11 ` Johannes Schindelin via GitGitGadget
  2018-10-15 10:11   ` [PATCH v2 01/13] ci: rename the library of common functions Johannes Schindelin via GitGitGadget
                     ` (14 more replies)
  2018-10-15 15:06 ` [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Ævar Arnfjörð Bjarmason
  11 siblings, 15 replies; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-10-15 10:11 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano

For a long time already, we have Git's source code continuously tested via
Travis CI, see e.g. https://travis-ci.org/git/git/builds/421738884. It has
served us well, and more and more developers actually pay attention and
benefit from the testing this gives us.

It is also an invaluable tool for contributors who can validate their code
contributions via PRs on GitHub, e.g. to verify that their tests do actually
run on macOS (i.e. with the BSD family of Unix tools instead of the GNU
one).

The one sad part about this is the Windows support. Travis lacks it, and we
work around that by using Azure Pipelines (the CI part of Azure DevOps,
formerly known as Visual Studio Team Services) indirectly: one phase in
Travis would trigger a build, wait for its log, and then paste that log.

As Git's Windows builds (and tests!) take quite a bit of time, Travis often
timed out, or somehow the trigger did not work, and for security reasons
(the Windows builds are performed in a private pool of containers), the
Windows builds are completely disabled for Pull Requests on GitHub.

One might ask why we did not use Azure Pipelines directly. There were a
couple of reasons for that:

 * most notably, Azure Pipelines' build logs could not be viewed
   anonymously,
 * while Azure Pipelines had Linux and Windows agents, it lacked macOS
   agents,
 * etc

The main two reasons no longer apply: macOS agents are available now
[https://docs.microsoft.com/en-us/azure/devops/release-notes/2018/jul-10-vsts]
, and there is a limited preview of "public projects"
[https://blogs.msdn.microsoft.com/devops/2018/04/27/vsts-public-projects-limited-preview/]
, i.e. it is possible to configure a Azure Pipelines project so that anybody
can view the logs.

I had secured such a public project for Git for Windows already, and I
recently also got one for Git. For now, the latter is hooked up with my
personal git.git fork on GitHub, but it is my hope that I convince y'all
that these Azure Pipelines builds are a good idea, and then hook it up with 
https://github.com/git/git.

As a special treat, this patch series adds the ability to present the
outcome of Git's test suite as JUnit-style .xml files. This allows the Azure
Pipelines build to present fun diagrams, trends, and makes it a lot easier
to drill down to test failures than before. See for example 
https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details
[https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details] 
(you can click on the label of the failed test, and then see the detailed
output in the right pane).

This patch series took way more time than I had originally planned, but I
think that in particular the advanced display of the test results was worth
it. Please let me know what you think about this.

Changes since v1:

 * Removed a superfluous eval.
 * Added the commit that fixes the Travis PR builds targeting master that 
   just happens to be tagged (see 
   https://travis-ci.org/git/git/jobs/424276413 for an incorrectly-skipped
   build).
 * The commit messages and the cover letter now reflect the name change from
   Visual Studio Team Services to Azure DevOps (and in particular, Azure
   Pipelines for the automated builds).
 * Now we're using test_atexit (which we introduced for that purpose)
   instead of hard-coding kill_p4d and stop_git_daemon.
 * The build should now also succeed for Pull Requests (where secret
   variables are not available, for security reasons, and as a consequence
   the file share cannot be mounted).
 * The shell scripted parts now use proper && chains.

Johannes Schindelin (13):
  ci: rename the library of common functions
  ci/lib.sh: encapsulate Travis-specific things
  test-date: add a subcommand to measure times in shell scripts
  tests: optionally write results as JUnit-style .xml
  ci/lib.sh: add support for Azure Pipelines
  Add a build definition for Azure DevOps
  tests: introduce `test_atexit`
  git-daemon: use `test_atexit` in the tests
  git-p4: use `test_atexit` to kill the daemon
  tests: include detailed trace logs with --write-junit-xml upon failure
  tests: record more stderr with --write-junit-xml in case of failure
  README: add a build badge (status of the Azure Pipelines build)
  travis: fix skipping tagged releases

 README.md                                  |   2 +
 azure-pipelines.yml                        | 319 +++++++++++++++++++++
 ci/install-dependencies.sh                 |   5 +-
 ci/{lib-travisci.sh => lib.sh}             |  67 ++++-
 ci/mount-fileshare.sh                      |  26 ++
 ci/print-test-failures.sh                  |   4 +-
 ci/run-build-and-tests.sh                  |   2 +-
 ci/run-linux32-docker.sh                   |   2 +-
 ci/run-static-analysis.sh                  |   2 +-
 ci/run-windows-build.sh                    |   2 +-
 ci/test-documentation.sh                   |   3 +-
 t/.gitignore                               |   1 +
 t/helper/test-date.c                       |  12 +
 t/interop/i5500-git-daemon.sh              |   1 -
 t/lib-git-daemon.sh                        |   3 +-
 t/lib-git-p4.sh                            |  10 +-
 t/t0000-basic.sh                           |  20 ++
 t/t5570-git-daemon.sh                      |   1 -
 t/t9800-git-p4-basic.sh                    |   4 -
 t/t9801-git-p4-branch.sh                   |   4 -
 t/t9802-git-p4-filetype.sh                 |   4 -
 t/t9803-git-p4-shell-metachars.sh          |   4 -
 t/t9804-git-p4-label.sh                    |   4 -
 t/t9805-git-p4-skip-submit-edit.sh         |   4 -
 t/t9806-git-p4-options.sh                  |   5 -
 t/t9807-git-p4-submit.sh                   |   4 -
 t/t9808-git-p4-chdir.sh                    |   4 -
 t/t9809-git-p4-client-view.sh              |   4 -
 t/t9810-git-p4-rcs.sh                      |   4 -
 t/t9811-git-p4-label-import.sh             |   5 -
 t/t9812-git-p4-wildcards.sh                |   4 -
 t/t9813-git-p4-preserve-users.sh           |   4 -
 t/t9814-git-p4-rename.sh                   |   4 -
 t/t9815-git-p4-submit-fail.sh              |   4 -
 t/t9816-git-p4-locked.sh                   |   4 -
 t/t9817-git-p4-exclude.sh                  |   4 -
 t/t9818-git-p4-block.sh                    |   4 -
 t/t9819-git-p4-case-folding.sh             |   4 -
 t/t9820-git-p4-editor-handling.sh          |   4 -
 t/t9821-git-p4-path-variations.sh          |   4 -
 t/t9822-git-p4-path-encoding.sh            |   4 -
 t/t9823-git-p4-mock-lfs.sh                 |   4 -
 t/t9824-git-p4-git-lfs.sh                  |   4 -
 t/t9825-git-p4-handle-utf16-without-bom.sh |   4 -
 t/t9826-git-p4-keep-empty-commits.sh       |   4 -
 t/t9827-git-p4-change-filetype.sh          |   4 -
 t/t9828-git-p4-map-user.sh                 |   4 -
 t/t9829-git-p4-jobs.sh                     |   4 -
 t/t9830-git-p4-symlink-dir.sh              |   4 -
 t/t9831-git-p4-triggers.sh                 |   4 -
 t/t9832-unshelve.sh                        |   4 -
 t/t9833-errors.sh                          |   5 -
 t/test-lib-functions.sh                    |  29 ++
 t/test-lib.sh                              | 140 +++++++++
 54 files changed, 616 insertions(+), 174 deletions(-)
 create mode 100644 azure-pipelines.yml
 rename ci/{lib-travisci.sh => lib.sh} (61%)
 create mode 100755 ci/mount-fileshare.sh


base-commit: 5a0cc8aca797dbd7d2be3b67458ff880ed45cddf
Published-As: https://github.com/gitgitgadget/git/releases/tags/pr-31%2Fdscho%2Fvsts-ci-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-31/dscho/vsts-ci-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/31

Range-diff vs v1:

  1:  858b80bfcf !  1:  c963184510 ci: rename the library of common functions
     @@ -5,8 +5,8 @@
          The name is hard-coded to reflect that we use Travis CI for continuous
          testing.
      
     -    In the next commits, we will extend this to be able use Visual Studio
     -    Team Services, too.
     +    In the next commits, we will extend this to be able use Azure DevOps,
     +    too.
      
          So let's adjust the name to make it more generic.
      
  2:  18e6beec5f !  2:  815152e0f5 ci/lib.sh: encapsulate Travis-specific things
     @@ -2,8 +2,9 @@
      
          ci/lib.sh: encapsulate Travis-specific things
      
     -    The upcoming patches will allow building git.git via VSTS CI, where
     -    variable names and URLs look a bit different than in Travis CI.
     +    The upcoming patches will allow building git.git via Azure Pipelines
     +    (i.e. Azure DevOps' Continuous Integration), where variable names and
     +    URLs look a bit different than in Travis CI.
      
          Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
      
     @@ -16,7 +17,7 @@
       	# brew install gnu-time
      -	brew install git-lfs gettext
      +	test -z "$BREW_INSTALL_PACKAGES" ||
     -+	eval brew install $BREW_INSTALL_PACKAGES
     ++	brew install $BREW_INSTALL_PACKAGES
       	brew link --force gettext
       	brew install caskroom/cask/perforce
       	;;
     @@ -27,7 +28,7 @@
      @@
       # Library of functions shared by all CI scripts
       
     -+if test -n "$TRAVIS_COMMIT"
     ++if test true = "$TRAVIS"
      +then
      +	# We are running within Travis CI
      +	CI_BRANCH="$TRAVIS_BRANCH"
  3:  064db77669 =  3:  52337f1875 test-date: add a subcommand to measure times in shell scripts
  4:  2b5d678594 !  4:  cf4c5ae470 tests: optionally write results as JUnit-style .xml
     @@ -3,7 +3,7 @@
          tests: optionally write results as JUnit-style .xml
      
          This will come in handy when publishing the results of Git's test suite
     -    during an automated VSTS CI run.
     +    during an automated Azure DevOps run.
      
          Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
      
  5:  127dfcfb09 !  5:  486d1d2518 ci/lib.sh: add support for VSTS CI
     @@ -1,6 +1,6 @@
      Author: Johannes Schindelin <johannes.schindelin@gmx.de>
      
     -    ci/lib.sh: add support for VSTS CI
     +    ci/lib.sh: add support for Azure Pipelines
      
          This patch introduces a conditional arm that defines some environment
          variables and a function that displays the URL given the job id (to
     @@ -17,7 +17,7 @@
       	export GIT_TEST_OPTS="--verbose-log -x --immediate"
      +elif test -n "$SYSTEM_TASKDEFINITIONSURI"
      +then
     -+	# We are running in VSTS CI
     ++	# We are running in Azure Pipelines
      +	CI_BRANCH="$BUILD_SOURCEBRANCH"
      +	CI_COMMIT="$BUILD_SOURCEVERSION"
      +	CI_JOB_ID="$BUILD_BUILDID"
     @@ -29,7 +29,7 @@
      +
      +	# use a subdirectory of the cache dir (because the file share is shared
      +	# among *all* phases)
     -+	cache_dir="$HOME/vsts-cache/$SYSTEM_PHASENAME"
     ++	cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME"
      +
      +	url_for_job_id () {
      +		echo "$SYSTEM_TASKDEFINITIONSURI$SYSTEM_TEAMPROJECT/_build/results?buildId=$1"
  6:  ab66060c57 !  6:  1a22efe849 Add a build definition for VSTS
     @@ -1,8 +1,8 @@
      Author: Johannes Schindelin <johannes.schindelin@gmx.de>
      
     -    Add a build definition for VSTS
     +    Add a build definition for Azure DevOps
      
     -    This commit adds a .vsts-ci.yml which is Visual Studio Team Services'
     +    This commit adds an azure-pipelines.yml file which is Azure DevOps'
          equivalent to Travis CI's .travis.yml.
      
          To make things a bit easier to understand, we refrain from using the
     @@ -19,10 +19,10 @@
      
          Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
      
     -diff --git a/.vsts-ci.yml b/.vsts-ci.yml
     +diff --git a/azure-pipelines.yml b/azure-pipelines.yml
      new file mode 100644
      --- /dev/null
     -+++ b/.vsts-ci.yml
     ++++ b/azure-pipelines.yml
      @@
      +resources:
      +- repo: self
     @@ -36,12 +36,12 @@
      +    name: Hosted Ubuntu 1604
      +  steps:
      +  - bash: |
     -+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
      +
     -+       sudo apt-get update
     -+       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin
     ++       sudo apt-get update &&
     ++       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin &&
      +
     -+       export CC=clang
     ++       export CC=clang || exit 1
      +
      +       ci/install-dependencies.sh
      +       ci/run-build-and-tests.sh || {
     @@ -49,8 +49,10 @@
      +           exit 1
      +       }
      +
     -+       sudo umount "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
      +    displayName: 'ci/run-build-and-tests.sh'
     ++    env:
     ++      GITFILESHAREPWD: $(gitfileshare.pwd)
      +  - task: PublishTestResults@2
      +    displayName: 'Publish Test Results **/TEST-*.xml'
      +    inputs:
     @@ -67,10 +69,10 @@
      +    name: Hosted Ubuntu 1604
      +  steps:
      +  - bash: |
     -+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
      +
     -+       sudo apt-get update
     -+       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin
     ++       sudo apt-get update &&
     ++       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin || exit 1
      +
      +       ci/install-dependencies.sh
      +       ci/run-build-and-tests.sh || {
     @@ -78,8 +80,10 @@
      +           exit 1
      +       }
      +
     -+       sudo umount "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
      +    displayName: 'ci/run-build-and-tests.sh'
     ++    env:
     ++      GITFILESHAREPWD: $(gitfileshare.pwd)
      +  - task: PublishTestResults@2
      +    displayName: 'Publish Test Results **/TEST-*.xml'
      +    inputs:
     @@ -96,7 +100,7 @@
      +    name: Hosted macOS
      +  steps:
      +  - bash: |
     -+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
      +
      +       export CC=clang
      +
     @@ -106,8 +110,10 @@
      +           exit 1
      +       }
      +
     -+       umount "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1
      +    displayName: 'ci/run-build-and-tests.sh'
     ++    env:
     ++      GITFILESHAREPWD: $(gitfileshare.pwd)
      +  - task: PublishTestResults@2
      +    displayName: 'Publish Test Results **/TEST-*.xml'
      +    inputs:
     @@ -124,7 +130,7 @@
      +    name: Hosted macOS
      +  steps:
      +  - bash: |
     -+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
      +
      +       ci/install-dependencies.sh
      +       ci/run-build-and-tests.sh || {
     @@ -132,8 +138,10 @@
      +           exit 1
      +       }
      +
     -+       umount "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1
      +    displayName: 'ci/run-build-and-tests.sh'
     ++    env:
     ++      GITFILESHAREPWD: $(gitfileshare.pwd)
      +  - task: PublishTestResults@2
      +    displayName: 'Publish Test Results **/TEST-*.xml'
      +    inputs:
     @@ -150,20 +158,22 @@
      +    name: Hosted Ubuntu 1604
      +  steps:
      +  - bash: |
     -+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
      +
     -+       sudo apt-get update
     -+       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev
     ++       sudo apt-get update &&
     ++       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev &&
      +
     -+       export jobname=GETTEXT_POISON
     ++       export jobname=GETTEXT_POISON || exit 1
      +
      +       ci/run-build-and-tests.sh || {
      +           ci/print-test-failures.sh
      +           exit 1
      +       }
      +
     -+       sudo umount "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
      +    displayName: 'ci/run-build-and-tests.sh'
     ++    env:
     ++      GITFILESHAREPWD: $(gitfileshare.pwd)
      +  - task: PublishTestResults@2
      +    displayName: 'Publish Test Results **/TEST-*.xml'
      +    inputs:
     @@ -184,8 +194,10 @@
      +       # Helper to check the error level of the latest command (exit with error when appropriate)
      +       function c() { if (!$?) { exit(1) } }
      +
     -+       net use s: \\gitfileshare.file.core.windows.net\vsts-cache "$(gitfileshare.pwd)" /user:AZURE\gitfileshare /persistent:no; c
     -+       cmd /c mklink /d "$(Build.SourcesDirectory)\vsts-cache" S:\; c
     ++       if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") {
     ++         net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no; c
     ++         cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\; c
     ++       }
      +
      +       # Add build agent's MinGit to PATH
      +       $env:PATH = $env:AGENT_HOMEDIRECTORY +"\externals\\git\cmd;" +$env:PATH
     @@ -215,7 +227,7 @@
      +
      +       # Initialize Git for Windows' SDK
      +       $sdk_path = "$(Build.SourcesDirectory)\git-sdk-64"
     -+       init "$sdk_path" "https://git-for-windows.visualstudio.com/_git/git-sdk-64" 0
     ++       init "$sdk_path" "https://dev.azure.com/git-for-windows/git-sdk-64/_git/git-sdk-64" 0
      +       init usr\src\build-extra https://github.com/git-for-windows/build-extra 1
      +
      +       cd "$(Build.SourcesDirectory)"; c
     @@ -227,14 +239,19 @@
      +
      +         make -j10 DEVELOPER=1 NO_PERL=1 || exit 1
      +         NO_PERL=1 NO_SVN_TESTS=1 GIT_TEST_OPTS=\"--quiet --write-junit-xml\" time make -j15 -k DEVELOPER=1 test || {
     -+           NO_PERL=1 NO_SVN_TESTS=1 GIT_TEST_OPTS=\"-i -v -x\" make -k failed; exit 1
     ++           NO_PERL=1 NO_SVN_TESTS=1 GIT_TEST_OPTS=\"-i -v -x\" make -k -C t failed; exit 1
      +         }
      +
      +         save_good_tree
      +       "@
     ++       c
      +
     -+       cmd /c rmdir "$(Build.SourcesDirectory)\vsts-cache"
     ++       if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") {
     ++         cmd /c rmdir "$(Build.SourcesDirectory)\test-cache"
     ++       }
      +    displayName: 'build & test'
     ++    env:
     ++      GITFILESHAREPWD: $(gitfileshare.pwd)
      +  - task: PublishTestResults@2
      +    displayName: 'Publish Test Results **/TEST-*.xml'
      +    inputs:
     @@ -251,28 +268,30 @@
      +    name: Hosted Ubuntu 1604
      +  steps:
      +  - bash: |
     -+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
      +
     -+       sudo apt-get update
     ++       sudo apt-get update &&
      +       sudo apt-get -y install \
      +           apt-transport-https \
      +           ca-certificates \
      +           curl \
     -+           software-properties-common
     -+       curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
     ++           software-properties-common &&
     ++       curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - &&
      +       sudo add-apt-repository \
      +          "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
      +          $(lsb_release -cs) \
     -+          stable"
     -+       sudo apt-get update
     -+       sudo apt-get -y install docker-ce
     ++          stable" &&
     ++       sudo apt-get update &&
     ++       sudo apt-get -y install docker-ce &&
      +
      +       sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS=-j3 bash -lxc ci/run-linux32-docker.sh || exit 1
      +
      +       sudo chmod a+r t/out/TEST-*.xml
      +
     -+       sudo umount "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
      +    displayName: 'ci/run-linux32-docker.sh'
     ++    env:
     ++      GITFILESHAREPWD: $(gitfileshare.pwd)
      +  - task: PublishTestResults@2
      +    displayName: 'Publish Test Results **/TEST-*.xml'
      +    inputs:
     @@ -289,17 +308,19 @@
      +    name: Hosted Ubuntu 1604
      +  steps:
      +  - bash: |
     -+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
      +
     -+       sudo apt-get update
     -+       sudo apt-get install -y coccinelle
     ++       sudo apt-get update &&
     ++       sudo apt-get install -y coccinelle &&
      +
     -+       export jobname=StaticAnalysis
     ++       export jobname=StaticAnalysis &&
      +
      +       ci/run-static-analysis.sh || exit 1
      +
     -+       sudo umount "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
      +    displayName: 'ci/run-static-analysis.sh'
     ++    env:
     ++      GITFILESHAREPWD: $(gitfileshare.pwd)
      +
      +- phase: documentation
      +  displayName: Documentation
     @@ -308,18 +329,20 @@
      +    name: Hosted Ubuntu 1604
      +  steps:
      +  - bash: |
     -+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
      +
     -+       sudo apt-get update
     -+       sudo apt-get install -y asciidoc xmlto asciidoctor
     ++       sudo apt-get update &&
     ++       sudo apt-get install -y asciidoc xmlto asciidoctor &&
      +
     -+       export ALREADY_HAVE_ASCIIDOCTOR=yes.
     -+       export jobname=Documentation
     ++       export ALREADY_HAVE_ASCIIDOCTOR=yes. &&
     ++       export jobname=Documentation &&
      +
      +       ci/test-documentation.sh || exit 1
      +
     -+       sudo umount "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
      +    displayName: 'ci/test-documentation.sh'
     ++    env:
     ++      GITFILESHAREPWD: $(gitfileshare.pwd)
      
      diff --git a/ci/mount-fileshare.sh b/ci/mount-fileshare.sh
      new file mode 100755
  -:  ---------- >  7:  12d6137f8d tests: introduce `test_atexit`
  -:  ---------- >  8:  3bb226b79b git-daemon: use `test_atexit` in the tests
  -:  ---------- >  9:  3e2193a73d git-p4: use `test_atexit` to kill the daemon
  7:  942bf423a4 ! 10:  ae3c42519a tests: include detailed trace logs with --write-junit-xml upon failure
     @@ -14,9 +14,9 @@
          almost all of those cases: only when a test fails do we have a way to
          include the trace.
      
     -    So let's do something different in VSTS: let's run all the tests with
     -    `--quiet` first, and only if a failure is encountered, try to trace the
     -    commands as they are executed.
     +    So let's do something different when using Azure DevOps: let's run all
     +    the tests with `--quiet` first, and only if a failure is encountered,
     +    try to trace the commands as they are executed.
      
          Of course, we cannot turn on `--verbose-log` after the fact. So let's
          just re-run the test with all the same options, adding `--verbose-log`.
     @@ -25,15 +25,10 @@
          Note: there is an off chance that re-running the test in verbose mode
          "fixes" the failures (and this does happen from time to time!). That is
          a possibility we should be able to live with. Ideally, we would label
     -    this as "Passed upon rerun", and that outcome even exists on VSTS, but
     -    it is not available when using the JUnit XML format for now:
     -    https://github.com/Microsoft/vsts-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.cs
     -
     -    This patch contains a slightly inelegant workaround for the p4 and
     -    git-daemon tests: when we try to re-run the p4/git-daemon tests after
     -    the daemon has been started already, we need to kill said daemon. We do
     -    this by detecting the presence of the `kill_p4d` and `stop_git_daemon`
     -    shell functions and calling them if available.
     +    this as "Passed upon rerun", and Azure Pipelines even know about that
     +    outcome, but it is not available when using the JUnit XML format for
     +    now:
     +    https://github.com/Microsoft/azure-pipelines-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.cs
      
          Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
      
     @@ -60,14 +55,8 @@
       	then
      +		if test -z "$GIT_TEST_TEE_OUTPUT_FILE"
      +		then
     -+			case "$(type kill_p4d 2>/dev/null | head -n 1)" in
     -+			*function*) kill_p4d;;
     -+			esac
     -+
     -+			case "$(type stop_git_daemon 2>/dev/null |
     -+				head -n 1)" in
     -+			*function*) stop_git_daemon;;
     -+			esac
     ++			# clean up
     ++			test_atexit_handler
      +
      +			# re-run with --verbose-log
      +			echo "# Re-running: $junit_rerun_options_sq" >&2
  8:  3e83c64090 = 11:  2466a48aa3 tests: record more stderr with --write-junit-xml in case of failure
  9:  dc1d890d71 ! 12:  d112b3fe86 README: add a build badge (status of the VSTS build)
     @@ -1,6 +1,6 @@
      Author: Johannes Schindelin <johannes.schindelin@gmx.de>
      
     -    README: add a build badge (status of the VSTS build)
     +    README: add a build badge (status of the Azure Pipelines build)
      
          Just like so many other OSS projects, we now also have a build badge.
      
     @@ -10,7 +10,7 @@
      --- a/README.md
      +++ b/README.md
      @@
     -+[![Build Status](https://git.visualstudio.com/git/_apis/build/status/test-git.git)](https://git.visualstudio.com/git/_build/latest?definitionId=2)
     ++[![Build Status](https:/dev.azure.com/git/git/_apis/build/status/test-git.git)](https://dev.azure.com/git/git/_build/latest?definitionId=2)
      +
       Git - fast, scalable, distributed revision control system
       =========================================================
  -:  ---------- > 13:  0a53f37135 travis: fix skipping tagged releases

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH v2 01/13] ci: rename the library of common functions
  2018-10-15 10:11 ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Johannes Schindelin via GitGitGadget
@ 2018-10-15 10:11   ` Johannes Schindelin via GitGitGadget
  2018-10-16  5:26     ` Junio C Hamano
  2018-10-15 10:12   ` [PATCH v2 02/13] ci/lib.sh: encapsulate Travis-specific things Johannes Schindelin via GitGitGadget
                     ` (13 subsequent siblings)
  14 siblings, 1 reply; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-10-15 10:11 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The name is hard-coded to reflect that we use Travis CI for continuous
testing.

In the next commits, we will extend this to be able use Azure DevOps,
too.

So let's adjust the name to make it more generic.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 ci/install-dependencies.sh     | 2 +-
 ci/{lib-travisci.sh => lib.sh} | 0
 ci/print-test-failures.sh      | 2 +-
 ci/run-build-and-tests.sh      | 2 +-
 ci/run-linux32-docker.sh       | 2 +-
 ci/run-static-analysis.sh      | 2 +-
 ci/run-windows-build.sh        | 2 +-
 ci/test-documentation.sh       | 2 +-
 8 files changed, 7 insertions(+), 7 deletions(-)
 rename ci/{lib-travisci.sh => lib.sh} (100%)

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 75a9fd2475..961064658e 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -3,7 +3,7 @@
 # Install dependencies required to build and test Git on Linux and macOS
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
 LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
diff --git a/ci/lib-travisci.sh b/ci/lib.sh
similarity index 100%
rename from ci/lib-travisci.sh
rename to ci/lib.sh
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index d55460a212..7aef39a2fd 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -3,7 +3,7 @@
 # Print output of failing tests
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 # Tracing executed commands would produce too much noise in the loop below.
 set +x
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 2a5bff4a1c..e28ac2fb9a 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -3,7 +3,7 @@
 # Build and test Git
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 ln -s "$cache_dir/.prove" t/.prove
 
diff --git a/ci/run-linux32-docker.sh b/ci/run-linux32-docker.sh
index 21637903ce..751acfcf8a 100755
--- a/ci/run-linux32-docker.sh
+++ b/ci/run-linux32-docker.sh
@@ -3,7 +3,7 @@
 # Download and run Docker image to build and test 32-bit Git
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 docker pull daald/ubuntu32:xenial
 
diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh
index 5688f261d0..dc189c7456 100755
--- a/ci/run-static-analysis.sh
+++ b/ci/run-static-analysis.sh
@@ -3,7 +3,7 @@
 # Perform various static code analysis checks
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 make --jobs=2 coccicheck
 
diff --git a/ci/run-windows-build.sh b/ci/run-windows-build.sh
index d99a180e52..a73a4eca0a 100755
--- a/ci/run-windows-build.sh
+++ b/ci/run-windows-build.sh
@@ -6,7 +6,7 @@
 # supported) and a commit hash.
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 test $# -ne 2 && echo "Unexpected number of parameters" && exit 1
 test -z "$GFW_CI_TOKEN" && echo "GFW_CI_TOKEN not defined" && exit
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index a20de9ca12..d3cdbac73f 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -3,7 +3,7 @@
 # Perform sanity checks on documentation and build it.
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 gem install asciidoctor
 
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH v2 02/13] ci/lib.sh: encapsulate Travis-specific things
  2018-10-15 10:11 ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Johannes Schindelin via GitGitGadget
  2018-10-15 10:11   ` [PATCH v2 01/13] ci: rename the library of common functions Johannes Schindelin via GitGitGadget
@ 2018-10-15 10:12   ` Johannes Schindelin via GitGitGadget
  2018-10-18 22:01     ` SZEDER Gábor
  2018-10-15 10:12   ` [PATCH v2 03/13] test-date: add a subcommand to measure times in shell scripts Johannes Schindelin via GitGitGadget
                     ` (12 subsequent siblings)
  14 siblings, 1 reply; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-10-15 10:12 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The upcoming patches will allow building git.git via Azure Pipelines
(i.e. Azure DevOps' Continuous Integration), where variable names and
URLs look a bit different than in Travis CI.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 ci/install-dependencies.sh |  3 ++-
 ci/lib.sh                  | 44 +++++++++++++++++++++++++++-----------
 ci/print-test-failures.sh  |  2 +-
 ci/test-documentation.sh   |  1 +
 4 files changed, 35 insertions(+), 15 deletions(-)

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 961064658e..63fa37f68f 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -28,7 +28,8 @@ osx-clang|osx-gcc)
 	brew update --quiet
 	# Uncomment this if you want to run perf tests:
 	# brew install gnu-time
-	brew install git-lfs gettext
+	test -z "$BREW_INSTALL_PACKAGES" ||
+	brew install $BREW_INSTALL_PACKAGES
 	brew link --force gettext
 	brew install caskroom/cask/perforce
 	;;
diff --git a/ci/lib.sh b/ci/lib.sh
index 06970f7213..8532555b4e 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -1,5 +1,26 @@
 # Library of functions shared by all CI scripts
 
+if test true = "$TRAVIS"
+then
+	# We are running within Travis CI
+	CI_BRANCH="$TRAVIS_BRANCH"
+	CI_COMMIT="$TRAVIS_COMMIT"
+	CI_JOB_ID="$TRAVIS_JOB_ID"
+	CI_JOB_NUMBER="$TRAVIS_JOB_NUMBER"
+	CI_OS_NAME="$TRAVIS_OS_NAME"
+	CI_REPO_SLUG="$TRAVIS_REPO_SLUG"
+
+	cache_dir="$HOME/travis-cache"
+
+	url_for_job_id () {
+		echo "https://travis-ci.org/$CI_REPO_SLUG/jobs/$1"
+	}
+
+	BREW_INSTALL_PACKAGES="git-lfs gettext"
+	export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
+	export GIT_TEST_OPTS="--verbose-log -x --immediate"
+fi
+
 skip_branch_tip_with_tag () {
 	# Sometimes, a branch is pushed at the same time the tag that points
 	# at the same commit as the tip of the branch is pushed, and building
@@ -13,10 +34,10 @@ skip_branch_tip_with_tag () {
 	# we can skip the build because we won't be skipping a build
 	# of a tag.
 
-	if TAG=$(git describe --exact-match "$TRAVIS_BRANCH" 2>/dev/null) &&
-		test "$TAG" != "$TRAVIS_BRANCH"
+	if TAG=$(git describe --exact-match "$CI_BRANCH" 2>/dev/null) &&
+		test "$TAG" != "$CI_BRANCH"
 	then
-		echo "$(tput setaf 2)Tip of $TRAVIS_BRANCH is exactly at $TAG$(tput sgr0)"
+		echo "$(tput setaf 2)Tip of $CI_BRANCH is exactly at $TAG$(tput sgr0)"
 		exit 0
 	fi
 }
@@ -25,7 +46,7 @@ skip_branch_tip_with_tag () {
 # job if we encounter the same tree again and can provide a useful info
 # message.
 save_good_tree () {
-	echo "$(git rev-parse $TRAVIS_COMMIT^{tree}) $TRAVIS_COMMIT $TRAVIS_JOB_NUMBER $TRAVIS_JOB_ID" >>"$good_trees_file"
+	echo "$(git rev-parse $CI_COMMIT^{tree}) $CI_COMMIT $CI_JOB_NUMBER $CI_JOB_ID" >>"$good_trees_file"
 	# limit the file size
 	tail -1000 "$good_trees_file" >"$good_trees_file".tmp
 	mv "$good_trees_file".tmp "$good_trees_file"
@@ -35,7 +56,7 @@ save_good_tree () {
 # successfully before (e.g. because the branch got rebased, changing only
 # the commit messages).
 skip_good_tree () {
-	if ! good_tree_info="$(grep "^$(git rev-parse $TRAVIS_COMMIT^{tree}) " "$good_trees_file")"
+	if ! good_tree_info="$(grep "^$(git rev-parse $CI_COMMIT^{tree}) " "$good_trees_file")"
 	then
 		# Haven't seen this tree yet, or no cached good trees file yet.
 		# Continue the build job.
@@ -45,18 +66,18 @@ skip_good_tree () {
 	echo "$good_tree_info" | {
 		read tree prev_good_commit prev_good_job_number prev_good_job_id
 
-		if test "$TRAVIS_JOB_ID" = "$prev_good_job_id"
+		if test "$CI_JOB_ID" = "$prev_good_job_id"
 		then
 			cat <<-EOF
-			$(tput setaf 2)Skipping build job for commit $TRAVIS_COMMIT.$(tput sgr0)
+			$(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0)
 			This commit has already been built and tested successfully by this build job.
 			To force a re-build delete the branch's cache and then hit 'Restart job'.
 			EOF
 		else
 			cat <<-EOF
-			$(tput setaf 2)Skipping build job for commit $TRAVIS_COMMIT.$(tput sgr0)
+			$(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0)
 			This commit's tree has already been built and tested successfully in build job $prev_good_job_number for commit $prev_good_commit.
-			The log of that build job is available at https://travis-ci.org/$TRAVIS_REPO_SLUG/jobs/$prev_good_job_id
+			The log of that build job is available at $(url_for_job_id $prev_good_job_id)
 			To force a re-build delete the branch's cache and then hit 'Restart job'.
 			EOF
 		fi
@@ -81,7 +102,6 @@ check_unignored_build_artifacts ()
 # and installing dependencies.
 set -ex
 
-cache_dir="$HOME/travis-cache"
 good_trees_file="$cache_dir/good-trees"
 
 mkdir -p "$cache_dir"
@@ -91,13 +111,11 @@ skip_good_tree
 
 if test -z "$jobname"
 then
-	jobname="$TRAVIS_OS_NAME-$CC"
+	jobname="$CI_OS_NAME-$CC"
 fi
 
 export DEVELOPER=1
 export DEFAULT_TEST_TARGET=prove
-export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
-export GIT_TEST_OPTS="--verbose-log -x --immediate"
 export GIT_TEST_CLONE_2GB=YesPlease
 if [ "$jobname" = linux-gcc ]; then
 	export CC=gcc-8
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 7aef39a2fd..d2045b63a6 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -69,7 +69,7 @@ do
 	fi
 done
 
-if [ $combined_trash_size -gt 0 ]
+if [ -n "$TRAVIS_JOB_ID" -a $combined_trash_size -gt 0 ]
 then
 	echo "------------------------------------------------------------------------"
 	echo "Trash directories embedded in this log can be extracted by running:"
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index d3cdbac73f..7d0beb2832 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -5,6 +5,7 @@
 
 . ${0%/*}/lib.sh
 
+test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
 gem install asciidoctor
 
 make check-builtins
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH v2 03/13] test-date: add a subcommand to measure times in shell scripts
  2018-10-15 10:11 ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Johannes Schindelin via GitGitGadget
  2018-10-15 10:11   ` [PATCH v2 01/13] ci: rename the library of common functions Johannes Schindelin via GitGitGadget
  2018-10-15 10:12   ` [PATCH v2 02/13] ci/lib.sh: encapsulate Travis-specific things Johannes Schindelin via GitGitGadget
@ 2018-10-15 10:12   ` Johannes Schindelin via GitGitGadget
  2018-10-15 10:12   ` [PATCH v2 04/13] tests: optionally write results as JUnit-style .xml Johannes Schindelin via GitGitGadget
                     ` (11 subsequent siblings)
  14 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-10-15 10:12 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

In the next commit, we want to teach Git's test suite to optionally
output test results in JUnit-style .xml files. These files contain
information about the time spent. So we need a way to measure time.

While we could use `date +%s` for that, this will give us only seconds,
i.e. very coarse-grained timings.

GNU `date` supports `date +%s.%N` (i.e. nanosecond-precision output),
but there is no equivalent in BSD `date` (read: on macOS, we would not
be able to obtain precise timings).

So let's introduce `test-tool date getnanos`, with an optional start
time, that outputs preciser values.

Granted, it is a bit pointless to try measuring times accurately in
shell scripts, certainly to nanosecond precision. But it is better than
second-granularity.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/helper/test-date.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/t/helper/test-date.c b/t/helper/test-date.c
index a0837371ab..792a805374 100644
--- a/t/helper/test-date.c
+++ b/t/helper/test-date.c
@@ -7,6 +7,7 @@ static const char *usage_msg = "\n"
 "  test-tool date parse [date]...\n"
 "  test-tool date approxidate [date]...\n"
 "  test-tool date timestamp [date]...\n"
+"  test-tool date getnanos [start-nanos]\n"
 "  test-tool date is64bit\n"
 "  test-tool date time_t-is64bit\n";
 
@@ -82,6 +83,15 @@ static void parse_approx_timestamp(const char **argv, struct timeval *now)
 	}
 }
 
+static void getnanos(const char **argv, struct timeval *now)
+{
+	double seconds = getnanotime() / 1.0e9;
+
+	if (*argv)
+		seconds -= strtod(*argv, NULL);
+	printf("%lf\n", seconds);
+}
+
 int cmd__date(int argc, const char **argv)
 {
 	struct timeval now;
@@ -108,6 +118,8 @@ int cmd__date(int argc, const char **argv)
 		parse_approxidate(argv+1, &now);
 	else if (!strcmp(*argv, "timestamp"))
 		parse_approx_timestamp(argv+1, &now);
+	else if (!strcmp(*argv, "getnanos"))
+		getnanos(argv+1, &now);
 	else if (!strcmp(*argv, "is64bit"))
 		return sizeof(timestamp_t) == 8 ? 0 : 1;
 	else if (!strcmp(*argv, "time_t-is64bit"))
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH v2 04/13] tests: optionally write results as JUnit-style .xml
  2018-10-15 10:11 ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Johannes Schindelin via GitGitGadget
                     ` (2 preceding siblings ...)
  2018-10-15 10:12   ` [PATCH v2 03/13] test-date: add a subcommand to measure times in shell scripts Johannes Schindelin via GitGitGadget
@ 2018-10-15 10:12   ` Johannes Schindelin via GitGitGadget
  2018-10-15 10:12   ` [PATCH v2 05/13] ci/lib.sh: add support for Azure Pipelines Johannes Schindelin via GitGitGadget
                     ` (10 subsequent siblings)
  14 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-10-15 10:12 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

This will come in handy when publishing the results of Git's test suite
during an automated Azure DevOps run.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/.gitignore  |  1 +
 t/test-lib.sh | 98 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 99 insertions(+)

diff --git a/t/.gitignore b/t/.gitignore
index 348715f0e4..91cf5772fe 100644
--- a/t/.gitignore
+++ b/t/.gitignore
@@ -2,3 +2,4 @@
 /test-results
 /.prove
 /chainlinttmp
+/out/
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 3f95bfda60..7ed0013f6d 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -288,6 +288,9 @@ do
 	--verbose-log)
 		verbose_log=t
 		shift ;;
+	--write-junit-xml)
+		write_junit_xml=t
+		shift ;;
 	*)
 		echo "error: unknown test option '$1'" >&2; exit 1 ;;
 	esac
@@ -431,11 +434,24 @@ trap 'exit $?' INT
 # the test_expect_* functions instead.
 
 test_ok_ () {
+	if test -n "$write_junit_xml"
+	then
+		write_junit_xml_testcase "$*"
+	fi
 	test_success=$(($test_success + 1))
 	say_color "" "ok $test_count - $@"
 }
 
 test_failure_ () {
+	if test -n "$write_junit_xml"
+	then
+		junit_insert="<failure message=\"not ok $test_count -"
+		junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
+		junit_insert="$junit_insert $(xml_attr_encode \
+			"$(printf '%s\n' "$@" | sed 1d)")"
+		junit_insert="$junit_insert</failure>"
+		write_junit_xml_testcase "$1" "      $junit_insert"
+	fi
 	test_failure=$(($test_failure + 1))
 	say_color error "not ok $test_count - $1"
 	shift
@@ -444,11 +460,19 @@ test_failure_ () {
 }
 
 test_known_broken_ok_ () {
+	if test -n "$write_junit_xml"
+	then
+		write_junit_xml_testcase "$* (breakage fixed)"
+	fi
 	test_fixed=$(($test_fixed+1))
 	say_color error "ok $test_count - $@ # TODO known breakage vanished"
 }
 
 test_known_broken_failure_ () {
+	if test -n "$write_junit_xml"
+	then
+		write_junit_xml_testcase "$* (known breakage)"
+	fi
 	test_broken=$(($test_broken+1))
 	say_color warn "not ok $test_count - $@ # TODO known breakage"
 }
@@ -706,6 +730,10 @@ test_start_ () {
 	test_count=$(($test_count+1))
 	maybe_setup_verbose
 	maybe_setup_valgrind
+	if test -n "$write_junit_xml"
+	then
+		junit_start=$(test-tool date getnanos)
+	fi
 }
 
 test_finish_ () {
@@ -743,6 +771,13 @@ test_skip () {
 
 	case "$to_skip" in
 	t)
+		if test -n "$write_junit_xml"
+		then
+			message="$(xml_attr_encode "$skipped_reason")"
+			write_junit_xml_testcase "$1" \
+				"      <skipped message=\"$message\" />"
+		fi
+
 		say_color skip >&3 "skipping test: $@"
 		say_color skip "ok $test_count # skip $1 ($skipped_reason)"
 		: true
@@ -758,9 +793,58 @@ test_at_end_hook_ () {
 	:
 }
 
+write_junit_xml () {
+	case "$1" in
+	--truncate)
+		>"$junit_xml_path"
+		junit_have_testcase=
+		shift
+		;;
+	esac
+	printf '%s\n' "$@" >>"$junit_xml_path"
+}
+
+xml_attr_encode () {
+	# We do not translate CR to &#x0d; because BSD sed does not handle
+	# \r in the regex. In practice, the output should not even have any
+	# carriage returns.
+	printf '%s\n' "$@" |
+	sed -e 's/&/\&amp;/g' -e "s/'/\&apos;/g" -e 's/"/\&quot;/g' \
+		-e 's/</\&lt;/g' -e 's/>/\&gt;/g' \
+		-e 's/	/\&#x09;/g' -e 's/$/\&#x0a;/' -e '$s/&#x0a;$//' |
+	tr -d '\012\015'
+}
+
+write_junit_xml_testcase () {
+	junit_attrs="name=\"$(xml_attr_encode "$this_test.$test_count $1")\""
+	shift
+	junit_attrs="$junit_attrs classname=\"$this_test\""
+	junit_attrs="$junit_attrs time=\"$(test-tool \
+		date getnanos $junit_start)\""
+	write_junit_xml "$(printf '%s\n' \
+		"    <testcase $junit_attrs>" "$@" "    </testcase>")"
+	junit_have_testcase=t
+}
+
 test_done () {
 	GIT_EXIT_OK=t
 
+	if test -n "$write_junit_xml" && test -n "$junit_xml_path"
+	then
+		test -n "$junit_have_testcase" || {
+			junit_start=$(test-tool date getnanos)
+			write_junit_xml_testcase "all tests skipped"
+		}
+
+		# adjust the overall time
+		junit_time=$(test-tool date getnanos $junit_suite_start)
+		sed "s/<testsuite [^>]*/& time=\"$junit_time\"/" \
+			<"$junit_xml_path" >"$junit_xml_path.new"
+		mv "$junit_xml_path.new" "$junit_xml_path"
+
+		write_junit_xml "  </testsuite>" "</testsuites>"
+	fi
+
 	if test -z "$HARNESS_ACTIVE"
 	then
 		test_results_dir="$TEST_OUTPUT_DIRECTORY/test-results"
@@ -996,6 +1080,7 @@ then
 else
 	mkdir -p "$TRASH_DIRECTORY"
 fi
+
 # Use -P to resolve symlinks in our working directory so that the cwd
 # in subprocesses like git equals our $PWD (for pathname comparisons).
 cd -P "$TRASH_DIRECTORY" || exit 1
@@ -1009,6 +1094,19 @@ then
 	test_done
 fi
 
+if test -n "$write_junit_xml"
+then
+	junit_xml_dir="$TEST_OUTPUT_DIRECTORY/out"
+	mkdir -p "$junit_xml_dir"
+	junit_xml_base=${0##*/}
+	junit_xml_path="$junit_xml_dir/TEST-${junit_xml_base%.sh}.xml"
+	junit_attrs="name=\"${junit_xml_base%.sh}\""
+	junit_attrs="$junit_attrs timestamp=\"$(TZ=UTC \
+		date +%Y-%m-%dT%H:%M:%S)\""
+	write_junit_xml --truncate "<testsuites>" "  <testsuite $junit_attrs>"
+	junit_suite_start=$(test-tool date getnanos)
+fi
+
 # Provide an implementation of the 'yes' utility
 yes () {
 	if test $# = 0
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH v2 05/13] ci/lib.sh: add support for Azure Pipelines
  2018-10-15 10:11 ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Johannes Schindelin via GitGitGadget
                     ` (3 preceding siblings ...)
  2018-10-15 10:12   ` [PATCH v2 04/13] tests: optionally write results as JUnit-style .xml Johannes Schindelin via GitGitGadget
@ 2018-10-15 10:12   ` Johannes Schindelin via GitGitGadget
  2018-10-16  9:43     ` SZEDER Gábor
  2018-10-15 10:12   ` [PATCH v2 06/13] Add a build definition for Azure DevOps Johannes Schindelin via GitGitGadget
                     ` (9 subsequent siblings)
  14 siblings, 1 reply; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-10-15 10:12 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

This patch introduces a conditional arm that defines some environment
variables and a function that displays the URL given the job id (to
identify previous runs for known-good trees).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 ci/lib.sh | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/ci/lib.sh b/ci/lib.sh
index 8532555b4e..584abcd529 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -19,6 +19,29 @@ then
 	BREW_INSTALL_PACKAGES="git-lfs gettext"
 	export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --immediate"
+elif test -n "$SYSTEM_TASKDEFINITIONSURI"
+then
+	# We are running in Azure Pipelines
+	CI_BRANCH="$BUILD_SOURCEBRANCH"
+	CI_COMMIT="$BUILD_SOURCEVERSION"
+	CI_JOB_ID="$BUILD_BUILDID"
+	CI_JOB_NUMBER="$BUILD_BUILDNUMBER"
+	CI_OS_NAME="$(echo "$AGENT_OS" | tr A-Z a-z)"
+	test darwin != "$CI_OS_NAME" || CI_OS_NAME=osx
+	CI_REPO_SLUG="$(expr "$BUILD_REPOSITORY_URI" : '.*/\([^/]*/[^/]*\)$')"
+	CC="${CC:-gcc}"
+
+	# use a subdirectory of the cache dir (because the file share is shared
+	# among *all* phases)
+	cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME"
+
+	url_for_job_id () {
+		echo "$SYSTEM_TASKDEFINITIONSURI$SYSTEM_TEAMPROJECT/_build/results?buildId=$1"
+	}
+
+	BREW_INSTALL_PACKAGES=
+	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
+	export GIT_TEST_OPTS="--quiet --write-junit-xml"
 fi
 
 skip_branch_tip_with_tag () {
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH v2 06/13] Add a build definition for Azure DevOps
  2018-10-15 10:11 ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Johannes Schindelin via GitGitGadget
                     ` (4 preceding siblings ...)
  2018-10-15 10:12   ` [PATCH v2 05/13] ci/lib.sh: add support for Azure Pipelines Johannes Schindelin via GitGitGadget
@ 2018-10-15 10:12   ` Johannes Schindelin via GitGitGadget
  2018-10-16  5:35     ` Junio C Hamano
  2018-10-16 19:12     ` SZEDER Gábor
  2018-10-15 10:12   ` [PATCH v2 07/13] tests: introduce `test_atexit` Johannes Schindelin via GitGitGadget
                     ` (8 subsequent siblings)
  14 siblings, 2 replies; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-10-15 10:12 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

This commit adds an azure-pipelines.yml file which is Azure DevOps'
equivalent to Travis CI's .travis.yml.

To make things a bit easier to understand, we refrain from using the
`matrix` feature here because (while it is powerful) it can be a bit
confusing to users who are not familiar with CI setups. Therefore, we
use a separate phase even for similar configurations (such as GCC vs
Clang on Linux, GCC vs Clang on macOS).

Also, we make use of the shiny new feature we just introduced where the
test suite can output JUnit-style .xml files. This information is made
available in a nice UI that allows the viewer to filter by phase and/or
test number, and to see trends such as: number of (failing) tests, time
spent running the test suite, etc.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 azure-pipelines.yml   | 319 ++++++++++++++++++++++++++++++++++++++++++
 ci/mount-fileshare.sh |  26 ++++
 2 files changed, 345 insertions(+)
 create mode 100644 azure-pipelines.yml
 create mode 100755 ci/mount-fileshare.sh

diff --git a/azure-pipelines.yml b/azure-pipelines.yml
new file mode 100644
index 0000000000..b5749121d2
--- /dev/null
+++ b/azure-pipelines.yml
@@ -0,0 +1,319 @@
+resources:
+- repo: self
+  fetchDepth: 1
+
+phases:
+- phase: linux_clang
+  displayName: linux-clang
+  condition: succeeded()
+  queue:
+    name: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       sudo apt-get update &&
+       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin &&
+
+       export CC=clang || exit 1
+
+       ci/install-dependencies.sh
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'linux-clang'
+      platform: Linux
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+
+- phase: linux_gcc
+  displayName: linux-gcc
+  condition: succeeded()
+  queue:
+    name: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       sudo apt-get update &&
+       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin || exit 1
+
+       ci/install-dependencies.sh
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'linux-gcc'
+      platform: Linux
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+
+- phase: osx_clang
+  displayName: osx-clang
+  condition: succeeded()
+  queue:
+    name: Hosted macOS
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       export CC=clang
+
+       ci/install-dependencies.sh
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'osx-clang'
+      platform: macOS
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+
+- phase: osx_gcc
+  displayName: osx-gcc
+  condition: succeeded()
+  queue:
+    name: Hosted macOS
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       ci/install-dependencies.sh
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'osx-gcc'
+      platform: macOS
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+
+- phase: gettext_poison
+  displayName: GETTEXT_POISON
+  condition: succeeded()
+  queue:
+    name: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       sudo apt-get update &&
+       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev &&
+
+       export jobname=GETTEXT_POISON || exit 1
+
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'gettext-poison'
+      platform: Linux
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+
+- phase: windows
+  displayName: Windows
+  condition: succeeded()
+  queue:
+    name: Hosted VS2017
+    timeoutInMinutes: 240
+  steps:
+  - powershell: |
+       # Helper to check the error level of the latest command (exit with error when appropriate)
+       function c() { if (!$?) { exit(1) } }
+
+       if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") {
+         net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no; c
+         cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\; c
+       }
+
+       # Add build agent's MinGit to PATH
+       $env:PATH = $env:AGENT_HOMEDIRECTORY +"\externals\\git\cmd;" +$env:PATH
+
+       # Helper to initialize (or update) a Git worktree
+       function init ($path, $url, $set_origin) {
+           if (Test-Path $path) {
+               cd $path; c
+               if (Test-Path .git) {
+                   git init; c
+               } else {
+                   git status
+               }
+           } else {
+               git init $path; c
+               cd $path; c
+           }
+           git config core.autocrlf false; c
+           git config core.untrackedCache true; c
+           if (($set_origin -ne 0) -and !(git config remote.origin.url)) {
+               git remote add origin $url; c
+           }
+           git fetch --depth=1 $url master; c
+           git reset --hard FETCH_HEAD; c
+           git clean -df; c
+       }
+
+       # Initialize Git for Windows' SDK
+       $sdk_path = "$(Build.SourcesDirectory)\git-sdk-64"
+       init "$sdk_path" "https://dev.azure.com/git-for-windows/git-sdk-64/_git/git-sdk-64" 0
+       init usr\src\build-extra https://github.com/git-for-windows/build-extra 1
+
+       cd "$(Build.SourcesDirectory)"; c
+
+       $env:HOME = "$(Build.SourcesDirectory)"
+       $env:MSYSTEM = "MINGW64"
+       git-sdk-64\git-cmd --command=usr\\bin\\bash.exe -lc @"
+         . ci/lib.sh
+
+         make -j10 DEVELOPER=1 NO_PERL=1 || exit 1
+         NO_PERL=1 NO_SVN_TESTS=1 GIT_TEST_OPTS=\"--quiet --write-junit-xml\" time make -j15 -k DEVELOPER=1 test || {
+           NO_PERL=1 NO_SVN_TESTS=1 GIT_TEST_OPTS=\"-i -v -x\" make -k -C t failed; exit 1
+         }
+
+         save_good_tree
+       "@
+       c
+
+       if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") {
+         cmd /c rmdir "$(Build.SourcesDirectory)\test-cache"
+       }
+    displayName: 'build & test'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'windows'
+      platform: Windows
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+
+- phase: linux32
+  displayName: Linux32
+  condition: succeeded()
+  queue:
+    name: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       sudo apt-get update &&
+       sudo apt-get -y install \
+           apt-transport-https \
+           ca-certificates \
+           curl \
+           software-properties-common &&
+       curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - &&
+       sudo add-apt-repository \
+          "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
+          $(lsb_release -cs) \
+          stable" &&
+       sudo apt-get update &&
+       sudo apt-get -y install docker-ce &&
+
+       sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS=-j3 bash -lxc ci/run-linux32-docker.sh || exit 1
+
+       sudo chmod a+r t/out/TEST-*.xml
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/run-linux32-docker.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'linux32'
+      platform: Linux
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+
+- phase: static_analysis
+  displayName: StaticAnalysis
+  condition: succeeded()
+  queue:
+    name: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       sudo apt-get update &&
+       sudo apt-get install -y coccinelle &&
+
+       export jobname=StaticAnalysis &&
+
+       ci/run-static-analysis.sh || exit 1
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/run-static-analysis.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+
+- phase: documentation
+  displayName: Documentation
+  condition: succeeded()
+  queue:
+    name: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       sudo apt-get update &&
+       sudo apt-get install -y asciidoc xmlto asciidoctor &&
+
+       export ALREADY_HAVE_ASCIIDOCTOR=yes. &&
+       export jobname=Documentation &&
+
+       ci/test-documentation.sh || exit 1
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/test-documentation.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
diff --git a/ci/mount-fileshare.sh b/ci/mount-fileshare.sh
new file mode 100755
index 0000000000..5fb5f74b70
--- /dev/null
+++ b/ci/mount-fileshare.sh
@@ -0,0 +1,26 @@
+#!/bin/sh
+
+die () {
+	echo "$*" >&2
+	exit 1
+}
+
+test $# = 4 ||
+die "Usage: $0 <share> <username> <password> <mountpoint"
+
+mkdir -p "$4" || die "Could not create $4"
+
+case "$(uname -s)" in
+Linux)
+	sudo mount -t cifs -o vers=3.0,username="$2",password="$3",dir_mode=0777,file_mode=0777,serverino "$1" "$4"
+	;;
+Darwin)
+	pass="$(echo "$3" | sed -e 's/\//%2F/g' -e 's/+/%2B/g')" &&
+	mount -t smbfs,soft "smb://$2:$pass@${1#//}" "$4"
+	;;
+*)
+	die "No support for $(uname -s)"
+	;;
+esac ||
+die "Could not mount $4"
+
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH v2 07/13] tests: introduce `test_atexit`
  2018-10-15 10:11 ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Johannes Schindelin via GitGitGadget
                     ` (5 preceding siblings ...)
  2018-10-15 10:12   ` [PATCH v2 06/13] Add a build definition for Azure DevOps Johannes Schindelin via GitGitGadget
@ 2018-10-15 10:12   ` Johannes Schindelin via GitGitGadget
  2018-10-21 11:20     ` SZEDER Gábor
  2018-10-15 10:12   ` [PATCH v2 08/13] git-daemon: use `test_atexit` in the tests Johannes Schindelin via GitGitGadget
                     ` (7 subsequent siblings)
  14 siblings, 1 reply; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-10-15 10:12 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

When running the p4 daemon or `git daemon`, we want to kill it at the
end of the test script.

So far, we do this "manually".

However, in the next few commits we want to teach the test suite to
optionally re-run scripts with different options, therefore we will have
to have a consistent way to stop daemons.

Let's introduce `test_atexit`, which is loosely modeled after
`test_when_finished` (but has a broader scope: rather than running the
commands after the current test case, run them when the test script
finishes, and also run them when the `--immediate` option is in effect).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t0000-basic.sh        | 18 ++++++++++++++++++
 t/test-lib-functions.sh | 29 +++++++++++++++++++++++++++++
 t/test-lib.sh           |  4 ++++
 3 files changed, 51 insertions(+)

diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index 391f910c6a..8c5faa6ce1 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -821,6 +821,24 @@ test_expect_success 'tests clean up even on failures' "
 	EOF
 "
 
+test_expect_success 'test_atexit is run' "
+	test_must_fail run_sub_test_lib_test \
+		atexit-cleanup 'Run atexit commands' -i <<-\\EOF &&
+	test_expect_success 'tests clean up even after a failure' '
+		> ../../clean-atexit &&
+		test_atexit rm ../../clean-atexit &&
+		> ../../also-clean-atexit &&
+		test_atexit rm ../../also-clean-atexit &&
+		> ../../dont-clean-atexit &&
+		(exit 1)
+	'
+	test_done
+	EOF
+	test_path_exists dont-clean-atexit &&
+	test_path_is_missing clean-atexit &&
+	test_path_is_missing also-clean-atexit
+"
+
 test_expect_success 'test_oid setup' '
 	test_oid_init
 '
diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index 78d8c3783b..d7dd0c1be9 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -891,6 +891,35 @@ test_when_finished () {
 		} && (exit \"\$eval_ret\"); eval_ret=\$?; $test_cleanup"
 }
 
+# This function can be used to schedule some commands to be run
+# unconditionally at the end of the test script, e.g. to stop a daemon:
+#
+#	test_expect_success 'test git daemon' '
+#		git daemon &
+#		daemon_pid=$! &&
+#		test_atexit "kill $daemon_pid" &&
+#		hello world
+#	'
+
+test_atexit () {
+	# We cannot detect when we are in a subshell in general, but by
+	# doing so on Bash is better than nothing (the test will
+	# silently pass on other shells).
+	test "${BASH_SUBSHELL-0}" = 0 ||
+	error "bug in test script: test_atexit does nothing in a subshell"
+	test_atexit_cleanup="{ $*
+		} && (exit \"\$eval_ret\"); eval_ret=\$?; $test_atexit_cleanup"
+}
+
+test_atexit_handler () {
+	test : != "$test_atexit_cleanup" || return 0
+
+	setup_malloc_check
+	test_eval_ "$test_atexit_cleanup"
+	test_atexit_cleanup=:
+	teardown_malloc_check
+}
+
 # Most tests can use the created repository, but some may need to create more.
 # Usage: test_create_repo <directory>
 test_create_repo () {
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 7ed0013f6d..6f9c1f5300 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -413,6 +413,7 @@ test_external_has_tap=0
 
 die () {
 	code=$?
+	test_atexit_handler || code=$?
 	if test -n "$GIT_EXIT_OK"
 	then
 		exit $code
@@ -826,9 +827,12 @@ write_junit_xml_testcase () {
 	junit_have_testcase=t
 }
 
+test_atexit_cleanup=:
 test_done () {
 	GIT_EXIT_OK=t
 
+	test -n "$immediate" || test_atexit_handler
+
 	if test -n "$write_junit_xml" && test -n "$junit_xml_path"
 	then
 		test -n "$junit_have_testcase" || {
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH v2 08/13] git-daemon: use `test_atexit` in the tests
  2018-10-15 10:11 ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Johannes Schindelin via GitGitGadget
                     ` (6 preceding siblings ...)
  2018-10-15 10:12   ` [PATCH v2 07/13] tests: introduce `test_atexit` Johannes Schindelin via GitGitGadget
@ 2018-10-15 10:12   ` Johannes Schindelin via GitGitGadget
  2018-10-15 10:12   ` [PATCH v2 09/13] git-p4: use `test_atexit` to kill the daemon Johannes Schindelin via GitGitGadget
                     ` (6 subsequent siblings)
  14 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-10-15 10:12 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

This makes use of the just-introduced consistent way to specify that a
long-running process needs to be terminated at the end of a test script
run.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/interop/i5500-git-daemon.sh | 1 -
 t/lib-git-daemon.sh           | 3 +--
 t/t5570-git-daemon.sh         | 1 -
 3 files changed, 1 insertion(+), 4 deletions(-)

diff --git a/t/interop/i5500-git-daemon.sh b/t/interop/i5500-git-daemon.sh
index 1daf69420b..4d22e42f84 100755
--- a/t/interop/i5500-git-daemon.sh
+++ b/t/interop/i5500-git-daemon.sh
@@ -37,5 +37,4 @@ test_expect_success "fetch with $VERSION_B" '
 	test_cmp expect actual
 '
 
-stop_git_daemon
 test_done
diff --git a/t/lib-git-daemon.sh b/t/lib-git-daemon.sh
index edbea2d986..a896af2284 100644
--- a/t/lib-git-daemon.sh
+++ b/t/lib-git-daemon.sh
@@ -13,7 +13,6 @@
 #
 #	test_expect_success ...
 #
-#	stop_git_daemon
 #	test_done
 
 test_tristate GIT_TEST_GIT_DAEMON
@@ -43,7 +42,7 @@ start_git_daemon() {
 
 	mkdir -p "$GIT_DAEMON_DOCUMENT_ROOT_PATH"
 
-	trap 'code=$?; stop_git_daemon; (exit $code); die' EXIT
+	test_atexit 'stop_git_daemon'
 
 	say >&3 "Starting git daemon ..."
 	mkfifo git_daemon_output
diff --git a/t/t5570-git-daemon.sh b/t/t5570-git-daemon.sh
index 7466aad111..08f95c80a2 100755
--- a/t/t5570-git-daemon.sh
+++ b/t/t5570-git-daemon.sh
@@ -211,5 +211,4 @@ test_expect_success FAKENC 'hostname interpolation works after LF-stripping' '
 	test_cmp expect actual
 '
 
-stop_git_daemon
 test_done
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH v2 09/13] git-p4: use `test_atexit` to kill the daemon
  2018-10-15 10:11 ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Johannes Schindelin via GitGitGadget
                     ` (7 preceding siblings ...)
  2018-10-15 10:12   ` [PATCH v2 08/13] git-daemon: use `test_atexit` in the tests Johannes Schindelin via GitGitGadget
@ 2018-10-15 10:12   ` Johannes Schindelin via GitGitGadget
  2018-10-15 11:00     ` Luke Diamand
                       ` (2 more replies)
  2018-10-15 10:12   ` [PATCH v2 10/13] tests: include detailed trace logs with --write-junit-xml upon failure Johannes Schindelin via GitGitGadget
                     ` (5 subsequent siblings)
  14 siblings, 3 replies; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-10-15 10:12 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

This should be more reliable than the current method, and prepares the
test suite for a consistent way to clean up before re-running the tests
with different options.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/lib-git-p4.sh                            | 10 +---------
 t/t0000-basic.sh                           |  2 ++
 t/t9800-git-p4-basic.sh                    |  4 ----
 t/t9801-git-p4-branch.sh                   |  4 ----
 t/t9802-git-p4-filetype.sh                 |  4 ----
 t/t9803-git-p4-shell-metachars.sh          |  4 ----
 t/t9804-git-p4-label.sh                    |  4 ----
 t/t9805-git-p4-skip-submit-edit.sh         |  4 ----
 t/t9806-git-p4-options.sh                  |  5 -----
 t/t9807-git-p4-submit.sh                   |  4 ----
 t/t9808-git-p4-chdir.sh                    |  4 ----
 t/t9809-git-p4-client-view.sh              |  4 ----
 t/t9810-git-p4-rcs.sh                      |  4 ----
 t/t9811-git-p4-label-import.sh             |  5 -----
 t/t9812-git-p4-wildcards.sh                |  4 ----
 t/t9813-git-p4-preserve-users.sh           |  4 ----
 t/t9814-git-p4-rename.sh                   |  4 ----
 t/t9815-git-p4-submit-fail.sh              |  4 ----
 t/t9816-git-p4-locked.sh                   |  4 ----
 t/t9817-git-p4-exclude.sh                  |  4 ----
 t/t9818-git-p4-block.sh                    |  4 ----
 t/t9819-git-p4-case-folding.sh             |  4 ----
 t/t9820-git-p4-editor-handling.sh          |  4 ----
 t/t9821-git-p4-path-variations.sh          |  4 ----
 t/t9822-git-p4-path-encoding.sh            |  4 ----
 t/t9823-git-p4-mock-lfs.sh                 |  4 ----
 t/t9824-git-p4-git-lfs.sh                  |  4 ----
 t/t9825-git-p4-handle-utf16-without-bom.sh |  4 ----
 t/t9826-git-p4-keep-empty-commits.sh       |  4 ----
 t/t9827-git-p4-change-filetype.sh          |  4 ----
 t/t9828-git-p4-map-user.sh                 |  4 ----
 t/t9829-git-p4-jobs.sh                     |  4 ----
 t/t9830-git-p4-symlink-dir.sh              |  4 ----
 t/t9831-git-p4-triggers.sh                 |  4 ----
 t/t9832-unshelve.sh                        |  4 ----
 t/t9833-errors.sh                          |  5 -----
 36 files changed, 3 insertions(+), 148 deletions(-)

diff --git a/t/lib-git-p4.sh b/t/lib-git-p4.sh
index c27599474c..f4f5d7d296 100644
--- a/t/lib-git-p4.sh
+++ b/t/lib-git-p4.sh
@@ -74,15 +74,6 @@ cli="$TRASH_DIRECTORY/cli"
 git="$TRASH_DIRECTORY/git"
 pidfile="$TRASH_DIRECTORY/p4d.pid"
 
-# Sometimes "prove" seems to hang on exit because p4d is still running
-cleanup () {
-	if test -f "$pidfile"
-	then
-		kill -9 $(cat "$pidfile") 2>/dev/null && exit 255
-	fi
-}
-trap cleanup EXIT
-
 # git p4 submit generates a temp file, which will
 # not get cleaned up if the submission fails.  Don't
 # clutter up /tmp on the test machine.
@@ -141,6 +132,7 @@ start_p4d () {
 		# p4d failed to start
 		return 1
 	fi
+	test_atexit kill_p4d
 
 	# build a p4 user so author@example.com has an entry
 	p4_add_user author
diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index 8c5faa6ce1..041bd7e3ce 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -134,6 +134,7 @@ check_sub_test_lib_test_err () {
 	)
 }
 
+cat >/dev/null <<\DDD
 test_expect_success 'pretend we have a fully passing test suite' "
 	run_sub_test_lib_test full-pass '3 passing tests' <<-\\EOF &&
 	for i in 1 2 3
@@ -820,6 +821,7 @@ test_expect_success 'tests clean up even on failures' "
 	> 1..2
 	EOF
 "
+DDD
 
 test_expect_success 'test_atexit is run' "
 	test_must_fail run_sub_test_lib_test \
diff --git a/t/t9800-git-p4-basic.sh b/t/t9800-git-p4-basic.sh
index 729cd25770..5856563068 100755
--- a/t/t9800-git-p4-basic.sh
+++ b/t/t9800-git-p4-basic.sh
@@ -326,8 +326,4 @@ test_expect_success 'submit from worktree' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9801-git-p4-branch.sh b/t/t9801-git-p4-branch.sh
index 6a86d6996b..50013132c8 100755
--- a/t/t9801-git-p4-branch.sh
+++ b/t/t9801-git-p4-branch.sh
@@ -610,8 +610,4 @@ test_expect_success 'Update a file in git side and submit to P4 using client vie
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9802-git-p4-filetype.sh b/t/t9802-git-p4-filetype.sh
index 9978352d78..94edebe272 100755
--- a/t/t9802-git-p4-filetype.sh
+++ b/t/t9802-git-p4-filetype.sh
@@ -333,8 +333,4 @@ test_expect_success SYMLINKS 'empty symlink target' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9803-git-p4-shell-metachars.sh b/t/t9803-git-p4-shell-metachars.sh
index d5c3675100..2913277013 100755
--- a/t/t9803-git-p4-shell-metachars.sh
+++ b/t/t9803-git-p4-shell-metachars.sh
@@ -105,8 +105,4 @@ test_expect_success 'branch with shell char' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9804-git-p4-label.sh b/t/t9804-git-p4-label.sh
index e30f80e617..3236457106 100755
--- a/t/t9804-git-p4-label.sh
+++ b/t/t9804-git-p4-label.sh
@@ -108,8 +108,4 @@ test_expect_failure 'two labels on the same changelist' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9805-git-p4-skip-submit-edit.sh b/t/t9805-git-p4-skip-submit-edit.sh
index 5fbf904dc8..90ef647db7 100755
--- a/t/t9805-git-p4-skip-submit-edit.sh
+++ b/t/t9805-git-p4-skip-submit-edit.sh
@@ -98,8 +98,4 @@ test_expect_success 'no config, edited' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9806-git-p4-options.sh b/t/t9806-git-p4-options.sh
index 3f5291b857..4e794a01bf 100755
--- a/t/t9806-git-p4-options.sh
+++ b/t/t9806-git-p4-options.sh
@@ -300,9 +300,4 @@ test_expect_success 'use --git-dir option and GIT_DIR' '
 	test_path_is_file "$git"/cli_file2.t
 '
 
-
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9807-git-p4-submit.sh b/t/t9807-git-p4-submit.sh
index 2325599ee6..488d916c10 100755
--- a/t/t9807-git-p4-submit.sh
+++ b/t/t9807-git-p4-submit.sh
@@ -542,8 +542,4 @@ test_expect_success 'submit --update-shelve' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9808-git-p4-chdir.sh b/t/t9808-git-p4-chdir.sh
index 11d2b5102c..58a9b3b71e 100755
--- a/t/t9808-git-p4-chdir.sh
+++ b/t/t9808-git-p4-chdir.sh
@@ -83,8 +83,4 @@ test_expect_success SYMLINKS 'p4 client root symlink should stay symbolic' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9809-git-p4-client-view.sh b/t/t9809-git-p4-client-view.sh
index 897b3c3034..3cff1fce1b 100755
--- a/t/t9809-git-p4-client-view.sh
+++ b/t/t9809-git-p4-client-view.sh
@@ -836,8 +836,4 @@ test_expect_success 'quotes on both sides' '
 	git_verify "cdir 1/file11" "cdir 1/file12"
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9810-git-p4-rcs.sh b/t/t9810-git-p4-rcs.sh
index cc53debe19..57b533dc6f 100755
--- a/t/t9810-git-p4-rcs.sh
+++ b/t/t9810-git-p4-rcs.sh
@@ -360,8 +360,4 @@ test_expect_failure 'Add keywords in git which do not match the default p4 value
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9811-git-p4-label-import.sh b/t/t9811-git-p4-label-import.sh
index 602b0a5d5c..b70e81c3cd 100755
--- a/t/t9811-git-p4-label-import.sh
+++ b/t/t9811-git-p4-label-import.sh
@@ -259,9 +259,4 @@ test_expect_success 'importing labels with missing revisions' '
 	)
 '
 
-
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9812-git-p4-wildcards.sh b/t/t9812-git-p4-wildcards.sh
index 0206771fbb..254a7c2446 100755
--- a/t/t9812-git-p4-wildcards.sh
+++ b/t/t9812-git-p4-wildcards.sh
@@ -211,8 +211,4 @@ test_expect_success 'wildcard files requiring keyword scrub' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9813-git-p4-preserve-users.sh b/t/t9813-git-p4-preserve-users.sh
index 783c6ad165..fd018c87a8 100755
--- a/t/t9813-git-p4-preserve-users.sh
+++ b/t/t9813-git-p4-preserve-users.sh
@@ -138,8 +138,4 @@ test_expect_success 'not preserving user with mixed authorship' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9814-git-p4-rename.sh b/t/t9814-git-p4-rename.sh
index 60baa06e27..468767cbf4 100755
--- a/t/t9814-git-p4-rename.sh
+++ b/t/t9814-git-p4-rename.sh
@@ -242,8 +242,4 @@ test_expect_success P4D_HAVE_CONFIGURABLE_RUN_MOVE_ALLOW \
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9815-git-p4-submit-fail.sh b/t/t9815-git-p4-submit-fail.sh
index eaf03a6563..9779dc0d11 100755
--- a/t/t9815-git-p4-submit-fail.sh
+++ b/t/t9815-git-p4-submit-fail.sh
@@ -422,8 +422,4 @@ test_expect_success 'cleanup chmod after submit cancel' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9816-git-p4-locked.sh b/t/t9816-git-p4-locked.sh
index d048bd33fa..932841003c 100755
--- a/t/t9816-git-p4-locked.sh
+++ b/t/t9816-git-p4-locked.sh
@@ -138,8 +138,4 @@ test_expect_failure 'move with lock taken' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9817-git-p4-exclude.sh b/t/t9817-git-p4-exclude.sh
index aac568eadf..96d25f0c02 100755
--- a/t/t9817-git-p4-exclude.sh
+++ b/t/t9817-git-p4-exclude.sh
@@ -64,8 +64,4 @@ test_expect_success 'clone, then sync with exclude' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9818-git-p4-block.sh b/t/t9818-git-p4-block.sh
index ce7cb22ad3..0db7ab9918 100755
--- a/t/t9818-git-p4-block.sh
+++ b/t/t9818-git-p4-block.sh
@@ -146,8 +146,4 @@ test_expect_success 'Clone repo with self-sizing block size' '
 	test_line_count \> 10 log
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9819-git-p4-case-folding.sh b/t/t9819-git-p4-case-folding.sh
index d808c008c1..600ce1e0b0 100755
--- a/t/t9819-git-p4-case-folding.sh
+++ b/t/t9819-git-p4-case-folding.sh
@@ -53,8 +53,4 @@ test_expect_failure 'Clone UC repo with lc name' '
 	test_must_fail git p4 clone //depot/uc/...
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9820-git-p4-editor-handling.sh b/t/t9820-git-p4-editor-handling.sh
index 3c22f74bd4..fa1bba1dd9 100755
--- a/t/t9820-git-p4-editor-handling.sh
+++ b/t/t9820-git-p4-editor-handling.sh
@@ -31,8 +31,4 @@ test_expect_success 'EDITOR with options' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9821-git-p4-path-variations.sh b/t/t9821-git-p4-path-variations.sh
index 81e46acfa8..ef80f1690b 100755
--- a/t/t9821-git-p4-path-variations.sh
+++ b/t/t9821-git-p4-path-variations.sh
@@ -193,8 +193,4 @@ test_expect_success 'Add a new file and clone path with new file (ignorecase)' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9822-git-p4-path-encoding.sh b/t/t9822-git-p4-path-encoding.sh
index c78477c19b..1bf7635016 100755
--- a/t/t9822-git-p4-path-encoding.sh
+++ b/t/t9822-git-p4-path-encoding.sh
@@ -67,8 +67,4 @@ test_expect_success 'Delete iso8859-1 encoded paths and clone' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9823-git-p4-mock-lfs.sh b/t/t9823-git-p4-mock-lfs.sh
index 1f2dc369bf..88b76dc4d6 100755
--- a/t/t9823-git-p4-mock-lfs.sh
+++ b/t/t9823-git-p4-mock-lfs.sh
@@ -185,8 +185,4 @@ test_expect_success 'Run git p4 submit in repo configured with large file system
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9824-git-p4-git-lfs.sh b/t/t9824-git-p4-git-lfs.sh
index ed80ca858c..a28dbbdd56 100755
--- a/t/t9824-git-p4-git-lfs.sh
+++ b/t/t9824-git-p4-git-lfs.sh
@@ -287,8 +287,4 @@ test_expect_success 'Add big files to repo and store files in LFS based on compr
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9825-git-p4-handle-utf16-without-bom.sh b/t/t9825-git-p4-handle-utf16-without-bom.sh
index 1551845dc1..f049ff8229 100755
--- a/t/t9825-git-p4-handle-utf16-without-bom.sh
+++ b/t/t9825-git-p4-handle-utf16-without-bom.sh
@@ -43,8 +43,4 @@ test_expect_failure 'clone depot with invalid UTF-16 file in non-verbose mode' '
 	git p4 clone --dest="$git" //depot
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9826-git-p4-keep-empty-commits.sh b/t/t9826-git-p4-keep-empty-commits.sh
index fa8b9daf1f..fd64afe064 100755
--- a/t/t9826-git-p4-keep-empty-commits.sh
+++ b/t/t9826-git-p4-keep-empty-commits.sh
@@ -127,8 +127,4 @@ test_expect_success 'Clone repo subdir with all history' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9827-git-p4-change-filetype.sh b/t/t9827-git-p4-change-filetype.sh
index 7433998f47..d3670bd7a2 100755
--- a/t/t9827-git-p4-change-filetype.sh
+++ b/t/t9827-git-p4-change-filetype.sh
@@ -59,8 +59,4 @@ test_expect_success SYMLINKS 'change symbolic link to file' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9828-git-p4-map-user.sh b/t/t9828-git-p4-map-user.sh
index e20395c89f..ca6c2942bd 100755
--- a/t/t9828-git-p4-map-user.sh
+++ b/t/t9828-git-p4-map-user.sh
@@ -54,8 +54,4 @@ test_expect_success 'Clone repo root path with all history' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9829-git-p4-jobs.sh b/t/t9829-git-p4-jobs.sh
index 971aeeea1f..88cfb1fcd3 100755
--- a/t/t9829-git-p4-jobs.sh
+++ b/t/t9829-git-p4-jobs.sh
@@ -92,8 +92,4 @@ test_expect_success 'check log message of changelist with more jobs' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9830-git-p4-symlink-dir.sh b/t/t9830-git-p4-symlink-dir.sh
index 2ad1b0810d..3fb6960c18 100755
--- a/t/t9830-git-p4-symlink-dir.sh
+++ b/t/t9830-git-p4-symlink-dir.sh
@@ -36,8 +36,4 @@ test_expect_success 'symlinked directory' '
 
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9831-git-p4-triggers.sh b/t/t9831-git-p4-triggers.sh
index be44c9751a..d743ca33ee 100755
--- a/t/t9831-git-p4-triggers.sh
+++ b/t/t9831-git-p4-triggers.sh
@@ -96,8 +96,4 @@ test_expect_success 'submit description with extra info lines from verbose p4 ch
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9832-unshelve.sh b/t/t9832-unshelve.sh
index 48ec7679b8..b649d1b7c3 100755
--- a/t/t9832-unshelve.sh
+++ b/t/t9832-unshelve.sh
@@ -131,8 +131,4 @@ test_expect_success 'try to unshelve the change' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
 test_done
diff --git a/t/t9833-errors.sh b/t/t9833-errors.sh
index 277d347012..1f3d879122 100755
--- a/t/t9833-errors.sh
+++ b/t/t9833-errors.sh
@@ -72,9 +72,4 @@ test_expect_success 'git operation with expired ticket' '
 	)
 '
 
-test_expect_success 'kill p4d' '
-	kill_p4d
-'
-
-
 test_done
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH v2 10/13] tests: include detailed trace logs with --write-junit-xml upon failure
  2018-10-15 10:11 ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Johannes Schindelin via GitGitGadget
                     ` (8 preceding siblings ...)
  2018-10-15 10:12   ` [PATCH v2 09/13] git-p4: use `test_atexit` to kill the daemon Johannes Schindelin via GitGitGadget
@ 2018-10-15 10:12   ` Johannes Schindelin via GitGitGadget
  2018-10-16 10:04     ` SZEDER Gábor
  2018-10-15 10:12   ` [PATCH v2 11/13] tests: record more stderr with --write-junit-xml in case of failure Johannes Schindelin via GitGitGadget
                     ` (4 subsequent siblings)
  14 siblings, 1 reply; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-10-15 10:12 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The JUnit XML format lends itself to be presented in a powerful UI,
where you can drill down to the information you are interested in very
quickly.

For test failures, this usually means that you want to see the detailed
trace of the failing tests.

With Travis CI, we passed the `--verbose-log` option to get those
traces. However, that seems excessive, as we do not need/use the logs in
almost all of those cases: only when a test fails do we have a way to
include the trace.

So let's do something different when using Azure DevOps: let's run all
the tests with `--quiet` first, and only if a failure is encountered,
try to trace the commands as they are executed.

Of course, we cannot turn on `--verbose-log` after the fact. So let's
just re-run the test with all the same options, adding `--verbose-log`.
And then munging the output file into the JUnit XML on the fly.

Note: there is an off chance that re-running the test in verbose mode
"fixes" the failures (and this does happen from time to time!). That is
a possibility we should be able to live with. Ideally, we would label
this as "Passed upon rerun", and Azure Pipelines even know about that
outcome, but it is not available when using the JUnit XML format for
now:
https://github.com/Microsoft/azure-pipelines-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.cs

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/test-lib.sh | 34 +++++++++++++++++++++++++++++++++-
 1 file changed, 33 insertions(+), 1 deletion(-)

diff --git a/t/test-lib.sh b/t/test-lib.sh
index 6f9c1f5300..8a60e39364 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -85,6 +85,13 @@ done,*)
 	test "$(cat "$BASE.exit")" = 0
 	exit
 	;;
+*' --write-junit-xml '*)
+	# record how to call this script *with* --verbose-log, in case
+	# we encounter a breakage
+	junit_rerun_options_sq="$(printf '%s\n' "$0" --verbose-log -x "$@" |
+		sed -e "s/'/'\\\\''/g" -e "s/^/'/" -e "s/\$/'/" |
+		tr '\012' ' ')"
+	;;
 esac
 
 # For repeatability, reset the environment to known value.
@@ -446,10 +453,31 @@ test_ok_ () {
 test_failure_ () {
 	if test -n "$write_junit_xml"
 	then
+		if test -z "$GIT_TEST_TEE_OUTPUT_FILE"
+		then
+			# clean up
+			test_atexit_handler
+
+			# re-run with --verbose-log
+			echo "# Re-running: $junit_rerun_options_sq" >&2
+
+			cd "$TEST_DIRECTORY" &&
+			eval "${TEST_SHELL_PATH}" "$junit_rerun_options_sq" \
+				>/dev/null 2>&1
+			status=$?
+
+			say_color "" "$(test 0 = $status ||
+				echo "not ")ok $test_count - (re-ran with trace)"
+			say "1..$test_count"
+			GIT_EXIT_OK=t
+			exit $status
+		fi
+
 		junit_insert="<failure message=\"not ok $test_count -"
 		junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
 		junit_insert="$junit_insert $(xml_attr_encode \
-			"$(printf '%s\n' "$@" | sed 1d)")"
+			"$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")"
+		>"$GIT_TEST_TEE_OUTPUT_FILE"
 		junit_insert="$junit_insert</failure>"
 		write_junit_xml_testcase "$1" "      $junit_insert"
 	fi
@@ -734,6 +762,10 @@ test_start_ () {
 	if test -n "$write_junit_xml"
 	then
 		junit_start=$(test-tool date getnanos)
+
+		# truncate output
+		test -z "$GIT_TEST_TEE_OUTPUT_FILE" ||
+		>"$GIT_TEST_TEE_OUTPUT_FILE"
 	fi
 }
 
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH v2 11/13] tests: record more stderr with --write-junit-xml in case of failure
  2018-10-15 10:11 ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Johannes Schindelin via GitGitGadget
                     ` (9 preceding siblings ...)
  2018-10-15 10:12   ` [PATCH v2 10/13] tests: include detailed trace logs with --write-junit-xml upon failure Johannes Schindelin via GitGitGadget
@ 2018-10-15 10:12   ` Johannes Schindelin via GitGitGadget
  2018-10-15 10:12   ` [PATCH v2 12/13] README: add a build badge (status of the Azure Pipelines build) Johannes Schindelin via GitGitGadget
                     ` (3 subsequent siblings)
  14 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-10-15 10:12 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

Sometimes, failures in a test case are actually caused by issues in
earlier test cases.

To make it easier to see those issues, let's attach the output from
before the failing test case (i.e. stdout/stderr since the previous
failing test case, or the start of the test script). This will be
visible in the "Attachments" of the details of the failed test.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/test-lib.sh | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/t/test-lib.sh b/t/test-lib.sh
index 8a60e39364..5f62418f9c 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -479,6 +479,9 @@ test_failure_ () {
 			"$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")"
 		>"$GIT_TEST_TEE_OUTPUT_FILE"
 		junit_insert="$junit_insert</failure>"
+		junit_insert="$junit_insert<system-err>$(xml_attr_encode \
+			"$(cat "$GIT_TEST_TEE_OUTPUT_FILE.err")")</system-err>"
+		>"$GIT_TEST_TEE_OUTPUT_FILE.err"
 		write_junit_xml_testcase "$1" "      $junit_insert"
 	fi
 	test_failure=$(($test_failure + 1))
@@ -763,9 +766,12 @@ test_start_ () {
 	then
 		junit_start=$(test-tool date getnanos)
 
-		# truncate output
-		test -z "$GIT_TEST_TEE_OUTPUT_FILE" ||
-		>"$GIT_TEST_TEE_OUTPUT_FILE"
+		# append to future <system-err>; truncate output
+		test -z "$GIT_TEST_TEE_OUTPUT_FILE" || {
+			cat "$GIT_TEST_TEE_OUTPUT_FILE" \
+				>>"$GIT_TEST_TEE_OUTPUT_FILE.err"
+			>"$GIT_TEST_TEE_OUTPUT_FILE"
+		}
 	fi
 }
 
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH v2 12/13] README: add a build badge (status of the Azure Pipelines build)
  2018-10-15 10:11 ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Johannes Schindelin via GitGitGadget
                     ` (10 preceding siblings ...)
  2018-10-15 10:12   ` [PATCH v2 11/13] tests: record more stderr with --write-junit-xml in case of failure Johannes Schindelin via GitGitGadget
@ 2018-10-15 10:12   ` Johannes Schindelin via GitGitGadget
  2018-10-16  3:44     ` Eric Sunshine
  2018-10-15 10:12   ` [PATCH v2 13/13] travis: fix skipping tagged releases Johannes Schindelin via GitGitGadget
                     ` (2 subsequent siblings)
  14 siblings, 1 reply; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-10-15 10:12 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

Just like so many other OSS projects, we now also have a build badge.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 README.md | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/README.md b/README.md
index f920a42fad..bf4780c22d 100644
--- a/README.md
+++ b/README.md
@@ -1,3 +1,5 @@
+[![Build Status](https:/dev.azure.com/git/git/_apis/build/status/test-git.git)](https://dev.azure.com/git/git/_build/latest?definitionId=2)
+
 Git - fast, scalable, distributed revision control system
 =========================================================
 
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* [PATCH v2 13/13] travis: fix skipping tagged releases
  2018-10-15 10:11 ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Johannes Schindelin via GitGitGadget
                     ` (11 preceding siblings ...)
  2018-10-15 10:12   ` [PATCH v2 12/13] README: add a build badge (status of the Azure Pipelines build) Johannes Schindelin via GitGitGadget
@ 2018-10-15 10:12   ` Johannes Schindelin via GitGitGadget
  2018-10-16  9:40     ` SZEDER Gábor
  2018-10-15 14:22   ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Taylor Blau
  2018-10-25  8:17   ` Junio C Hamano
  14 siblings, 1 reply; 88+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2018-10-15 10:12 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

When building a PR, TRAVIS_BRANCH refers to the *target branch*.
Therefore, if a PR targets `master`, and `master` happened to be tagged,
we skipped the build by mistake.

Fix this by using TRAVIS_PULL_REQUEST_BRANCH (i.e. the *source branch*)
when available, falling back to TRAVIS_BRANCH (i.e. for CI builds, also
known as "push builds").

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 ci/lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ci/lib.sh b/ci/lib.sh
index 584abcd529..e1858ae609 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -3,7 +3,7 @@
 if test true = "$TRAVIS"
 then
 	# We are running within Travis CI
-	CI_BRANCH="$TRAVIS_BRANCH"
+	CI_BRANCH="${TRAVIS_PULL_REQUEST_BRANCH:-$TRAVIS_BRANCH}"
 	CI_COMMIT="$TRAVIS_COMMIT"
 	CI_JOB_ID="$TRAVIS_JOB_ID"
 	CI_JOB_NUMBER="$TRAVIS_JOB_NUMBER"
-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 09/13] git-p4: use `test_atexit` to kill the daemon
  2018-10-15 10:12   ` [PATCH v2 09/13] git-p4: use `test_atexit` to kill the daemon Johannes Schindelin via GitGitGadget
@ 2018-10-15 11:00     ` Luke Diamand
  2018-10-15 15:02       ` Johannes Schindelin
  2018-10-16  3:34     ` Eric Sunshine
  2018-10-17 23:29     ` SZEDER Gábor
  2 siblings, 1 reply; 88+ messages in thread
From: Luke Diamand @ 2018-10-15 11:00 UTC (permalink / raw)
  To: gitgitgadget; +Cc: Git Users, Junio C Hamano, Johannes Schindelin

On Mon, 15 Oct 2018 at 11:12, Johannes Schindelin via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>
> From: Johannes Schindelin <johannes.schindelin@gmx.de>
>
> This should be more reliable than the current method, and prepares the
> test suite for a consistent way to clean up before re-running the tests
> with different options.
>

I'm finding that it's leaving p4d processes lying around.

e.g.

$ ./t9820-git-p4-editor-handling.sh
<passes>
$ ./t9820-git-p4-editor-handling.sh
<fails>

And also

$ ./t9800-git-p4-basic.sh
<starts running tests, but I get bored easily>
Ctrl-C

$ ./t9800-git-p4-basic.sh
<fails>

$ ps | grep p4d
21392 pts/1    00:00:00 p4d  <<<<<


Luke

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines
  2018-10-15 10:11 ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Johannes Schindelin via GitGitGadget
                     ` (12 preceding siblings ...)
  2018-10-15 10:12   ` [PATCH v2 13/13] travis: fix skipping tagged releases Johannes Schindelin via GitGitGadget
@ 2018-10-15 14:22   ` Taylor Blau
  2018-10-15 14:55     ` Johannes Schindelin
  2018-10-16 10:30     ` SZEDER Gábor
  2018-10-25  8:17   ` Junio C Hamano
  14 siblings, 2 replies; 88+ messages in thread
From: Taylor Blau @ 2018-10-15 14:22 UTC (permalink / raw)
  To: Johannes Schindelin via GitGitGadget; +Cc: git, Junio C Hamano

Hi Johannes,

Thanks for putting this together, and offering to build Git on Azure
Pipelines. I haven't followed v1 of this series very closely, so please
excuse me if my comments have already been addressed, and I missed them
in a skim of the last revision.

On Mon, Oct 15, 2018 at 03:11:57AM -0700, Johannes Schindelin via GitGitGadget wrote:
> It is also an invaluable tool for contributors who can validate their code
> contributions via PRs on GitHub, e.g. to verify that their tests do actually
> run on macOS (i.e. with the BSD family of Unix tools instead of the GNU
> one).

Agree.

> The one sad part about this is the Windows support. Travis lacks it, and we
> work around that by using Azure Pipelines (the CI part of Azure DevOps,
> formerly known as Visual Studio Team Services) indirectly: one phase in
> Travis would trigger a build, wait for its log, and then paste that log.

I wonder if Travis' recent announcement [1] affects this at all. To
summarize [1], Travis is offering an early version of adding Windows to
their list of supported builder operations systems. This brings the list
to macOS, Linux, and Windows, which I think satisfies what we would
like to regularly build git.git on.

Would we like to abandon Travis as our main CI service for upstream
git.git, and build on Azure Pipelines only? If so, I think that this is
an OK way to go, but I think that I would be opposed to having more than
one build system. (FWIW, we tend to _three_ for Git LFS, and it can be a
hassle at times).

I see some benefit to sticking with Travis, since we already have a
build configuration that works there. But, you've done the work to
"port" that build configuration over to Azure, so perhaps the point is
moot.

> As Git's Windows builds (and tests!) take quite a bit of time, Travis often
> timed out, or somehow the trigger did not work, and for security reasons
> (the Windows builds are performed in a private pool of containers), the
> Windows builds are completely disabled for Pull Requests on GitHub.

This would be a concession of [1], in my mind: is it possible to run the
tests on Windows in a time such that Travis will not time out?

> As a special treat, this patch series adds the ability to present the
> outcome of Git's test suite as JUnit-style .xml files. This allows the Azure
> Pipelines build to present fun diagrams, trends, and makes it a lot easier
> to drill down to test failures than before. See for example
> https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details
> [https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details]
> (you can click on the label of the failed test, and then see the detailed
> output in the right pane).

That's pretty cool. Travis doesn't support this (to the best of my
knowledge).

Thanks,
Taylor

[1]: https://blog.travis-ci.com/2018-10-11-windows-early-release

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines
  2018-10-15 14:22   ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Taylor Blau
@ 2018-10-15 14:55     ` Johannes Schindelin
  2018-10-16  2:53       ` Taylor Blau
  2018-10-16 10:30     ` SZEDER Gábor
  1 sibling, 1 reply; 88+ messages in thread
From: Johannes Schindelin @ 2018-10-15 14:55 UTC (permalink / raw)
  To: Taylor Blau; +Cc: Johannes Schindelin via GitGitGadget, git, Junio C Hamano

Hi Taylor,

On Mon, 15 Oct 2018, Taylor Blau wrote:

> Thanks for putting this together, and offering to build Git on Azure
> Pipelines. I haven't followed v1 of this series very closely, so please
> excuse me if my comments have already been addressed, and I missed them
> in a skim of the last revision.
> 
> On Mon, Oct 15, 2018 at 03:11:57AM -0700, Johannes Schindelin via GitGitGadget wrote:
> > It is also an invaluable tool for contributors who can validate their code
> > contributions via PRs on GitHub, e.g. to verify that their tests do actually
> > run on macOS (i.e. with the BSD family of Unix tools instead of the GNU
> > one).
> 
> Agree.
> 
> > The one sad part about this is the Windows support. Travis lacks it, and we
> > work around that by using Azure Pipelines (the CI part of Azure DevOps,
> > formerly known as Visual Studio Team Services) indirectly: one phase in
> > Travis would trigger a build, wait for its log, and then paste that log.
> 
> I wonder if Travis' recent announcement [1] affects this at all.

:-)

It did not escape my notice that after years and years of trying to get
*anybody* at Travis to listen to my offers to help get this started, the
announcement of Azure Pipelines for OSS seemed to finally do the trick
(they still don't bother to talk to me, of course).

And to answer your question without a question mark: I do not really think
that the Travis announcement affects this here patch series: I have a ton
of good experience with Azure Pipelines, use it in Git for Windows for
ages, and I am finally able to have it in core Git, too. So I want it, and
I spent a lot of time getting there, and I think it probably won't hurt
core Git at all (besides, it seems that at least some of the phases are a
bit faster on Azure Pipelines than Travis).

Another really good reason for me to do this is that I can prod the Azure
Pipelines team directly. And I even get an answer, usually within minutes.
Which is a lot faster than the Travis team answers my questions, which
is... not yet? (I tried to get in contact with them in late 2015 or early
2016, and I tried again a year later, and then a couple of months later,
and I have yet to hear back.)

Also, I am not quite sure about the timeouts on Travis, but at least
AppVeyor had rather short timeouts: the Windows build (due to our
extensive use of Unix shell scripting in our test suite) takes 1h40m
currently, and AppVeyor times out after 20 or 30 minutes. I could imagine
that Travis times out after the same time, or maybe 60 minutes, which
would still be too short. On Azure Pipelines, the maximum timeout (which
can be configured via the azure-pipelines.yml file) is four hours IIRC.
Plenty enough even for our test suite on Windows.

> To summarize [1], Travis is offering an early version of adding Windows
> to their list of supported builder operations systems. This brings the
> list to macOS, Linux, and Windows, which I think satisfies what we would
> like to regularly build git.git on.

Honestly, I would love to have also FreeBSD and other platforms being
tested. And with Azure Pipelines, I can make that happen (eventually), by
adding another pool of VMs (given that I have a free $150/month Azure
subscription, I'd use Azure VMs, of course). As long as a platform can run
.NET Core software, it can run Azure Pipelines agents.

With Travis, I don't think I can add private agent pools.

> Would we like to abandon Travis as our main CI service for upstream
> git.git, and build on Azure Pipelines only? If so, I think that this is
> an OK way to go, but I think that I would be opposed to having more than
> one build system. (FWIW, we tend to _three_ for Git LFS, and it can be a
> hassle at times).

This question of abandoning Travis in favor of Azure Pipelines is a bit of
a hornets' nest, as I really, really only want to bring the goodness of
Azure Pipelines to git.git, and I am *clearly* biased, as I work at
Microsoft.

Which is the reason why I did not even hint at it in the cover letter, let
alone included a patch to make it so.

My patch series is purely about adding support for running CI/PR builds of
https://github.com/git/git via Azure Pipelines.

> I see some benefit to sticking with Travis, since we already have a
> build configuration that works there. But, you've done the work to
> "port" that build configuration over to Azure, so perhaps the point is
> moot.

It is not so much a port, as an attempt to generalize our ci/* files.

> > As Git's Windows builds (and tests!) take quite a bit of time, Travis often
> > timed out, or somehow the trigger did not work, and for security reasons
> > (the Windows builds are performed in a private pool of containers), the
> > Windows builds are completely disabled for Pull Requests on GitHub.
> 
> This would be a concession of [1], in my mind: is it possible to run the
> tests on Windows in a time such that Travis will not time out?

To be honest, I spent such a lot of time to get things to work on Azure
Pipelines, *and* we get a nice view on the test failures there, too (which
Travis will probably also offer soon, in response to what Azure Pipelines
offer ;-)), I cannot really justify spending time on trying to make things
work on Travis' Windows VMs, too. Especially when I have to expect to run
into timeout issues anyway.

> > As a special treat, this patch series adds the ability to present the
> > outcome of Git's test suite as JUnit-style .xml files. This allows the Azure
> > Pipelines build to present fun diagrams, trends, and makes it a lot easier
> > to drill down to test failures than before. See for example
> > https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details
> > [https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details]
> > (you can click on the label of the failed test, and then see the detailed
> > output in the right pane).
> 
> That's pretty cool. Travis doesn't support this (to the best of my
> knowledge).

Exactly.

Plus, if things don't work in Azure Pipelines, I (or one of the other
Microsoft employees among the core Git developers) can easily take a
shortcut to the team and get things fixed. In my mind, that counts for a
lot, too, especially given my own, frustrating personal experience with
Travis.

Ciao,
Dscho

> [1]: https://blog.travis-ci.com/2018-10-11-windows-early-release

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 09/13] git-p4: use `test_atexit` to kill the daemon
  2018-10-15 11:00     ` Luke Diamand
@ 2018-10-15 15:02       ` Johannes Schindelin
  2018-10-15 20:19         ` Luke Diamand
  0 siblings, 1 reply; 88+ messages in thread
From: Johannes Schindelin @ 2018-10-15 15:02 UTC (permalink / raw)
  To: Luke Diamand; +Cc: gitgitgadget, Git Users, Junio C Hamano

Hi Luke,

On Mon, 15 Oct 2018, Luke Diamand wrote:

> On Mon, 15 Oct 2018 at 11:12, Johannes Schindelin via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
> >
> > From: Johannes Schindelin <johannes.schindelin@gmx.de>
> >
> > This should be more reliable than the current method, and prepares the
> > test suite for a consistent way to clean up before re-running the tests
> > with different options.
> >
> 
> I'm finding that it's leaving p4d processes lying around.

That's a bummer!

> e.g.
> 
> $ ./t9820-git-p4-editor-handling.sh
> <passes>
> $ ./t9820-git-p4-editor-handling.sh
> <fails>

Since I do not currently have a setup with p4d installed, can you run that
with `sh -x ...` and see whether this code path is hit?

 test_done () {
        GIT_EXIT_OK=t

+       test -n "$immediate" || test_atexit_handler
+
        if test -n "$write_junit_xml" && test -n "$junit_xml_path"
        then

> And also
> 
> $ ./t9800-git-p4-basic.sh
> <starts running tests, but I get bored easily>
> Ctrl-C

Oh, you're right. I think I need to do something in this line:

	trap 'exit $?' INT

in t/test-lib.sh, something like

	trap 'exit_code=$?; test_atexit_handler; exit $exit_code' INT

would you agree? (And: could you test?)

Thanks,
Dscho

> $ ./t9800-git-p4-basic.sh
> <fails>
> 
> $ ps | grep p4d
> 21392 pts/1    00:00:00 p4d  <<<<<
> 
> 
> Luke
> 

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services
  2018-09-03 21:10 [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Johannes Schindelin via GitGitGadget
                   ` (10 preceding siblings ...)
  2018-10-15 10:11 ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Johannes Schindelin via GitGitGadget
@ 2018-10-15 15:06 ` Ævar Arnfjörð Bjarmason
  2018-10-15 15:42   ` Duy Nguyen
  2018-10-16  4:47   ` [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Junio C Hamano
  11 siblings, 2 replies; 88+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2018-10-15 15:06 UTC (permalink / raw)
  To: Johannes Schindelin via GitGitGadget; +Cc: git, Junio C Hamano


On Mon, Sep 03 2018, Johannes Schindelin via GitGitGadget wrote:

> As a special treat, this patch series adds the ability to present the
> outcome of Git's test suite as JUnit-style .xml files. This allows the VSTS
> build to present fun diagrams, trends, and makes it a lot easier to drill
> down to test failures than before. See for example
> https://git.visualstudio.com/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details
> [https://git.visualstudio.com/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details]
> (you can click on the label of the failed test, and then see the detailed
> output in the right pane).
>
> This patch series took way more time than I had originally planned, but I
> think that in particular the advanced display of the test results was worth
> it. Please let me know what you think about this.

I have not reviewed this in any detail, but incorporating this in some
form or other seems like a no-brainer to me.

If we have "free" (from the perspective of the project) CPU being
offered by various CI setups let's use it.

If people don't like it they don't have to look at those CI runs, and
the burden of carrying another CI in git.git is minimal, and actually
this patch series makes it much easier to incorporate other CI's later,
since you took care of moving the Travis-specific bits out.

As an aside I poked Junio via private mail in late August to see if he'd
be interested in pushing to gitlab.com/git/git.git too as part of his
normal push-outs. One neat thing that would buy us is the ability to
have a .gitlab-ci.yml in git.git and use their CI implementation.

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services
  2018-10-15 15:06 ` [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Ævar Arnfjörð Bjarmason
@ 2018-10-15 15:42   ` Duy Nguyen
  2018-10-15 16:17     ` Christian Couder
  2018-10-16  4:47   ` [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Junio C Hamano
  1 sibling, 1 reply; 88+ messages in thread
From: Duy Nguyen @ 2018-10-15 15:42 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: gitgitgadget, Git Mailing List, Junio C Hamano

On Mon, Oct 15, 2018 at 5:08 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> As an aside I poked Junio via private mail in late August to see if he'd
> be interested in pushing to gitlab.com/git/git.git too as part of his
> normal push-outs. One neat thing that would buy us is the ability to
> have a .gitlab-ci.yml in git.git and use their CI implementation.

That would be great since it allows gitlab forks to have CI support
from the beginning. But gitlab ci has time execution limits if I
remember correctly and I'm not sure if we'll use it all up before the
end of the month (or whatever period that is), or do they offer
something special to git.git?
-- 
Duy

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services
  2018-10-15 15:42   ` Duy Nguyen
@ 2018-10-15 16:17     ` Christian Couder
  2018-10-15 18:33       ` Johannes Schindelin
  0 siblings, 1 reply; 88+ messages in thread
From: Christian Couder @ 2018-10-15 16:17 UTC (permalink / raw)
  To: Nguyen Thai Ngoc Duy
  Cc: Ævar Arnfjörð Bjarmason, gitgitgadget, git,
	Junio C Hamano

On Mon, Oct 15, 2018 at 5:46 PM Duy Nguyen <pclouds@gmail.com> wrote:
>
> On Mon, Oct 15, 2018 at 5:08 PM Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
> > As an aside I poked Junio via private mail in late August to see if he'd
> > be interested in pushing to gitlab.com/git/git.git too as part of his
> > normal push-outs.

People at GitLab also have wondered if it would be possible for GitLab
to host an official Git mirror, and I think it would be nice indeed if
Junio could push to gitlab.com.

> > One neat thing that would buy us is the ability to
> > have a .gitlab-ci.yml in git.git and use their CI implementation.

Yeah, that would be nice for people who prefer to use GitLab CI.

In general I agree that it is a good idea to have support for many CI
tools even if it is not always up-to-date and even if it's not used by
many people.

> That would be great since it allows gitlab forks to have CI support
> from the beginning. But gitlab ci has time execution limits if I
> remember correctly and I'm not sure if we'll use it all up before the
> end of the month (or whatever period that is), or do they offer
> something special to git.git?

I can ask for more time if that would help.

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services
  2018-10-15 16:17     ` Christian Couder
@ 2018-10-15 18:33       ` Johannes Schindelin
  2018-10-16  0:24         ` Christian Couder
  2018-10-16  4:50         ` Junio C Hamano
  0 siblings, 2 replies; 88+ messages in thread
From: Johannes Schindelin @ 2018-10-15 18:33 UTC (permalink / raw)
  To: Christian Couder
  Cc: Nguyen Thai Ngoc Duy, Ævar Arnfjörð Bjarmason,
	gitgitgadget, git, Junio C Hamano

[-- Attachment #1: Type: text/plain, Size: 713 bytes --]

Hi team,

On Mon, 15 Oct 2018, Christian Couder wrote:

> On Mon, Oct 15, 2018 at 5:46 PM Duy Nguyen <pclouds@gmail.com> wrote:
> >
> > On Mon, Oct 15, 2018 at 5:08 PM Ævar Arnfjörð Bjarmason
> > <avarab@gmail.com> wrote:
> > > As an aside I poked Junio via private mail in late August to see if he'd
> > > be interested in pushing to gitlab.com/git/git.git too as part of his
> > > normal push-outs.
> 
> People at GitLab also have wondered if it would be possible for GitLab
> to host an official Git mirror, and I think it would be nice indeed if
> Junio could push to gitlab.com.

AFAIR Junio does not push to github.com/git/git, it is an automatic
mirror.

GitLab could easily do the same.

Ciao,
Johannes

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 09/13] git-p4: use `test_atexit` to kill the daemon
  2018-10-15 15:02       ` Johannes Schindelin
@ 2018-10-15 20:19         ` Luke Diamand
  2018-10-16  9:39           ` Johannes Schindelin
  0 siblings, 1 reply; 88+ messages in thread
From: Luke Diamand @ 2018-10-15 20:19 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: gitgitgadget, Git Users, Junio C Hamano

On Mon, 15 Oct 2018 at 16:02, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Luke,
>
> On Mon, 15 Oct 2018, Luke Diamand wrote:
>
> > On Mon, 15 Oct 2018 at 11:12, Johannes Schindelin via GitGitGadget
> > <gitgitgadget@gmail.com> wrote:
> > >
> > > From: Johannes Schindelin <johannes.schindelin@gmx.de>
> > >
> > > This should be more reliable than the current method, and prepares the
> > > test suite for a consistent way to clean up before re-running the tests
> > > with different options.
> > >
> >
> > I'm finding that it's leaving p4d processes lying around.
>
> That's a bummer!
>
> > e.g.
> >
> > $ ./t9820-git-p4-editor-handling.sh
> > <passes>
> > $ ./t9820-git-p4-editor-handling.sh
> > <fails>
>
> Since I do not currently have a setup with p4d installed, can you run that
> with `sh -x ...` and see whether this code path is hit?

All you need to do is to put p4 and p4d in your PATH.

https://www.perforce.com/downloads/helix-core-p4d
https://www.perforce.com/downloads/helix-command-line-client-p4

The server is free to use for a small number of users, you don't need
to do anything to make it go.


>
>  test_done () {
>         GIT_EXIT_OK=t
>
> +       test -n "$immediate" || test_atexit_handler
> +

+ test -n
+ test_atexit_handler
./t9820-git-p4-editor-handling.sh: 764:
./t9820-git-p4-editor-handling.sh: test_atexit_handler: not found

Is that expected?




>         if test -n "$write_junit_xml" && test -n "$junit_xml_path"
>         then
>
> > And also
> >
> > $ ./t9800-git-p4-basic.sh
> > <starts running tests, but I get bored easily>
> > Ctrl-C
>
> Oh, you're right. I think I need to do something in this line:
>
>         trap 'exit $?' INT
>
> in t/test-lib.sh, something like
>
>         trap 'exit_code=$?; test_atexit_handler; exit $exit_code' INT
>
> would you agree? (And: could you test?)

Not sure.
Send me a patch and I can try it out.

Thanks,
Luke

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services
  2018-10-15 18:33       ` Johannes Schindelin
@ 2018-10-16  0:24         ` Christian Couder
  2018-10-16  4:50         ` Junio C Hamano
  1 sibling, 0 replies; 88+ messages in thread
From: Christian Couder @ 2018-10-16  0:24 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Nguyen Thai Ngoc Duy, Ævar Arnfjörð Bjarmason,
	gitgitgadget, git, Junio C Hamano

On Mon, Oct 15, 2018 at 8:33 PM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi team,
>
> On Mon, 15 Oct 2018, Christian Couder wrote:
>
> > On Mon, Oct 15, 2018 at 5:46 PM Duy Nguyen <pclouds@gmail.com> wrote:
> > >
> > > On Mon, Oct 15, 2018 at 5:08 PM Ævar Arnfjörð Bjarmason
> > > <avarab@gmail.com> wrote:
> > > > As an aside I poked Junio via private mail in late August to see if he'd
> > > > be interested in pushing to gitlab.com/git/git.git too as part of his
> > > > normal push-outs.
> >
> > People at GitLab also have wondered if it would be possible for GitLab
> > to host an official Git mirror, and I think it would be nice indeed if
> > Junio could push to gitlab.com.
>
> AFAIR Junio does not push to github.com/git/git, it is an automatic
> mirror.

On https://git-blame.blogspot.com/p/git-public-repositories.html there is:

"Copies of the source code to Git live in many repositories, and this
is a list of the ones I push into. Some repositories have only a
subset of branches.

With maint, master, next, pu, todo:

[...]
https://github.com/git/git/
"

> GitLab could easily do the same.

Yeah, GitLab could easily create an automatic mirror. I think it would
be a bit better though if the git repo on GitLab could be independent.
It could make things more resilient against outages or security
issues.

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines
  2018-10-15 14:55     ` Johannes Schindelin
@ 2018-10-16  2:53       ` Taylor Blau
  2018-10-16 11:55         ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 88+ messages in thread
From: Taylor Blau @ 2018-10-16  2:53 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Taylor Blau, Johannes Schindelin via GitGitGadget, git, Junio C Hamano

On Mon, Oct 15, 2018 at 04:55:25PM +0200, Johannes Schindelin wrote:
> Hi Taylor,
>
> On Mon, 15 Oct 2018, Taylor Blau wrote:
>
> > Thanks for putting this together, and offering to build Git on Azure
> > Pipelines. I haven't followed v1 of this series very closely, so please
> > excuse me if my comments have already been addressed, and I missed them
> > in a skim of the last revision.
> >
> > On Mon, Oct 15, 2018 at 03:11:57AM -0700, Johannes Schindelin via GitGitGadget wrote:
> > > It is also an invaluable tool for contributors who can validate their code
> > > contributions via PRs on GitHub, e.g. to verify that their tests do actually
> > > run on macOS (i.e. with the BSD family of Unix tools instead of the GNU
> > > one).
> >
> > Agree.
> >
> > > The one sad part about this is the Windows support. Travis lacks it, and we
> > > work around that by using Azure Pipelines (the CI part of Azure DevOps,
> > > formerly known as Visual Studio Team Services) indirectly: one phase in
> > > Travis would trigger a build, wait for its log, and then paste that log.
> >
> > I wonder if Travis' recent announcement [1] affects this at all.
>
> :-)
>
> It did not escape my notice that after years and years of trying to get
> *anybody* at Travis to listen to my offers to help get this started, the
> announcement of Azure Pipelines for OSS seemed to finally do the trick
> (they still don't bother to talk to me, of course).
>
> And to answer your question without a question mark: I do not really think
> that the Travis announcement affects this here patch series: I have a ton
> of good experience with Azure Pipelines, use it in Git for Windows for
> ages, and I am finally able to have it in core Git, too. So I want it, and
> I spent a lot of time getting there, and I think it probably won't hurt
> core Git at all (besides, it seems that at least some of the phases are a
> bit faster on Azure Pipelines than Travis).

I think that there are fair reasons to prefer Azure Pipelines over
Travis. In particular, I am encouraged by the fact that we (1) know that
we won't timeout, and (2) can have a standardized CI interface on Git
and Git for Windows. Certainly, the Windows support on Azure Pipelines
is more developed than that of Travis', so that's another point for
Azure.

> Another really good reason for me to do this is that I can prod the Azure
> Pipelines team directly. And I even get an answer, usually within minutes.
> Which is a lot faster than the Travis team answers my questions, which
> is... not yet? (I tried to get in contact with them in late 2015 or early
> 2016, and I tried again a year later, and then a couple of months later,
> and I have yet to hear back.)

Certainly a good reason. To be clear/fair, I've sent in a number of
support tickets to Travis CI over the years, and have always been
responded to in a short amount of time with helpful answers. I think
that we would really be fine in either case, TBH.

> Also, I am not quite sure about the timeouts on Travis, but at least
> AppVeyor had rather short timeouts: the Windows build (due to our
> extensive use of Unix shell scripting in our test suite) takes 1h40m
> currently, and AppVeyor times out after 20 or 30 minutes. I could imagine
> that Travis times out after the same time, or maybe 60 minutes, which
> would still be too short. On Azure Pipelines, the maximum timeout (which
> can be configured via the azure-pipelines.yml file) is four hours IIRC.
> Plenty enough even for our test suite on Windows.
>
> > To summarize [1], Travis is offering an early version of adding Windows
> > to their list of supported builder operations systems. This brings the
> > list to macOS, Linux, and Windows, which I think satisfies what we would
> > like to regularly build git.git on.
>
> Honestly, I would love to have also FreeBSD and other platforms being
> tested. And with Azure Pipelines, I can make that happen (eventually), by
> adding another pool of VMs (given that I have a free $150/month Azure
> subscription, I'd use Azure VMs, of course). As long as a platform can run
> .NET Core software, it can run Azure Pipelines agents.
>
> With Travis, I don't think I can add private agent pools.

I think that's right.

> > Would we like to abandon Travis as our main CI service for upstream
> > git.git, and build on Azure Pipelines only? If so, I think that this is
> > an OK way to go, but I think that I would be opposed to having more than
> > one build system. (FWIW, we tend to _three_ for Git LFS, and it can be a
> > hassle at times).
>
> This question of abandoning Travis in favor of Azure Pipelines is a bit of
> a hornets' nest, as I really, really only want to bring the goodness of
> Azure Pipelines to git.git, and I am *clearly* biased, as I work at
> Microsoft.
>
> Which is the reason why I did not even hint at it in the cover letter, let
> alone included a patch to make it so.
>
> My patch series is purely about adding support for running CI/PR builds of
> https://github.com/git/git via Azure Pipelines.

I think that adding support for one CI system does carry the weight of
removing the other, for the sake of having few CI systems running at any
given time. In other words, even if you don't intend to start a
discussion about removing Travis, those that want run the smallest
number of services (including myself) will ask you about it ;-).

But I don't see a benefit to dragging this series through the mud by
spending too much time talking about Travis' future. I think that your
cover letter does a fine job to address the point, and we can revisit it
in the future if more people than just myself are opposed to running >1
CI service.

> > I see some benefit to sticking with Travis, since we already have a
> > build configuration that works there. But, you've done the work to
> > "port" that build configuration over to Azure, so perhaps the point is
> > moot.
>
> It is not so much a port, as an attempt to generalize our ci/* files.
>
> > > As Git's Windows builds (and tests!) take quite a bit of time, Travis often
> > > timed out, or somehow the trigger did not work, and for security reasons
> > > (the Windows builds are performed in a private pool of containers), the
> > > Windows builds are completely disabled for Pull Requests on GitHub.
> >
> > This would be a concession of [1], in my mind: is it possible to run the
> > tests on Windows in a time such that Travis will not time out?
>
> To be honest, I spent such a lot of time to get things to work on Azure
> Pipelines, *and* we get a nice view on the test failures there, too (which
> Travis will probably also offer soon, in response to what Azure Pipelines
> offer ;-)), I cannot really justify spending time on trying to make things
> work on Travis' Windows VMs, too. Especially when I have to expect to run
> into timeout issues anyway.

Agreed.

> > > As a special treat, this patch series adds the ability to present the
> > > outcome of Git's test suite as JUnit-style .xml files. This allows the Azure
> > > Pipelines build to present fun diagrams, trends, and makes it a lot easier
> > > to drill down to test failures than before. See for example
> > > https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details
> > > [https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details]
> > > (you can click on the label of the failed test, and then see the detailed
> > > output in the right pane).
> >
> > That's pretty cool. Travis doesn't support this (to the best of my
> > knowledge).
>
> Exactly.
>
> Plus, if things don't work in Azure Pipelines, I (or one of the other
> Microsoft employees among the core Git developers) can easily take a
> shortcut to the team and get things fixed. In my mind, that counts for a
> lot, too, especially given my own, frustrating personal experience with
> Travis.

Agreed, and thanks for your response.

> Ciao,
> Dscho
>
> > [1]: https://blog.travis-ci.com/2018-10-11-windows-early-release

Thanks,
Taylor

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 09/13] git-p4: use `test_atexit` to kill the daemon
  2018-10-15 10:12   ` [PATCH v2 09/13] git-p4: use `test_atexit` to kill the daemon Johannes Schindelin via GitGitGadget
  2018-10-15 11:00     ` Luke Diamand
@ 2018-10-16  3:34     ` Eric Sunshine
  2018-10-16  8:51       ` Johannes Schindelin
  2018-10-17 23:29     ` SZEDER Gábor
  2 siblings, 1 reply; 88+ messages in thread
From: Eric Sunshine @ 2018-10-16  3:34 UTC (permalink / raw)
  To: gitgitgadget; +Cc: Git List, Junio C Hamano, Johannes Schindelin

On Mon, Oct 15, 2018 at 6:12 AM Johannes Schindelin via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> This should be more reliable than the current method, and prepares the
> test suite for a consistent way to clean up before re-running the tests
> with different options.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
> @@ -134,6 +134,7 @@ check_sub_test_lib_test_err () {
> +cat >/dev/null <<\DDD
>  test_expect_success 'pretend we have a fully passing test suite' "
>         run_sub_test_lib_test full-pass '3 passing tests' <<-\\EOF &&
>         for i in 1 2 3
> @@ -820,6 +821,7 @@ test_expect_success 'tests clean up even on failures' "
>         > 1..2
>         EOF
>  "
> +DDD

Is this "DDD" here-doc leftover debugging goop?

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 12/13] README: add a build badge (status of the Azure Pipelines build)
  2018-10-15 10:12   ` [PATCH v2 12/13] README: add a build badge (status of the Azure Pipelines build) Johannes Schindelin via GitGitGadget
@ 2018-10-16  3:44     ` Eric Sunshine
  2018-10-16  8:48       ` Johannes Schindelin
  0 siblings, 1 reply; 88+ messages in thread
From: Eric Sunshine @ 2018-10-16  3:44 UTC (permalink / raw)
  To: gitgitgadget; +Cc: Git List, Junio C Hamano, Johannes Schindelin

On Mon, Oct 15, 2018 at 6:12 AM Johannes Schindelin via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> Just like so many other OSS projects, we now also have a build badge.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> diff --git a/README.md b/README.md
> @@ -1,3 +1,5 @@
> +[![Build Status](https:/dev.azure.com/git/git/_apis/build/status/test-git.git)](https://dev.azure.com/git/git/_build/latest?definitionId=2)

The first URL is broken "https:/..." rather than "https://...".

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services
  2018-10-15 15:06 ` [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Ævar Arnfjörð Bjarmason
  2018-10-15 15:42   ` Duy Nguyen
@ 2018-10-16  4:47   ` Junio C Hamano
  1 sibling, 0 replies; 88+ messages in thread
From: Junio C Hamano @ 2018-10-16  4:47 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Johannes Schindelin via GitGitGadget, git

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> I have not reviewed this in any detail, but incorporating this in some
> form or other seems like a no-brainer to me.
>
> If we have "free" (from the perspective of the project) CPU being
> offered by various CI setups let's use it.

Somebody else said in a separate subthread that there are people who
want to limit the CI, and if we pick this we may want to retire
Travis.  Naively I'd say the more the merrier (i.e. agreeing with
your above statement), but I may be missing something.

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services
  2018-10-15 18:33       ` Johannes Schindelin
  2018-10-16  0:24         ` Christian Couder
@ 2018-10-16  4:50         ` Junio C Hamano
  2018-10-16  9:14           ` Ævar Arnfjörð Bjarmason
  1 sibling, 1 reply; 88+ messages in thread
From: Junio C Hamano @ 2018-10-16  4:50 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Christian Couder, Nguyen Thai Ngoc Duy,
	Ævar Arnfjörð Bjarmason, gitgitgadget, git

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> AFAIR Junio does not push to github.com/git/git, it is an automatic
> mirror.
>
> GitLab could easily do the same.

It used to be in the early days but these days git/git and
gitster/git are updated in a same for loop that pushes to various
destinations.  You are correct that GitLab or any other hosting
sites could do the same polling and mirroring.  I am just too lazy
to open a new account at yet another hosting site to add that for
loop, but I may choose to when I am absolutely bored and nothing
else to do ;-).


^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 01/13] ci: rename the library of common functions
  2018-10-15 10:11   ` [PATCH v2 01/13] ci: rename the library of common functions Johannes Schindelin via GitGitGadget
@ 2018-10-16  5:26     ` Junio C Hamano
  0 siblings, 0 replies; 88+ messages in thread
From: Junio C Hamano @ 2018-10-16  5:26 UTC (permalink / raw)
  To: Johannes Schindelin via GitGitGadget; +Cc: git, Johannes Schindelin

"Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
writes:

> From: Johannes Schindelin <johannes.schindelin@gmx.de>
>
> The name is hard-coded to reflect that we use Travis CI for continuous
> testing.
>
> In the next commits, we will extend this to be able use Azure DevOps,
> too.
>
> So let's adjust the name to make it more generic.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---

With many pieces still shared across these two CI systems, this
looks like an excellent move.

I was wondering if the other extreme of moving ci/* to ci/travis/*,
introducing ci/azure/* and sharing nothing was needed, but being
able to share meaning amount of set-up sounds quite good.

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 06/13] Add a build definition for Azure DevOps
  2018-10-15 10:12   ` [PATCH v2 06/13] Add a build definition for Azure DevOps Johannes Schindelin via GitGitGadget
@ 2018-10-16  5:35     ` Junio C Hamano
  2018-10-16  9:58       ` Johannes Schindelin
  2018-10-16 19:12     ` SZEDER Gábor
  1 sibling, 1 reply; 88+ messages in thread
From: Junio C Hamano @ 2018-10-16  5:35 UTC (permalink / raw)
  To: Johannes Schindelin via GitGitGadget; +Cc: git, Johannes Schindelin

"Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
writes:

> Also, we make use of the shiny new feature we just introduced where the
> test suite can output JUnit-style .xml files. This information is made
> available in a nice UI that allows the viewer to filter by phase and/or
> test number, and to see trends such as: number of (failing) tests, time
> spent running the test suite, etc.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  azure-pipelines.yml   | 319 ++++++++++++++++++++++++++++++++++++++++++
>  ci/mount-fileshare.sh |  26 ++++
>  2 files changed, 345 insertions(+)
>  create mode 100644 azure-pipelines.yml
>  create mode 100755 ci/mount-fileshare.sh

I wonder if there is a need to keep what is tested by this and
Travis in sync in any way, but most of the logic is not defined in
these "steps" but implemented in ci/*.sh scripts to be shared, so it
would be OK, I guess.

> diff --git a/ci/mount-fileshare.sh b/ci/mount-fileshare.sh
> new file mode 100755
> index 0000000000..5fb5f74b70
> --- /dev/null
> +++ b/ci/mount-fileshare.sh
> @@ -0,0 +1,26 @@
> +#!/bin/sh
> +
> +die () {
> +	echo "$*" >&2
> +	exit 1
> +}
> +
> +test $# = 4 ||
> +die "Usage: $0 <share> <username> <password> <mountpoint"

Missing closing '>'.

> +
> +mkdir -p "$4" || die "Could not create $4"
> +
> +case "$(uname -s)" in
> +Linux)
> +	sudo mount -t cifs -o vers=3.0,username="$2",password="$3",dir_mode=0777,file_mode=0777,serverino "$1" "$4"
> +	;;
> +Darwin)
> +	pass="$(echo "$3" | sed -e 's/\//%2F/g' -e 's/+/%2B/g')" &&
> +	mount -t smbfs,soft "smb://$2:$pass@${1#//}" "$4"
> +	;;
> +*)
> +	die "No support for $(uname -s)"
> +	;;
> +esac ||
> +die "Could not mount $4"
> +

Trailing blank line.

Thanks.

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 12/13] README: add a build badge (status of the Azure Pipelines build)
  2018-10-16  3:44     ` Eric Sunshine
@ 2018-10-16  8:48       ` Johannes Schindelin
  0 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin @ 2018-10-16  8:48 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: gitgitgadget, Git List, Junio C Hamano

Hi Eric,


On Mon, 15 Oct 2018, Eric Sunshine wrote:

> On Mon, Oct 15, 2018 at 6:12 AM Johannes Schindelin via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
> > Just like so many other OSS projects, we now also have a build badge.
> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> > diff --git a/README.md b/README.md
> > @@ -1,3 +1,5 @@
> > +[![Build Status](https:/dev.azure.com/git/git/_apis/build/status/test-git.git)](https://dev.azure.com/git/git/_build/latest?definitionId=2)
> 
> The first URL is broken "https:/..." rather than "https://...".

I *swear* I fixed this already. Darn.

Thank you,
Dscho

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 09/13] git-p4: use `test_atexit` to kill the daemon
  2018-10-16  3:34     ` Eric Sunshine
@ 2018-10-16  8:51       ` Johannes Schindelin
  0 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin @ 2018-10-16  8:51 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: gitgitgadget, Git List, Junio C Hamano

Hi Eric,

On Mon, 15 Oct 2018, Eric Sunshine wrote:

> On Mon, Oct 15, 2018 at 6:12 AM Johannes Schindelin via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
> > This should be more reliable than the current method, and prepares the
> > test suite for a consistent way to clean up before re-running the tests
> > with different options.
> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> > diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
> > @@ -134,6 +134,7 @@ check_sub_test_lib_test_err () {
> > +cat >/dev/null <<\DDD
> >  test_expect_success 'pretend we have a fully passing test suite' "
> >         run_sub_test_lib_test full-pass '3 passing tests' <<-\\EOF &&
> >         for i in 1 2 3
> > @@ -820,6 +821,7 @@ test_expect_success 'tests clean up even on failures' "
> >         > 1..2
> >         EOF
> >  "
> > +DDD
> 
> Is this "DDD" here-doc leftover debugging goop?

Oy, oy, oy. This is definitely a left-over from debugging (as you can
imagine, it is pretty slow to run t0000-init.sh on Windows, and if I add a
test case, the development cycle is much faster with the trick you see
aboive). This left-over even made it into Git for Windows' `master`
branch! (Which is the reason I missed it before contributing v2).

Will fix,
Dscho

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services
  2018-10-16  4:50         ` Junio C Hamano
@ 2018-10-16  9:14           ` Ævar Arnfjörð Bjarmason
  2018-10-18  2:03             ` Junio C Hamano
  0 siblings, 1 reply; 88+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2018-10-16  9:14 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Johannes Schindelin, Christian Couder,
	Nguyễn Thái Ngọc Duy,
	Johannes Schindelin via GitGitGadget, Git Mailing List

On Tue, Oct 16, 2018 at 6:50 AM Junio C Hamano <gitster@pobox.com> wrote:
>
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> > AFAIR Junio does not push to github.com/git/git, it is an automatic
> > mirror.
> >
> > GitLab could easily do the same.
>
> It used to be in the early days but these days git/git and
> gitster/git are updated in a same for loop that pushes to various
> destinations.  You are correct that GitLab or any other hosting
> sites could do the same polling and mirroring.  I am just too lazy
> to open a new account at yet another hosting site to add that for
> loop, but I may choose to when I am absolutely bored and nothing
> else to do ;-).

Do you mind if I squat gitlab.com/git/git in the meantime (i.e. create
an org etc.) and have it mirror github.com/git/git?, I'll hand the
keys over to you if/when you'd like to update your for-loop :)

In the meantime sometimes GitLab's UI s better than GitHub's, or the
other way around, so it would be useful to have it there as mirror for
that reason alone, and then we can always experiment with a
.gitlab-ci.yml in-repo.

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 09/13] git-p4: use `test_atexit` to kill the daemon
  2018-10-15 20:19         ` Luke Diamand
@ 2018-10-16  9:39           ` Johannes Schindelin
  0 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin @ 2018-10-16  9:39 UTC (permalink / raw)
  To: Luke Diamand; +Cc: gitgitgadget, Git Users, Junio C Hamano

Hi Luke,

On Mon, 15 Oct 2018, Luke Diamand wrote:

> On Mon, 15 Oct 2018 at 16:02, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > Hi Luke,
> >
> > On Mon, 15 Oct 2018, Luke Diamand wrote:
> >
> > > On Mon, 15 Oct 2018 at 11:12, Johannes Schindelin via GitGitGadget
> > > <gitgitgadget@gmail.com> wrote:
> > > >
> > > > From: Johannes Schindelin <johannes.schindelin@gmx.de>
> > > >
> > > > This should be more reliable than the current method, and prepares the
> > > > test suite for a consistent way to clean up before re-running the tests
> > > > with different options.
> > > >
> > >
> > > I'm finding that it's leaving p4d processes lying around.
> >
> > That's a bummer!
> >
> > > e.g.
> > >
> > > $ ./t9820-git-p4-editor-handling.sh
> > > <passes>
> > > $ ./t9820-git-p4-editor-handling.sh
> > > <fails>
> >
> > Since I do not currently have a setup with p4d installed, can you run that
> > with `sh -x ...` and see whether this code path is hit?
> 
> All you need to do is to put p4 and p4d in your PATH.
> 
> https://www.perforce.com/downloads/helix-core-p4d
> https://www.perforce.com/downloads/helix-command-line-client-p4

I did download p4d.exe and p4.exe to $HOME/custom/p4/ (similar to the way
ci/install-dependencies.sh does it), from

	http://filehost.perforce.com/perforce/r18.1/bin.ntx64/p4d.exe
	http://filehost.perforce.com/perforce/r18.1/bin.ntx64/p4.exe

and then prefixed PATH with $HOME/custom/p4, just to run t9820. However,
it does not work here at all:

-- snip --
[...]
++ git p4 clone '--dest=/usr/src/git/azure-pipelines/t/trash directory.t9820-git-p4-editor-handling/git' //depot
Usage: git-p4 clone [options] //depot/path[@revRange]

[...]
-- snap --

I *think* the reason for that is that the MSYS path mangling kicks in when
`git.exe` is called with the `//depot` argument (the leading `//` can be
used in MSYS and MSYS2 to indicate that the non-MSYS .exe wants an
argument that starts with a single slash, but is not a path that needs to
be translated to a Windows path).

So I did the next best thing I could do: try things in the Windows
Subsystem for Linux (AKA Bash on Ubuntu on Windows).

> The server is free to use for a small number of users, you don't need
> to do anything to make it go.
> 
> 
> >
> >  test_done () {
> >         GIT_EXIT_OK=t
> >
> > +       test -n "$immediate" || test_atexit_handler

On a slight tangent: I made up my mind on the `test -n "$immediate"` part:
I will skip that. The daemon should be stopped also when `-i` was passed
to the test script (to stop at the first failing test case). There is
very, very little use in keeping the daemon alive in that case.

The commit message of "tests: introduce `test_atexit`" already made the
case for that, but somehow I had not made up my mind yet.

> > +
> 
> + test -n
> + test_atexit_handler
> ./t9820-git-p4-editor-handling.sh: 764:
> ./t9820-git-p4-editor-handling.sh: test_atexit_handler: not found
> 
> Is that expected?

No, that is not expected. For me, it looks quite a bit different, though:

-- snip --
[...]
+ test_done
+ GIT_EXIT_OK=t
+ test -n
+ test_atexit_handler
+ test : != { kill_p4d
                } && (exit "$eval_ret"); eval_ret=$?; :
+ setup_malloc_check
+ MALLOC_CHECK_=3 MALLOC_PERTURB_=165
+ export MALLOC_CHECK_ MALLOC_PERTURB_
+ test_eval_ { kill_p4d
                } && (exit "$eval_ret"); eval_ret=$?; :
+ test_eval_inner_ { kill_p4d
                } && (exit "$eval_ret"); eval_ret=$?; :
+ eval
                want_trace && set -x
                { kill_p4d
                } && (exit "$eval_ret"); eval_ret=$?; :
+ want_trace
+ test  = t
+ kill_p4d
+ cat /home/wsl/git/wip/t/trash directory.t9820-git-p4-editor-handling/p4d.pid
+ pid=20093
+ retry_until_fail kill 20093
+ time_in_seconds
+ cd /
+ /usr/bin/python -c import time; print(int(time.time()))
+ timeout=1539681637
+ kill 20093
+ time_in_seconds
+ cd /
+ /usr/bin/python -c import time; print(int(time.time()))
+ test 1539681577 -gt 1539681637
+ sleep 1
+ true
+ time_in_seconds
+ cd /
+ /usr/bin/python -c import time; print(int(time.time()))
+ test 1539681578 -gt 1539681871
+ sleep 1
+ kill 20093
+ retry_until_fail kill -9 20093
+ time_in_seconds
+ cd /
+ /usr/bin/python -c import time; print(int(time.time()))
+ timeout=1539681638
+ kill -9 20093
+ test_must_fail kill 20093
+ _test_ok=
+ kill 20093
+ exit_code=1
+ test 1 -eq 0
+ test_match_signal 13 1
+ test 1 = 141
+ test 1 = 269
+ return 1
+ test 1 -gt 129
+ test 1 -eq 127
+ test 1 -eq 126
+ return 0
[...]
-- snap --

As you can see, it works quite well over here.

Maybe I could trouble you to fetch the `vsts-ci` branch from
https://github.com/dscho/git and test things on that one, just in case
that anything important got "lost in the mail"?

> >         if test -n "$write_junit_xml" && test -n "$junit_xml_path"
> >         then
> >
> > > And also
> > >
> > > $ ./t9800-git-p4-basic.sh
> > > <starts running tests, but I get bored easily>
> > > Ctrl-C
> >
> > Oh, you're right.

So I tested this in WSL, and the relevant part of the output is this:

-- snip --
[...]
+ eval test_prereq_lazily_UTF8_NFD_TO_NFC=$2
+ test_prereq_lazily_UTF8_NFD_TO_NFC=
        # check whether FS converts nfd unicode to nfc
        au^CTraceback (most recent call last):
  File "/home/virtualbox/git/wip/git-p4", line 4143, in <module>
    main()
  File "/home/virtualbox/git/wip/git-p4", line 4137, in main
    if not cmd.run(args):
  File "/home/virtualbox/git/wip/git-p4", line 3925, in run
    if not P4Sync.run(self, depotPaths):
  File "/home/virtualbox/git/wip/git-p4", line 3824, in run
    system(["git", "symbolic-ref", head_ref, self.branch])
  File "/home/virtualbox/git/wip/git-p4", line 278, in system
    retcode = subprocess.call(cmd, shell=expand)
  File "/usr/lib/python2.7/subprocess.py", line 523, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/usr/lib/python2.7/subprocess.py", line 1392, in wait
    pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
  File "/usr/lib/python2.7/subprocess.py", line 476, in _eintr_retry_call
    return func(*args)
KeyboardInterrupt
+ exit 130
+ die
+ code=130
+ test_atexit_handler
+ test : != { kill_p4d
                } && (exit "$eval_ret"); eval_ret=$?; :
[...]
-- snap --

As you can see, the Ctrl+C managed to call `die` very well, without my
proposed change:

> > I think I need to do something in this line:
> >
> >         trap 'exit $?' INT
> >
> > in t/test-lib.sh, something like
> >
> >         trap 'exit_code=$?; test_atexit_handler; exit $exit_code' INT
> >
> > would you agree? (And: could you test?)
> 
> Not sure.

I *think* what happens is that the INT trap already calls `exit 130`
alright, and then the EXIT trap comes into effect, as planned.

Just to make sure that it was not Python that caused the exit code 130, I
tried it again, and luckily hit a spot where shell code was interpreted:

-- snip --
[...]
+ test_prereq_lazily_LONG_IS_64BIT=
        test 8 -le "$(build_option sizeof-long)"

+ test_lazy_^C+ exit 130
+ die
+ code=130
+ test_atexit_handler
+ test : != { kill_p4d
                } && (exit "$eval_ret"); eval_ret=$?; :
[...]
-- snap --

Same here. The Ctrl+C triggers an `exit 130`, this time I am certain that
it comes from the INT trap, and that `die` in turn is most definitely from
the EXIT trap, as intended.

My tests were performed with Dash, BTW, not with Bash:

-- snip --
$ dpkg -S $(which sh)
diversion by dash from: /bin/sh
diversion by dash to: /bin/sh.distrib
dash: /bin/sh
-- snap --

But I also repeated the tests with Bash, with almost the same result: the
only difference was that I did not manage to get the traces of
test_atexit_handler being called, but the p4d was no longer alive, so it
must have been killed as intended.

> Send me a patch and I can try it out.

I would *love* to figure out what is happening in your setup, and to fix
it. In my hands, the patches work, though...

Ciao,
Dscho

> 
> Thanks,
> Luke
> 

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 13/13] travis: fix skipping tagged releases
  2018-10-15 10:12   ` [PATCH v2 13/13] travis: fix skipping tagged releases Johannes Schindelin via GitGitGadget
@ 2018-10-16  9:40     ` SZEDER Gábor
  2018-10-16 12:45       ` Johannes Schindelin
  0 siblings, 1 reply; 88+ messages in thread
From: SZEDER Gábor @ 2018-10-16  9:40 UTC (permalink / raw)
  To: Johannes Schindelin via GitGitGadget
  Cc: git, Junio C Hamano, Johannes Schindelin

On Mon, Oct 15, 2018 at 03:12:17AM -0700, Johannes Schindelin via GitGitGadget wrote:
> From: Johannes Schindelin <johannes.schindelin@gmx.de>
> 
> When building a PR, TRAVIS_BRANCH refers to the *target branch*.
> Therefore, if a PR targets `master`, and `master` happened to be tagged,
> we skipped the build by mistake.
> 
> Fix this by using TRAVIS_PULL_REQUEST_BRANCH (i.e. the *source branch*)
> when available, falling back to TRAVIS_BRANCH (i.e. for CI builds, also
> known as "push builds").

This all makes sense, but this patch is fixing a long-standing issue
in our Travis CI build scripts (present since 09f5e9746c (travis-ci:
skip a branch build if equal tag is present, 2017-09-10)), so it
should be the first in the series.  So it could be picked up and
perhaps even graduated faster than the rest of this patch series.

> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  ci/lib.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/ci/lib.sh b/ci/lib.sh
> index 584abcd529..e1858ae609 100755
> --- a/ci/lib.sh
> +++ b/ci/lib.sh
> @@ -3,7 +3,7 @@
>  if test true = "$TRAVIS"
>  then
>  	# We are running within Travis CI
> -	CI_BRANCH="$TRAVIS_BRANCH"
> +	CI_BRANCH="${TRAVIS_PULL_REQUEST_BRANCH:-$TRAVIS_BRANCH}"
>  	CI_COMMIT="$TRAVIS_COMMIT"
>  	CI_JOB_ID="$TRAVIS_JOB_ID"
>  	CI_JOB_NUMBER="$TRAVIS_JOB_NUMBER"
> -- 
> gitgitgadget

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 05/13] ci/lib.sh: add support for Azure Pipelines
  2018-10-15 10:12   ` [PATCH v2 05/13] ci/lib.sh: add support for Azure Pipelines Johannes Schindelin via GitGitGadget
@ 2018-10-16  9:43     ` SZEDER Gábor
  2018-10-16 12:53       ` Johannes Schindelin
  0 siblings, 1 reply; 88+ messages in thread
From: SZEDER Gábor @ 2018-10-16  9:43 UTC (permalink / raw)
  To: Johannes Schindelin via GitGitGadget
  Cc: git, Junio C Hamano, Johannes Schindelin

On Mon, Oct 15, 2018 at 03:12:05AM -0700, Johannes Schindelin via GitGitGadget wrote:
> From: Johannes Schindelin <johannes.schindelin@gmx.de>
> 
> This patch introduces a conditional arm that defines some environment
> variables and a function that displays the URL given the job id (to
> identify previous runs for known-good trees).
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  ci/lib.sh | 23 +++++++++++++++++++++++
>  1 file changed, 23 insertions(+)
> 
> diff --git a/ci/lib.sh b/ci/lib.sh
> index 8532555b4e..584abcd529 100755
> --- a/ci/lib.sh
> +++ b/ci/lib.sh
> @@ -19,6 +19,29 @@ then
>  	BREW_INSTALL_PACKAGES="git-lfs gettext"

So we install these two additional packages in the macOS build jobs on
Travis CI ...

>  	export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
>  	export GIT_TEST_OPTS="--verbose-log -x --immediate"
> +elif test -n "$SYSTEM_TASKDEFINITIONSURI"
> +then
> +	# We are running in Azure Pipelines
> +	CI_BRANCH="$BUILD_SOURCEBRANCH"
> +	CI_COMMIT="$BUILD_SOURCEVERSION"
> +	CI_JOB_ID="$BUILD_BUILDID"
> +	CI_JOB_NUMBER="$BUILD_BUILDNUMBER"
> +	CI_OS_NAME="$(echo "$AGENT_OS" | tr A-Z a-z)"
> +	test darwin != "$CI_OS_NAME" || CI_OS_NAME=osx
> +	CI_REPO_SLUG="$(expr "$BUILD_REPOSITORY_URI" : '.*/\([^/]*/[^/]*\)$')"
> +	CC="${CC:-gcc}"
> +
> +	# use a subdirectory of the cache dir (because the file share is shared
> +	# among *all* phases)
> +	cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME"
> +
> +	url_for_job_id () {
> +		echo "$SYSTEM_TASKDEFINITIONSURI$SYSTEM_TEAMPROJECT/_build/results?buildId=$1"
> +	}
> +
> +	BREW_INSTALL_PACKAGES=

... but not on Azure Pipelines.  Is this mere oversight or
intentional?  If it's intentional, then I think the commit message
should mention why.

> +	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
> +	export GIT_TEST_OPTS="--quiet --write-junit-xml"
>  fi
>  
>  skip_branch_tip_with_tag () {
> -- 
> gitgitgadget
> 

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 06/13] Add a build definition for Azure DevOps
  2018-10-16  5:35     ` Junio C Hamano
@ 2018-10-16  9:58       ` Johannes Schindelin
  0 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin @ 2018-10-16  9:58 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Johannes Schindelin via GitGitGadget, git

Hi Junio,

On Tue, 16 Oct 2018, Junio C Hamano wrote:

> "Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
> writes:
> 
> > Also, we make use of the shiny new feature we just introduced where the
> > test suite can output JUnit-style .xml files. This information is made
> > available in a nice UI that allows the viewer to filter by phase and/or
> > test number, and to see trends such as: number of (failing) tests, time
> > spent running the test suite, etc.
> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  azure-pipelines.yml   | 319 ++++++++++++++++++++++++++++++++++++++++++
> >  ci/mount-fileshare.sh |  26 ++++
> >  2 files changed, 345 insertions(+)
> >  create mode 100644 azure-pipelines.yml
> >  create mode 100755 ci/mount-fileshare.sh
> 
> I wonder if there is a need to keep what is tested by this and
> Travis in sync in any way, but most of the logic is not defined in
> these "steps" but implemented in ci/*.sh scripts to be shared, so it
> would be OK, I guess.

Indeed, that was my intention.

These ci scripts are not only useful for Travis and Azure Pipelines, after
all, but also a good documentation how to test locally. For example, to
repeat Luke's Perforce testing, I could simply use the URLs listed in
those ci scripts to get almost the same setup locally (close enough for
testing).

So it is not only about sharing, and ease of maintenance, but it is also
about documenting.

And yes, sharing means that we do not have to waste brain cycles on
keeping things in sync.

> > diff --git a/ci/mount-fileshare.sh b/ci/mount-fileshare.sh
> > new file mode 100755
> > index 0000000000..5fb5f74b70
> > --- /dev/null
> > +++ b/ci/mount-fileshare.sh
> > @@ -0,0 +1,26 @@
> > +#!/bin/sh
> > +
> > +die () {
> > +	echo "$*" >&2
> > +	exit 1
> > +}
> > +
> > +test $# = 4 ||
> > +die "Usage: $0 <share> <username> <password> <mountpoint"
> 
> Missing closing '>'.

Thanks!

> > +
> > +mkdir -p "$4" || die "Could not create $4"
> > +
> > +case "$(uname -s)" in
> > +Linux)
> > +	sudo mount -t cifs -o vers=3.0,username="$2",password="$3",dir_mode=0777,file_mode=0777,serverino "$1" "$4"
> > +	;;
> > +Darwin)
> > +	pass="$(echo "$3" | sed -e 's/\//%2F/g' -e 's/+/%2B/g')" &&
> > +	mount -t smbfs,soft "smb://$2:$pass@${1#//}" "$4"
> > +	;;
> > +*)
> > +	die "No support for $(uname -s)"
> > +	;;
> > +esac ||
> > +die "Could not mount $4"
> > +
> 
> Trailing blank line.
> 
> Thanks.

Thank you, both issues will be fixed in the next iteration,
Dscho

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 10/13] tests: include detailed trace logs with --write-junit-xml upon failure
  2018-10-15 10:12   ` [PATCH v2 10/13] tests: include detailed trace logs with --write-junit-xml upon failure Johannes Schindelin via GitGitGadget
@ 2018-10-16 10:04     ` SZEDER Gábor
  2018-10-16 13:02       ` Johannes Schindelin
  0 siblings, 1 reply; 88+ messages in thread
From: SZEDER Gábor @ 2018-10-16 10:04 UTC (permalink / raw)
  To: Johannes Schindelin via GitGitGadget
  Cc: git, Junio C Hamano, Johannes Schindelin

On Mon, Oct 15, 2018 at 03:12:12AM -0700, Johannes Schindelin via GitGitGadget wrote:
> From: Johannes Schindelin <johannes.schindelin@gmx.de>
> 
> The JUnit XML format lends itself to be presented in a powerful UI,
> where you can drill down to the information you are interested in very
> quickly.
> 
> For test failures, this usually means that you want to see the detailed
> trace of the failing tests.
> 
> With Travis CI, we passed the `--verbose-log` option to get those
> traces. However, that seems excessive, as we do not need/use the logs in

As someone who has dug into a few occasional failures found by Travis
CI, I'd say that the output of '--verbose-log -x' is not excessive,
but downright essential.

> almost all of those cases: only when a test fails do we have a way to
> include the trace.
> 
> So let's do something different when using Azure DevOps: let's run all
> the tests with `--quiet` first, and only if a failure is encountered,
> try to trace the commands as they are executed.
> 
> Of course, we cannot turn on `--verbose-log` after the fact. So let's
> just re-run the test with all the same options, adding `--verbose-log`.
> And then munging the output file into the JUnit XML on the fly.
> 
> Note: there is an off chance that re-running the test in verbose mode
> "fixes" the failures (and this does happen from time to time!). That is
> a possibility we should be able to live with.

Any CI system worth its salt should provide as much information about
any failures as possible, especially when it was lucky enough to
stumble upon a rare and hard to reproduce non-deterministic failure.

> Ideally, we would label
> this as "Passed upon rerun", and Azure Pipelines even know about that
> outcome, but it is not available when using the JUnit XML format for
> now:
> https://github.com/Microsoft/azure-pipelines-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.cs
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines
  2018-10-15 14:22   ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Taylor Blau
  2018-10-15 14:55     ` Johannes Schindelin
@ 2018-10-16 10:30     ` SZEDER Gábor
  1 sibling, 0 replies; 88+ messages in thread
From: SZEDER Gábor @ 2018-10-16 10:30 UTC (permalink / raw)
  To: Taylor Blau; +Cc: Johannes Schindelin via GitGitGadget, git, Junio C Hamano

On Mon, Oct 15, 2018 at 07:22:15AM -0700, Taylor Blau wrote:
> Would we like to abandon Travis as our main CI service for upstream
> git.git, and build on Azure Pipelines only?

It's not only about "upstream git.git", but also about contributors,
who might have enabled Travis CI integration on their forks on GitHub.
Having a '.travis.yml' and associated 'ci/*' scripts in git.git makes
it possible for them to easily build and test their branches on their
own.


^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines
  2018-10-16  2:53       ` Taylor Blau
@ 2018-10-16 11:55         ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 88+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2018-10-16 11:55 UTC (permalink / raw)
  To: Taylor Blau
  Cc: Johannes Schindelin, Johannes Schindelin via GitGitGadget,
	Git Mailing List, Junio C Hamano

On Tue, Oct 16, 2018 at 4:55 AM Taylor Blau <me@ttaylorr.com> wrote:
>
> On Mon, Oct 15, 2018 at 04:55:25PM +0200, Johannes Schindelin wrote:
> > Another really good reason for me to do this is that I can prod the Azure
> > Pipelines team directly. And I even get an answer, usually within minutes.
> > Which is a lot faster than the Travis team answers my questions, which
> > is... not yet? (I tried to get in contact with them in late 2015 or early
> > 2016, and I tried again a year later, and then a couple of months later,
> > and I have yet to hear back.)
>
> Certainly a good reason. To be clear/fair, I've sent in a number of
> support tickets to Travis CI over the years, and have always been
> responded to in a short amount of time with helpful answers. I think
> that we would really be fine in either case, TBH.

Was this in the context of "I'm this random dude using Travis" or "as
you guys know I work for GitHub, your biggest? customer..." ? :)

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 13/13] travis: fix skipping tagged releases
  2018-10-16  9:40     ` SZEDER Gábor
@ 2018-10-16 12:45       ` Johannes Schindelin
  0 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin @ 2018-10-16 12:45 UTC (permalink / raw)
  To: SZEDER Gábor
  Cc: Johannes Schindelin via GitGitGadget, git, Junio C Hamano

[-- Attachment #1: Type: text/plain, Size: 1559 bytes --]

Hi Junio,

On Tue, 16 Oct 2018, SZEDER Gábor wrote:

> On Mon, Oct 15, 2018 at 03:12:17AM -0700, Johannes Schindelin via GitGitGadget wrote:
> > From: Johannes Schindelin <johannes.schindelin@gmx.de>
> > 
> > When building a PR, TRAVIS_BRANCH refers to the *target branch*.
> > Therefore, if a PR targets `master`, and `master` happened to be tagged,
> > we skipped the build by mistake.
> > 
> > Fix this by using TRAVIS_PULL_REQUEST_BRANCH (i.e. the *source branch*)
> > when available, falling back to TRAVIS_BRANCH (i.e. for CI builds, also
> > known as "push builds").
> 
> This all makes sense, but this patch is fixing a long-standing issue
> in our Travis CI build scripts (present since 09f5e9746c (travis-ci:
> skip a branch build if equal tag is present, 2017-09-10)), so it
> should be the first in the series.  So it could be picked up and
> perhaps even graduated faster than the rest of this patch series.

Makes sense.

Thanks,
Dscho

> 
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  ci/lib.sh | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/ci/lib.sh b/ci/lib.sh
> > index 584abcd529..e1858ae609 100755
> > --- a/ci/lib.sh
> > +++ b/ci/lib.sh
> > @@ -3,7 +3,7 @@
> >  if test true = "$TRAVIS"
> >  then
> >  	# We are running within Travis CI
> > -	CI_BRANCH="$TRAVIS_BRANCH"
> > +	CI_BRANCH="${TRAVIS_PULL_REQUEST_BRANCH:-$TRAVIS_BRANCH}"
> >  	CI_COMMIT="$TRAVIS_COMMIT"
> >  	CI_JOB_ID="$TRAVIS_JOB_ID"
> >  	CI_JOB_NUMBER="$TRAVIS_JOB_NUMBER"
> > -- 
> > gitgitgadget
> 
^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 05/13] ci/lib.sh: add support for Azure Pipelines
  2018-10-16  9:43     ` SZEDER Gábor
@ 2018-10-16 12:53       ` Johannes Schindelin
  0 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin @ 2018-10-16 12:53 UTC (permalink / raw)
  To: SZEDER Gábor
  Cc: Johannes Schindelin via GitGitGadget, git, Junio C Hamano

[-- Attachment #1: Type: text/plain, Size: 2257 bytes --]

Hi Gábor,

On Tue, 16 Oct 2018, SZEDER Gábor wrote:

> On Mon, Oct 15, 2018 at 03:12:05AM -0700, Johannes Schindelin via GitGitGadget wrote:
> > From: Johannes Schindelin <johannes.schindelin@gmx.de>
> > 
> > This patch introduces a conditional arm that defines some environment
> > variables and a function that displays the URL given the job id (to
> > identify previous runs for known-good trees).
> > 
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  ci/lib.sh | 23 +++++++++++++++++++++++
> >  1 file changed, 23 insertions(+)
> > 
> > diff --git a/ci/lib.sh b/ci/lib.sh
> > index 8532555b4e..584abcd529 100755
> > --- a/ci/lib.sh
> > +++ b/ci/lib.sh
> > @@ -19,6 +19,29 @@ then
> >  	BREW_INSTALL_PACKAGES="git-lfs gettext"
> 
> So we install these two additional packages in the macOS build jobs on
> Travis CI ...
> 
> >  	export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
> >  	export GIT_TEST_OPTS="--verbose-log -x --immediate"
> > +elif test -n "$SYSTEM_TASKDEFINITIONSURI"
> > +then
> > +	# We are running in Azure Pipelines
> > +	CI_BRANCH="$BUILD_SOURCEBRANCH"
> > +	CI_COMMIT="$BUILD_SOURCEVERSION"
> > +	CI_JOB_ID="$BUILD_BUILDID"
> > +	CI_JOB_NUMBER="$BUILD_BUILDNUMBER"
> > +	CI_OS_NAME="$(echo "$AGENT_OS" | tr A-Z a-z)"
> > +	test darwin != "$CI_OS_NAME" || CI_OS_NAME=osx
> > +	CI_REPO_SLUG="$(expr "$BUILD_REPOSITORY_URI" : '.*/\([^/]*/[^/]*\)$')"
> > +	CC="${CC:-gcc}"
> > +
> > +	# use a subdirectory of the cache dir (because the file share is shared
> > +	# among *all* phases)
> > +	cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME"
> > +
> > +	url_for_job_id () {
> > +		echo "$SYSTEM_TASKDEFINITIONSURI$SYSTEM_TEAMPROJECT/_build/results?buildId=$1"
> > +	}
> > +
> > +	BREW_INSTALL_PACKAGES=
> 
> ... but not on Azure Pipelines.  Is this mere oversight or
> intentional?  If it's intentional, then I think the commit message
> should mention why.

Both packages are already available on Azure Pipelines' VMs. I will
mention this in the commit message.

Thanks,
Dscho

> 
> > +	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
> > +	export GIT_TEST_OPTS="--quiet --write-junit-xml"
> >  fi
> >  
> >  skip_branch_tip_with_tag () {
> > -- 
> > gitgitgadget
> > 
> 
^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 10/13] tests: include detailed trace logs with --write-junit-xml upon failure
  2018-10-16 10:04     ` SZEDER Gábor
@ 2018-10-16 13:02       ` Johannes Schindelin
  2018-10-16 16:03         ` SZEDER Gábor
  0 siblings, 1 reply; 88+ messages in thread
From: Johannes Schindelin @ 2018-10-16 13:02 UTC (permalink / raw)
  To: SZEDER Gábor
  Cc: Johannes Schindelin via GitGitGadget, git, Junio C Hamano

[-- Attachment #1: Type: text/plain, Size: 3140 bytes --]

Hi Gábor,

On Tue, 16 Oct 2018, SZEDER Gábor wrote:

> On Mon, Oct 15, 2018 at 03:12:12AM -0700, Johannes Schindelin via GitGitGadget wrote:
> > From: Johannes Schindelin <johannes.schindelin@gmx.de>
> > 
> > The JUnit XML format lends itself to be presented in a powerful UI,
> > where you can drill down to the information you are interested in very
> > quickly.
> > 
> > For test failures, this usually means that you want to see the detailed
> > trace of the failing tests.
> > 
> > With Travis CI, we passed the `--verbose-log` option to get those
> > traces. However, that seems excessive, as we do not need/use the logs in
> 
> As someone who has dug into a few occasional failures found by Travis
> CI, I'd say that the output of '--verbose-log -x' is not excessive,
> but downright essential.

I agree that the output is essential for drilling down into failures. This
paragraph, however, talks about the general case: where there are *no*
failures. See here:

> > almost all of those cases: only when a test fails do we have a way to
> > include the trace.
> > 
> > So let's do something different when using Azure DevOps: let's run all
> > the tests with `--quiet` first, and only if a failure is encountered,
> > try to trace the commands as they are executed.
> > 
> > Of course, we cannot turn on `--verbose-log` after the fact. So let's
> > just re-run the test with all the same options, adding `--verbose-log`.
> > And then munging the output file into the JUnit XML on the fly.
> > 
> > Note: there is an off chance that re-running the test in verbose mode
> > "fixes" the failures (and this does happen from time to time!). That is
> > a possibility we should be able to live with.
> 
> Any CI system worth its salt should provide as much information about
> any failures as possible, especially when it was lucky enough to
> stumble upon a rare and hard to reproduce non-deterministic failure.

I would agree with you if more people started to pay attention to our CI
failures. And if we had some sort of a development model where a CI
failure would halt development on that particular topic until the failure
is fixed, with the responsibility assigned to somebody to fix it.

This is not the case here, though. pu is broken for ages, at least on
Windows, and even a *single* topic is enough to do that. And this is even
worse with flakey tests. I cannot remember *how often* I saw CI failures
in t5570-git-daemon.sh, for example. It is rare enough that it is obvious
that this is a problem of the *regression test*, rather than a problem of
the code that is to be tested.

So I would suggest to go forward with my proposed strategy for the moment,
right up until the time when we have had the resources to fix t5570, for
starters.

Ciao,
Dscho

> > Ideally, we would label this as "Passed upon rerun", and Azure
> > Pipelines even know about that outcome, but it is not available when
> > using the JUnit XML format for now:
> > https://github.com/Microsoft/azure-pipelines-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.cs
> > 
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> 
^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 10/13] tests: include detailed trace logs with --write-junit-xml upon failure
  2018-10-16 13:02       ` Johannes Schindelin
@ 2018-10-16 16:03         ` SZEDER Gábor
  2018-10-16 20:53           ` Johannes Schindelin
  0 siblings, 1 reply; 88+ messages in thread
From: SZEDER Gábor @ 2018-10-16 16:03 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Johannes Schindelin via GitGitGadget, git, Junio C Hamano

On Tue, Oct 16, 2018 at 03:02:38PM +0200, Johannes Schindelin wrote:
> Hi Gábor,
> 
> On Tue, 16 Oct 2018, SZEDER Gábor wrote:
> 
> > On Mon, Oct 15, 2018 at 03:12:12AM -0700, Johannes Schindelin via GitGitGadget wrote:
> > > From: Johannes Schindelin <johannes.schindelin@gmx.de>
> > > 
> > > The JUnit XML format lends itself to be presented in a powerful UI,
> > > where you can drill down to the information you are interested in very
> > > quickly.
> > > 
> > > For test failures, this usually means that you want to see the detailed
> > > trace of the failing tests.
> > > 
> > > With Travis CI, we passed the `--verbose-log` option to get those
> > > traces. However, that seems excessive, as we do not need/use the logs in
> > 
> > As someone who has dug into a few occasional failures found by Travis
> > CI, I'd say that the output of '--verbose-log -x' is not excessive,
> > but downright essential.
> 
> I agree that the output is essential for drilling down into failures. This
> paragraph, however, talks about the general case: where there are *no*
> failures. See here:

But you don't know in advance whether there will be any failures or
not, so it only makes sense to run all tests with '--verbose-log -x'
by default, just in case a Heisenbug decides to make an appearance.

> > > almost all of those cases: only when a test fails do we have a way to
> > > include the trace.
> > > 
> > > So let's do something different when using Azure DevOps: let's run all
> > > the tests with `--quiet` first, and only if a failure is encountered,
> > > try to trace the commands as they are executed.
> > > 
> > > Of course, we cannot turn on `--verbose-log` after the fact. So let's
> > > just re-run the test with all the same options, adding `--verbose-log`.
> > > And then munging the output file into the JUnit XML on the fly.
> > > 
> > > Note: there is an off chance that re-running the test in verbose mode
> > > "fixes" the failures (and this does happen from time to time!). That is
> > > a possibility we should be able to live with.
> > 
> > Any CI system worth its salt should provide as much information about
> > any failures as possible, especially when it was lucky enough to
> > stumble upon a rare and hard to reproduce non-deterministic failure.
> 
> I would agree with you if more people started to pay attention to our CI
> failures. And if we had some sort of a development model where a CI
> failure would halt development on that particular topic until the failure
> is fixed, with the responsibility assigned to somebody to fix it.
> 
> This is not the case here, though. pu is broken for ages, at least on
> Windows, and even a *single* topic is enough to do that. And this is even
> worse with flakey tests. I cannot remember *how often* I saw CI failures
> in t5570-git-daemon.sh, for example. It is rare enough that it is obvious
> that this is a problem of the *regression test*, rather than a problem of
> the code that is to be tested.

Some occasional failures in t5570 are actually caused by issues in Git
on certain platforms:

  https://public-inbox.org/git/CAM0VKj=MCS+cmOgzf_XyPeb+qZrFmuMH52-PV_NDMZA9X+rRoA@mail.gmail.com/T/#u

> So I would suggest to go forward with my proposed strategy for the moment,
> right up until the time when we have had the resources to fix t5570, for
> starters.

I don't really understand what the occasional failures in t5570 have
to do with the amount of information a CI system should gather about
failures in general.  Or how many people pay attention to it, or what
kind of development model we have, for that matter.  The way I see it
these are unrelated issues, and a CI system should always provide as
much information about failures as possible.  If only a few people pay
attention to it, then for the sake of those few.


> > > Ideally, we would label this as "Passed upon rerun", and Azure
> > > Pipelines even know about that outcome, but it is not available when
> > > using the JUnit XML format for now:
> > > https://github.com/Microsoft/azure-pipelines-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.cs
> > > 
> > > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > 


^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 06/13] Add a build definition for Azure DevOps
  2018-10-15 10:12   ` [PATCH v2 06/13] Add a build definition for Azure DevOps Johannes Schindelin via GitGitGadget
  2018-10-16  5:35     ` Junio C Hamano
@ 2018-10-16 19:12     ` SZEDER Gábor
  2018-10-17 14:58       ` Johannes Schindelin
  1 sibling, 1 reply; 88+ messages in thread
From: SZEDER Gábor @ 2018-10-16 19:12 UTC (permalink / raw)
  To: Johannes Schindelin via GitGitGadget
  Cc: git, Junio C Hamano, Johannes Schindelin

On Mon, Oct 15, 2018 at 03:12:06AM -0700, Johannes Schindelin via GitGitGadget wrote:
> diff --git a/azure-pipelines.yml b/azure-pipelines.yml
> new file mode 100644
> index 0000000000..b5749121d2
> --- /dev/null
> +++ b/azure-pipelines.yml
> @@ -0,0 +1,319 @@
> +resources:
> +- repo: self
> +  fetchDepth: 1
> +
> +phases:
> +- phase: linux_clang
> +  displayName: linux-clang
> +  condition: succeeded()
> +  queue:
> +    name: Hosted Ubuntu 1604
> +  steps:
> +  - bash: |
> +       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
> +
> +       sudo apt-get update &&
> +       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin &&
> +
> +       export CC=clang || exit 1
> +
> +       ci/install-dependencies.sh

I think you would want to 'exit 1' when this script fails.
This applies to other build jobs (erm, phases?) below as well.

> +       ci/run-build-and-tests.sh || {
> +           ci/print-test-failures.sh
> +           exit 1
> +       }
> +
> +       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
> +    displayName: 'ci/run-build-and-tests.sh'
> +    env:
> +      GITFILESHAREPWD: $(gitfileshare.pwd)
> +  - task: PublishTestResults@2
> +    displayName: 'Publish Test Results **/TEST-*.xml'
> +    inputs:
> +      mergeTestResults: true
> +      testRunTitle: 'linux-clang'
> +      platform: Linux
> +      publishRunAttachments: false
> +    condition: succeededOrFailed()
> +
> +- phase: linux_gcc
> +  displayName: linux-gcc
> +  condition: succeeded()
> +  queue:
> +    name: Hosted Ubuntu 1604
> +  steps:
> +  - bash: |
> +       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
> +
> +       sudo apt-get update &&
> +       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin || exit 1
> +

On Travis CI the Linux GCC build job uses gcc-8 instead of whatever
the default is in that old-ish Ubuntu LTS; see 37fa4b3c78 (travis-ci:
run gcc-8 on linux-gcc jobs, 2018-05-19).

> +       ci/install-dependencies.sh
> +       ci/run-build-and-tests.sh || {
> +           ci/print-test-failures.sh
> +           exit 1
> +       }
> +
> +       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
> +    displayName: 'ci/run-build-and-tests.sh'
> +    env:
> +      GITFILESHAREPWD: $(gitfileshare.pwd)
> +  - task: PublishTestResults@2
> +    displayName: 'Publish Test Results **/TEST-*.xml'
> +    inputs:
> +      mergeTestResults: true
> +      testRunTitle: 'linux-gcc'
> +      platform: Linux
> +      publishRunAttachments: false
> +    condition: succeededOrFailed()
> +
> +- phase: osx_clang
> +  displayName: osx-clang
> +  condition: succeeded()
> +  queue:
> +    name: Hosted macOS
> +  steps:
> +  - bash: |
> +       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
> +
> +       export CC=clang
> +
> +       ci/install-dependencies.sh
> +       ci/run-build-and-tests.sh || {
> +           ci/print-test-failures.sh
> +           exit 1
> +       }
> +
> +       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1
> +    displayName: 'ci/run-build-and-tests.sh'
> +    env:
> +      GITFILESHAREPWD: $(gitfileshare.pwd)
> +  - task: PublishTestResults@2
> +    displayName: 'Publish Test Results **/TEST-*.xml'
> +    inputs:
> +      mergeTestResults: true
> +      testRunTitle: 'osx-clang'
> +      platform: macOS
> +      publishRunAttachments: false
> +    condition: succeededOrFailed()
> +
> +- phase: osx_gcc
> +  displayName: osx-gcc
> +  condition: succeeded()
> +  queue:
> +    name: Hosted macOS
> +  steps:
> +  - bash: |
> +       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
> +

Here you should 'export CC=gcc', because on macOS 'cc' is 'clang' by
default.

Note, however, that setting 'CC' in the environment alone has no
effect on the build process, it will still use 'cc'.  Keep an eye on
where this thread will lead to:

  https://public-inbox.org/git/20181016184537.GN19800@szeder.dev/T/#u

> +       ci/install-dependencies.sh
> +       ci/run-build-and-tests.sh || {
> +           ci/print-test-failures.sh
> +           exit 1
> +       }
> +
> +       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1
> +    displayName: 'ci/run-build-and-tests.sh'
> +    env:
> +      GITFILESHAREPWD: $(gitfileshare.pwd)
> +  - task: PublishTestResults@2
> +    displayName: 'Publish Test Results **/TEST-*.xml'
> +    inputs:
> +      mergeTestResults: true
> +      testRunTitle: 'osx-gcc'
> +      platform: macOS
> +      publishRunAttachments: false
> +    condition: succeededOrFailed()
> +

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 10/13] tests: include detailed trace logs with --write-junit-xml upon failure
  2018-10-16 16:03         ` SZEDER Gábor
@ 2018-10-16 20:53           ` Johannes Schindelin
  0 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin @ 2018-10-16 20:53 UTC (permalink / raw)
  To: SZEDER Gábor
  Cc: Johannes Schindelin via GitGitGadget, git, Junio C Hamano

[-- Attachment #1: Type: text/plain, Size: 813 bytes --]

Hi Gábor,

On Tue, 16 Oct 2018, SZEDER Gábor wrote:

> On Tue, Oct 16, 2018 at 03:02:38PM +0200, Johannes Schindelin wrote:
>
> > So I would suggest to go forward with my proposed strategy for the
> > moment, right up until the time when we have had the resources to fix
> > t5570, for starters.
> 
> I don't really understand what the occasional failures in t5570 have
> to do with the amount of information a CI system should gather about
> failures in general.

I see it plenty of times that too many CI failures essentially render
every developer numb.

If every 3rd CI run causes a failure, and seemingly every of these
failures indicates a mistake in the regression test, rather than a
regression, developers stop paying attention.

Which is the exact opposite of what I want to achieve here.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 06/13] Add a build definition for Azure DevOps
  2018-10-16 19:12     ` SZEDER Gábor
@ 2018-10-17 14:58       ` Johannes Schindelin
  0 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin @ 2018-10-17 14:58 UTC (permalink / raw)
  To: SZEDER Gábor
  Cc: Johannes Schindelin via GitGitGadget, git, Junio C Hamano

[-- Attachment #1: Type: text/plain, Size: 6498 bytes --]

Hi Gábor,

On Tue, 16 Oct 2018, SZEDER Gábor wrote:

> On Mon, Oct 15, 2018 at 03:12:06AM -0700, Johannes Schindelin via GitGitGadget wrote:
> > diff --git a/azure-pipelines.yml b/azure-pipelines.yml
> > new file mode 100644
> > index 0000000000..b5749121d2
> > --- /dev/null
> > +++ b/azure-pipelines.yml
> > @@ -0,0 +1,319 @@
> > +resources:
> > +- repo: self
> > +  fetchDepth: 1
> > +
> > +phases:
> > +- phase: linux_clang
> > +  displayName: linux-clang
> > +  condition: succeeded()
> > +  queue:
> > +    name: Hosted Ubuntu 1604
> > +  steps:
> > +  - bash: |
> > +       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
> > +
> > +       sudo apt-get update &&
> > +       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin &&
> > +
> > +       export CC=clang || exit 1
> > +
> > +       ci/install-dependencies.sh
> 
> I think you would want to 'exit 1' when this script fails.
> This applies to other build jobs (erm, phases?) below as well.

True.

FWIW the nomenclature is "build" or "job" or "build job" for the entire
run, from what I understand. The "phase" is the individual chunk that is
run in an individual agent, i.e. you can have a single job running test on
different OSes in separate phases.

> > +       ci/run-build-and-tests.sh || {
> > +           ci/print-test-failures.sh
> > +           exit 1
> > +       }
> > +
> > +       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
> > +    displayName: 'ci/run-build-and-tests.sh'
> > +    env:
> > +      GITFILESHAREPWD: $(gitfileshare.pwd)
> > +  - task: PublishTestResults@2
> > +    displayName: 'Publish Test Results **/TEST-*.xml'
> > +    inputs:
> > +      mergeTestResults: true
> > +      testRunTitle: 'linux-clang'
> > +      platform: Linux
> > +      publishRunAttachments: false
> > +    condition: succeededOrFailed()
> > +
> > +- phase: linux_gcc
> > +  displayName: linux-gcc
> > +  condition: succeeded()
> > +  queue:
> > +    name: Hosted Ubuntu 1604
> > +  steps:
> > +  - bash: |
> > +       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
> > +
> > +       sudo apt-get update &&
> > +       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin || exit 1
> > +
> 
> On Travis CI the Linux GCC build job uses gcc-8 instead of whatever
> the default is in that old-ish Ubuntu LTS; see 37fa4b3c78 (travis-ci:
> run gcc-8 on linux-gcc jobs, 2018-05-19).

I'll add those dependencies explicitly. It does seem, however, from a
cursory look at the log, that gcc-8 should not even be picked up, as it is
set via the environment variable `CC` (which, as you point out in
below-referenced thread, is not respected):

[...]
2018-10-16T10:00:36.0177072Z ++ '[' linux-gcc = linux-gcc ']'
2018-10-16T10:00:36.0177380Z ++ export CC=gcc-8
2018-10-16T10:00:36.0177630Z ++ CC=gcc-8
2018-10-16T10:00:36.0177917Z ++ case "$jobname"
[...]

(see https://dev.azure.com/git/git/_build/results?buildId=192&view=logs)

> > +       ci/install-dependencies.sh
> > +       ci/run-build-and-tests.sh || {
> > +           ci/print-test-failures.sh
> > +           exit 1
> > +       }
> > +
> > +       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
> > +    displayName: 'ci/run-build-and-tests.sh'
> > +    env:
> > +      GITFILESHAREPWD: $(gitfileshare.pwd)
> > +  - task: PublishTestResults@2
> > +    displayName: 'Publish Test Results **/TEST-*.xml'
> > +    inputs:
> > +      mergeTestResults: true
> > +      testRunTitle: 'linux-gcc'
> > +      platform: Linux
> > +      publishRunAttachments: false
> > +    condition: succeededOrFailed()
> > +
> > +- phase: osx_clang
> > +  displayName: osx-clang
> > +  condition: succeeded()
> > +  queue:
> > +    name: Hosted macOS
> > +  steps:
> > +  - bash: |
> > +       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
> > +
> > +       export CC=clang
> > +
> > +       ci/install-dependencies.sh
> > +       ci/run-build-and-tests.sh || {
> > +           ci/print-test-failures.sh
> > +           exit 1
> > +       }
> > +
> > +       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1
> > +    displayName: 'ci/run-build-and-tests.sh'
> > +    env:
> > +      GITFILESHAREPWD: $(gitfileshare.pwd)
> > +  - task: PublishTestResults@2
> > +    displayName: 'Publish Test Results **/TEST-*.xml'
> > +    inputs:
> > +      mergeTestResults: true
> > +      testRunTitle: 'osx-clang'
> > +      platform: macOS
> > +      publishRunAttachments: false
> > +    condition: succeededOrFailed()
> > +
> > +- phase: osx_gcc
> > +  displayName: osx-gcc
> > +  condition: succeeded()
> > +  queue:
> > +    name: Hosted macOS
> > +  steps:
> > +  - bash: |
> > +       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
> > +
> 
> Here you should 'export CC=gcc', because on macOS 'cc' is 'clang' by
> default.
> 
> Note, however, that setting 'CC' in the environment alone has no
> effect on the build process, it will still use 'cc'.  Keep an eye on
> where this thread will lead to:
> 
>   https://public-inbox.org/git/20181016184537.GN19800@szeder.dev/T/#u

Will do.

Thanks,
Dscho

> 
> > +       ci/install-dependencies.sh
> > +       ci/run-build-and-tests.sh || {
> > +           ci/print-test-failures.sh
> > +           exit 1
> > +       }
> > +
> > +       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1
> > +    displayName: 'ci/run-build-and-tests.sh'
> > +    env:
> > +      GITFILESHAREPWD: $(gitfileshare.pwd)
> > +  - task: PublishTestResults@2
> > +    displayName: 'Publish Test Results **/TEST-*.xml'
> > +    inputs:
> > +      mergeTestResults: true
> > +      testRunTitle: 'osx-gcc'
> > +      platform: macOS
> > +      publishRunAttachments: false
> > +    condition: succeededOrFailed()
> > +
> 
^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 09/13] git-p4: use `test_atexit` to kill the daemon
  2018-10-15 10:12   ` [PATCH v2 09/13] git-p4: use `test_atexit` to kill the daemon Johannes Schindelin via GitGitGadget
  2018-10-15 11:00     ` Luke Diamand
  2018-10-16  3:34     ` Eric Sunshine
@ 2018-10-17 23:29     ` SZEDER Gábor
  2 siblings, 0 replies; 88+ messages in thread
From: SZEDER Gábor @ 2018-10-17 23:29 UTC (permalink / raw)
  To: Johannes Schindelin via GitGitGadget
  Cc: git, Junio C Hamano, Johannes Schindelin, Luke Diamand

On Mon, Oct 15, 2018 at 03:12:11AM -0700, Johannes Schindelin via GitGitGadget wrote:
> From: Johannes Schindelin <johannes.schindelin@gmx.de>
> 
> This should be more reliable than the current method, and prepares the
> test suite for a consistent way to clean up before re-running the tests
> with different options.

This patch makes the test suite hang in all four Travis CI build jobs
with P4 installed without any of the P4 tests finishing.

Reverting this patch from the whole patch series makes it work again.

I've also tried to revert only this first hunk of the patch below,
because based on the comment I thought it's worth a try, but it didn't
really help.  It did make a difference: the 300s watchdog timer
eventually kicked in, and then the test scripts could finish
successfully...  but there are a lot of P4 test scripts, and with each
taking 300s the build job still timeouted.

All this may (or may not) be related to and be a different symptom of
the leftover p4d processes Luke mentioned.  I couldn't reproduce any
of this on my machine so far.

> diff --git a/t/lib-git-p4.sh b/t/lib-git-p4.sh
> index c27599474c..f4f5d7d296 100644
> --- a/t/lib-git-p4.sh
> +++ b/t/lib-git-p4.sh
> @@ -74,15 +74,6 @@ cli="$TRASH_DIRECTORY/cli"
>  git="$TRASH_DIRECTORY/git"
>  pidfile="$TRASH_DIRECTORY/p4d.pid"
>  
> -# Sometimes "prove" seems to hang on exit because p4d is still running
> -cleanup () {
> -	if test -f "$pidfile"
> -	then
> -		kill -9 $(cat "$pidfile") 2>/dev/null && exit 255
> -	fi
> -}
> -trap cleanup EXIT
> -
>  # git p4 submit generates a temp file, which will
>  # not get cleaned up if the submission fails.  Don't
>  # clutter up /tmp on the test machine.

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services
  2018-10-16  9:14           ` Ævar Arnfjörð Bjarmason
@ 2018-10-18  2:03             ` Junio C Hamano
  2018-10-19 10:40               ` Mirror of git.git on gitlab.com Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 88+ messages in thread
From: Junio C Hamano @ 2018-10-18  2:03 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Johannes Schindelin, Christian Couder,
	Nguyễn Thái Ngọc Duy,
	Johannes Schindelin via GitGitGadget, Git Mailing List

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

>> sites could do the same polling and mirroring.  I am just too lazy
>> to open a new account at yet another hosting site to add that for
>> loop, but I may choose to when I am absolutely bored and nothing
>> else to do ;-).
>
> Do you mind if I squat gitlab.com/git/git in the meantime (i.e. create
> an org etc.) and have it mirror github.com/git/git?, I'll hand the

Obviously somebody who is not even interested in obtaining an
account would appreciate, not just "would not mind", if a trusted
member in the community did that for the community ;-)

Thanks.

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 02/13] ci/lib.sh: encapsulate Travis-specific things
  2018-10-15 10:12   ` [PATCH v2 02/13] ci/lib.sh: encapsulate Travis-specific things Johannes Schindelin via GitGitGadget
@ 2018-10-18 22:01     ` SZEDER Gábor
  2018-10-19  2:06       ` Junio C Hamano
  0 siblings, 1 reply; 88+ messages in thread
From: SZEDER Gábor @ 2018-10-18 22:01 UTC (permalink / raw)
  To: Johannes Schindelin via GitGitGadget
  Cc: git, Junio C Hamano, Johannes Schindelin

On Mon, Oct 15, 2018 at 03:12:00AM -0700, Johannes Schindelin via GitGitGadget wrote:
> diff --git a/ci/lib.sh b/ci/lib.sh
> index 06970f7213..8532555b4e 100755
> --- a/ci/lib.sh
> +++ b/ci/lib.sh
> @@ -1,5 +1,26 @@
>  # Library of functions shared by all CI scripts
>  
> +if test true = "$TRAVIS"
> +then
> +	# We are running within Travis CI
> +	CI_BRANCH="$TRAVIS_BRANCH"
> +	CI_COMMIT="$TRAVIS_COMMIT"
> +	CI_JOB_ID="$TRAVIS_JOB_ID"
> +	CI_JOB_NUMBER="$TRAVIS_JOB_NUMBER"
> +	CI_OS_NAME="$TRAVIS_OS_NAME"
> +	CI_REPO_SLUG="$TRAVIS_REPO_SLUG"
> +
> +	cache_dir="$HOME/travis-cache"
> +
> +	url_for_job_id () {
> +		echo "https://travis-ci.org/$CI_REPO_SLUG/jobs/$1"
> +	}
> +
> +	BREW_INSTALL_PACKAGES="git-lfs gettext"
> +	export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
> +	export GIT_TEST_OPTS="--verbose-log -x --immediate"
> +fi

Please set all these variables ...

> +
>  skip_branch_tip_with_tag () {
>  	# Sometimes, a branch is pushed at the same time the tag that points
>  	# at the same commit as the tip of the branch is pushed, and building
> @@ -13,10 +34,10 @@ skip_branch_tip_with_tag () {
>  	# we can skip the build because we won't be skipping a build
>  	# of a tag.
>  
> -	if TAG=$(git describe --exact-match "$TRAVIS_BRANCH" 2>/dev/null) &&
> -		test "$TAG" != "$TRAVIS_BRANCH"
> +	if TAG=$(git describe --exact-match "$CI_BRANCH" 2>/dev/null) &&
> +		test "$TAG" != "$CI_BRANCH"
>  	then
> -		echo "$(tput setaf 2)Tip of $TRAVIS_BRANCH is exactly at $TAG$(tput sgr0)"
> +		echo "$(tput setaf 2)Tip of $CI_BRANCH is exactly at $TAG$(tput sgr0)"
>  		exit 0
>  	fi
>  }
> @@ -25,7 +46,7 @@ skip_branch_tip_with_tag () {
>  # job if we encounter the same tree again and can provide a useful info
>  # message.
>  save_good_tree () {
> -	echo "$(git rev-parse $TRAVIS_COMMIT^{tree}) $TRAVIS_COMMIT $TRAVIS_JOB_NUMBER $TRAVIS_JOB_ID" >>"$good_trees_file"
> +	echo "$(git rev-parse $CI_COMMIT^{tree}) $CI_COMMIT $CI_JOB_NUMBER $CI_JOB_ID" >>"$good_trees_file"
>  	# limit the file size
>  	tail -1000 "$good_trees_file" >"$good_trees_file".tmp
>  	mv "$good_trees_file".tmp "$good_trees_file"
> @@ -35,7 +56,7 @@ save_good_tree () {
>  # successfully before (e.g. because the branch got rebased, changing only
>  # the commit messages).
>  skip_good_tree () {
> -	if ! good_tree_info="$(grep "^$(git rev-parse $TRAVIS_COMMIT^{tree}) " "$good_trees_file")"
> +	if ! good_tree_info="$(grep "^$(git rev-parse $CI_COMMIT^{tree}) " "$good_trees_file")"
>  	then
>  		# Haven't seen this tree yet, or no cached good trees file yet.
>  		# Continue the build job.
> @@ -45,18 +66,18 @@ skip_good_tree () {
>  	echo "$good_tree_info" | {
>  		read tree prev_good_commit prev_good_job_number prev_good_job_id
>  
> -		if test "$TRAVIS_JOB_ID" = "$prev_good_job_id"
> +		if test "$CI_JOB_ID" = "$prev_good_job_id"
>  		then
>  			cat <<-EOF
> -			$(tput setaf 2)Skipping build job for commit $TRAVIS_COMMIT.$(tput sgr0)
> +			$(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0)
>  			This commit has already been built and tested successfully by this build job.
>  			To force a re-build delete the branch's cache and then hit 'Restart job'.
>  			EOF
>  		else
>  			cat <<-EOF
> -			$(tput setaf 2)Skipping build job for commit $TRAVIS_COMMIT.$(tput sgr0)
> +			$(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0)
>  			This commit's tree has already been built and tested successfully in build job $prev_good_job_number for commit $prev_good_commit.
> -			The log of that build job is available at https://travis-ci.org/$TRAVIS_REPO_SLUG/jobs/$prev_good_job_id
> +			The log of that build job is available at $(url_for_job_id $prev_good_job_id)
>  			To force a re-build delete the branch's cache and then hit 'Restart job'.
>  			EOF
>  		fi
> @@ -81,7 +102,6 @@ check_unignored_build_artifacts ()
>  # and installing dependencies.
>  set -ex

... after we turn on 'set -x', so the variables' values will be
visible in the logs.

(Or move this 'set -ex' to the beginning of the script?  Then we
could perhaps avoid similar issues in the future.)

> -cache_dir="$HOME/travis-cache"
>  good_trees_file="$cache_dir/good-trees"
>  
>  mkdir -p "$cache_dir"

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 02/13] ci/lib.sh: encapsulate Travis-specific things
  2018-10-18 22:01     ` SZEDER Gábor
@ 2018-10-19  2:06       ` Junio C Hamano
  2018-10-19  8:25         ` SZEDER Gábor
  0 siblings, 1 reply; 88+ messages in thread
From: Junio C Hamano @ 2018-10-19  2:06 UTC (permalink / raw)
  To: SZEDER Gábor
  Cc: Johannes Schindelin via GitGitGadget, git, Johannes Schindelin

SZEDER Gábor <szeder.dev@gmail.com> writes:

> On Mon, Oct 15, 2018 at 03:12:00AM -0700, Johannes Schindelin via GitGitGadget wrote:
>> diff --git a/ci/lib.sh b/ci/lib.sh
>> index 06970f7213..8532555b4e 100755
>> --- a/ci/lib.sh
>> +++ b/ci/lib.sh
>> @@ -1,5 +1,26 @@
>>  # Library of functions shared by all CI scripts
>>  
>> +if test true = "$TRAVIS"
>> +then
>> +...
>> +	export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
>> +	export GIT_TEST_OPTS="--verbose-log -x --immediate"
>> +fi
>
> Please set all these variables ...

Do you mean "VAR=VAL; export VAR" is kosher, "export VAR=VAL" is
not?

>> @@ -81,7 +102,6 @@ check_unignored_build_artifacts ()
>>  # and installing dependencies.
>>  set -ex
>
> ... after we turn on 'set -x', so the variables' values will be
> visible in the logs.

Ah, no, you didn't.  Although I think both are valid points, I think
ci/lib.sh is expected to be used only inside a more predictable
environment (e.g. we know the shell used is not a random POSIX shell
but one that is happy with "export VAR=VAL"), so it should be OK.
Showing the values of these variables in the log may still be good
idea.

> (Or move this 'set -ex' to the beginning of the script?  Then we
> could perhaps avoid similar issues in the future.)

Sure (provided that it is an issue to begin with---if we are
interested in the value of TRAVIS_BRANCH, for example, being able to
see it only because "CI_BRANCH=$TRAVIS_BRANCH" assignment is made
feels a bit sideways---we'd be better off explicitly logging
anything we are interested in in the longer term, no?).

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 02/13] ci/lib.sh: encapsulate Travis-specific things
  2018-10-19  2:06       ` Junio C Hamano
@ 2018-10-19  8:25         ` SZEDER Gábor
  2018-10-19  8:35           ` Junio C Hamano
  0 siblings, 1 reply; 88+ messages in thread
From: SZEDER Gábor @ 2018-10-19  8:25 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Johannes Schindelin via GitGitGadget, git, Johannes Schindelin

On Fri, Oct 19, 2018 at 11:06:25AM +0900, Junio C Hamano wrote:
> SZEDER Gábor <szeder.dev@gmail.com> writes:
> 
> > On Mon, Oct 15, 2018 at 03:12:00AM -0700, Johannes Schindelin via GitGitGadget wrote:
> >> diff --git a/ci/lib.sh b/ci/lib.sh
> >> index 06970f7213..8532555b4e 100755
> >> --- a/ci/lib.sh
> >> +++ b/ci/lib.sh
> >> @@ -1,5 +1,26 @@
> >>  # Library of functions shared by all CI scripts
> >>  
> >> +if test true = "$TRAVIS"
> >> +then
> >> +...
> >> +	export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
> >> +	export GIT_TEST_OPTS="--verbose-log -x --immediate"
> >> +fi
> >
> > Please set all these variables ...
> 
> Do you mean "VAR=VAL; export VAR" is kosher, "export VAR=VAL" is
> not?
> 
> >> @@ -81,7 +102,6 @@ check_unignored_build_artifacts ()
> >>  # and installing dependencies.
> >>  set -ex
> >
> > ... after we turn on 'set -x', so the variables' values will be
> > visible in the logs.
> 
> Ah, no, you didn't.  Although I think both are valid points, I think
> ci/lib.sh is expected to be used only inside a more predictable
> environment (e.g. we know the shell used is not a random POSIX shell
> but one that is happy with "export VAR=VAL"), so it should be OK.

Yes.  Travis CI runs an Ubuntu LTS, where /bin/sh is dash, which
understands 'export VAR=val' just fine.  I don't know what Linux
distro runs on Azure Pipelines, but the build definition in patch 6
explicitly asks for Bash, so that should be fine as well.

> Showing the values of these variables in the log may still be good
> idea.
> 
> > (Or move this 'set -ex' to the beginning of the script?  Then we
> > could perhaps avoid similar issues in the future.)
> 
> Sure (provided that it is an issue to begin with---if we are
> interested in the value of TRAVIS_BRANCH, for example, being able to
> see it only because "CI_BRANCH=$TRAVIS_BRANCH" assignment is made
> feels a bit sideways---we'd be better off explicitly logging
> anything we are interested in in the longer term, no?).

Well, all the $TRAVIS_* and $CI_* variables are not that interessant,
because they are only used in the build scripts, and then we can see
their substituted values in the build logs.  The same applies to
the variables $cache_dir and $BREW_INSTALL_PACKAGES as well.

$GIT_PROVE_OPTS and $GIT_TEST_OPTS, however, are only used in
't/Makefile' but not in the build scripts, thus their values don't
show up in the build logs.

I run some Travis CI builds with custom $GIT_TEST_OPTS, which, of
course, conflicted with this patch, and I messed up the conflict
resolution.  I think I would have noticed sooner what went wrong, if
the value of $GIT_TEST_OPTS were visible in the build logs.

I've found the output with 'set -x' sufficient so far, and don't think
that an explicit logging facility is worth it.


^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 02/13] ci/lib.sh: encapsulate Travis-specific things
  2018-10-19  8:25         ` SZEDER Gábor
@ 2018-10-19  8:35           ` Junio C Hamano
  0 siblings, 0 replies; 88+ messages in thread
From: Junio C Hamano @ 2018-10-19  8:35 UTC (permalink / raw)
  To: SZEDER Gábor
  Cc: Johannes Schindelin via GitGitGadget, git, Johannes Schindelin

SZEDER Gábor <szeder.dev@gmail.com> writes:

>> >> +if test true = "$TRAVIS"
>> >> +then
>> >> +...
>> >> +	export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
>> >> +	export GIT_TEST_OPTS="--verbose-log -x --immediate"
>> >> +fi
>> > ...
>
> $GIT_PROVE_OPTS and $GIT_TEST_OPTS, however, are only used in
> 't/Makefile' but not in the build scripts, thus their values don't
> show up in the build logs.
>
> I run some Travis CI builds with custom $GIT_TEST_OPTS, which, of
> course, conflicted with this patch, and I messed up the conflict
> resolution.  I think I would have noticed sooner what went wrong, if
> the value of $GIT_TEST_OPTS were visible in the build logs.
>
> I've found the output with 'set -x' sufficient so far, and don't think
> that an explicit logging facility is worth it.

That's an interesting perspective.  I would have thoguht that the
values of these two variables we can see above, that are constants
without any substitution or customization, are the least interesting
things to see in the log.



^ permalink raw reply	[flat|nested] 88+ messages in thread

* Mirror of git.git on gitlab.com
  2018-10-18  2:03             ` Junio C Hamano
@ 2018-10-19 10:40               ` Ævar Arnfjörð Bjarmason
  2018-10-22  5:41                 ` Junio C Hamano
  0 siblings, 1 reply; 88+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2018-10-19 10:40 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Johannes Schindelin, Christian Couder,
	Nguyễn Thái Ngọc Duy,
	Johannes Schindelin via GitGitGadget, Git Mailing List


On Thu, Oct 18 2018, Junio C Hamano wrote:

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>>> sites could do the same polling and mirroring.  I am just too lazy
>>> to open a new account at yet another hosting site to add that for
>>> loop, but I may choose to when I am absolutely bored and nothing
>>> else to do ;-).
>>
>> Do you mind if I squat gitlab.com/git/git in the meantime (i.e. create
>> an org etc.) and have it mirror github.com/git/git?, I'll hand the
>
> Obviously somebody who is not even interested in obtaining an
> account would appreciate, not just "would not mind", if a trusted
> member in the community did that for the community ;-)

I've set this up at https://gitlab.com/git-vcs

The /git namespace was taken (and I asked GitLab support if it was
stale, they said no). Also tried /git-scm and /gitscm, ditto. So I
settled on /git-vcs (version control system).

I mirrored all the repos that were on github.com/git (except for
cabal.git of course). This is being kept up-to-date with GitLab's own
mirroring feature, so it should always be max ~15m out of date v.s. the
GitHub version.

As an aside, I noticed that
https://github.com/git/sha1collisiondetection/ has never worked in
combination with git.git, i.e. it's cloned at a version that pre-dates
the initial introduction of the sha1collisiondetection submodule. Our
other mirrors don't seem to have it at all relative to
../sha1collisiondetection.git from their git.git.

Junio: What was the plan with that? It's never been used in combination
with git.git, so maybe we should just drop it?

If we'd like to keep it and if it was kept up-to-date some scripts of
yours we could use a relative URL in the .gitmodules, so e.g. if you
cloned from kernel.org you'd also get the submodule from there.

Also since we're fixing it, it would be good to delete the existing repo
and re-make it by clicking "fork" on
https://github.com/cr-marcstevens/sha1collisiondetection so GitHub shows
"forked from...", i.e. it's associated with the parent project in
GitHub's UI.

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 07/13] tests: introduce `test_atexit`
  2018-10-15 10:12   ` [PATCH v2 07/13] tests: introduce `test_atexit` Johannes Schindelin via GitGitGadget
@ 2018-10-21 11:20     ` SZEDER Gábor
  0 siblings, 0 replies; 88+ messages in thread
From: SZEDER Gábor @ 2018-10-21 11:20 UTC (permalink / raw)
  To: Johannes Schindelin via GitGitGadget
  Cc: git, Junio C Hamano, Johannes Schindelin

On Mon, Oct 15, 2018 at 03:12:08AM -0700, Johannes Schindelin via GitGitGadget wrote:
> From: Johannes Schindelin <johannes.schindelin@gmx.de>
> 
> When running the p4 daemon or `git daemon`, we want to kill it at the
> end of the test script.
> 
> So far, we do this "manually".
> 
> However, in the next few commits we want to teach the test suite to
> optionally re-run scripts with different options, therefore we will have
> to have a consistent way to stop daemons.
> 
> Let's introduce `test_atexit`, which is loosely modeled after
> `test_when_finished` (but has a broader scope: rather than running the
> commands after the current test case, run them when the test script
> finishes, and also run them when the `--immediate` option is in effect).

I think killing daemons on failure even with '--immediate' will make
life a bit easier.  I remember several occasions when I tried to debug
a failing test with '--immediate', some daemon stayed alive, and then
the next time I run the test it failed during setup.

Besides 'git daemon' and P4, the httpd tests could benefit from this
as well.


> diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
> index 78d8c3783b..d7dd0c1be9 100644
> --- a/t/test-lib-functions.sh
> +++ b/t/test-lib-functions.sh
> @@ -891,6 +891,35 @@ test_when_finished () {
>  		} && (exit \"\$eval_ret\"); eval_ret=\$?; $test_cleanup"
>  }
>  
> +# This function can be used to schedule some commands to be run
> +# unconditionally at the end of the test script, e.g. to stop a daemon:
> +#
> +#	test_expect_success 'test git daemon' '
> +#		git daemon &
> +#		daemon_pid=$! &&
> +#		test_atexit "kill $daemon_pid" &&
> +#		hello world
> +#	'

I think we should add something like this as well:

# Note that these commands will be run even when a test script run
# with '--immediate' fails.  Be careful with your commands to minimize
# any changes to the failed state.

> +
> +test_atexit () {
> +	# We cannot detect when we are in a subshell in general, but by
> +	# doing so on Bash is better than nothing (the test will
> +	# silently pass on other shells).
> +	test "${BASH_SUBSHELL-0}" = 0 ||
> +	error "bug in test script: test_atexit does nothing in a subshell"
> +	test_atexit_cleanup="{ $*
> +		} && (exit \"\$eval_ret\"); eval_ret=\$?; $test_atexit_cleanup"
> +}
> +
> +test_atexit_handler () {
> +	test : != "$test_atexit_cleanup" || return 0
> +
> +	setup_malloc_check
> +	test_eval_ "$test_atexit_cleanup"
> +	test_atexit_cleanup=:
> +	teardown_malloc_check
> +}

The file 'test-lib-functions.sh' contains helper functions to be used
in tests.  'test_atexit_handler' is not such a function, so I think it
would be better to add it to 'test-lib.sh'.

> +
>  # Most tests can use the created repository, but some may need to create more.
>  # Usage: test_create_repo <directory>
>  test_create_repo () {
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> index 7ed0013f6d..6f9c1f5300 100644
> --- a/t/test-lib.sh
> +++ b/t/test-lib.sh
> @@ -413,6 +413,7 @@ test_external_has_tap=0
>  
>  die () {
>  	code=$?

# Run atexit handlers even when a test script run with
# '--immediate' fails.

> +	test_atexit_handler || code=$?
>  	if test -n "$GIT_EXIT_OK"
>  	then
>  		exit $code
> @@ -826,9 +827,12 @@ write_junit_xml_testcase () {
>  	junit_have_testcase=t
>  }
>  
> +test_atexit_cleanup=:
>  test_done () {
>  	GIT_EXIT_OK=t
>  
> +	test -n "$immediate" || test_atexit_handler

You mentioned elsewhere [1] that you'll remove the condition to run
'test_atexit_handler' unconditionally.  I think this is the right
thing to do, because this condition is responsible for the hanging P4
tests I reported in [2].

So, 'lib-git-p4.sh' stores the pid of the 'p4d' process in the file
"$TRASH_DIRECTORY/p4d.pid" in 'start_p4d', to be read later in
'kill_p4d' to know which process to kill.  When a test script run with
'--immediate' ends successfully, and the atexit handlers are run in
'die' above instead of in 'test_done', then you've got the problem
that 'test_done' removes the trash directory along with the 'p4d.pid'
file before the atexit handlers are run.  With the pidfile gone
'kill_p4d' won't know which process to kill, and the 'p4d' process
will stay alive.

If that condition is removed, then the atexit handlers will be run
before the trash directory is removed, so 'p4d' processes will be
stopped as they should even when an '--immediate' run succeeds.  Good.
Note, however, that in this case 'test_atexit_handler' will be called
twice: first in 'test_done' and then in 'die' as well.  This should be
fine, because 'test_atexit_handler' resets 'test_atexit_cleanup=:', so
the second call will do nothing.  Great.  Perhaps this would be worth
pointing out explicitly in the commit message and/or a comment.

[1] https://public-inbox.org/git/nycvar.QRO.7.76.6.1810161111240.4546@tvgsbejvaqbjf.bet/
[2] https://public-inbox.org/git/20181017232952.GT19800@szeder.dev/

> +
>  	if test -n "$write_junit_xml" && test -n "$junit_xml_path"
>  	then
>  		test -n "$junit_have_testcase" || {
> -- 
> gitgitgadget


^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: Mirror of git.git on gitlab.com
  2018-10-19 10:40               ` Mirror of git.git on gitlab.com Ævar Arnfjörð Bjarmason
@ 2018-10-22  5:41                 ` Junio C Hamano
  2018-10-25  8:37                   ` Jeff King
  0 siblings, 1 reply; 88+ messages in thread
From: Junio C Hamano @ 2018-10-22  5:41 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Johannes Schindelin, Christian Couder,
	Nguyễn Thái Ngọc Duy,
	Johannes Schindelin via GitGitGadget, Git Mailing List

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> I've set this up at https://gitlab.com/git-vcs
>
> The /git namespace was taken (and I asked GitLab support if it was
> stale, they said no). Also tried /git-scm and /gitscm, ditto. So I
> settled on /git-vcs (version control system).

Squatters X-<.  Thanks.

> As an aside, I noticed that
> https://github.com/git/sha1collisiondetection/ has never worked in
> combination with git.git, i.e. it's cloned at a version that pre-dates
> the initial introduction of the sha1collisiondetection submodule. Our
> other mirrors don't seem to have it at all relative to
> ../sha1collisiondetection.git from their git.git.

I do not recall who cloned it or forked it there or what our longer
term plans for that repository would be.  I think we are using this
thing: https://github.com/cr-marcstevens/sha1collisiondetection.git
and there is probably no reason to have our own copy.  Perhaps we
should just get rid of it.



^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines
  2018-10-15 10:11 ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Johannes Schindelin via GitGitGadget
                     ` (13 preceding siblings ...)
  2018-10-15 14:22   ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Taylor Blau
@ 2018-10-25  8:17   ` Junio C Hamano
  2018-10-26  8:37     ` Johannes Schindelin
  14 siblings, 1 reply; 88+ messages in thread
From: Junio C Hamano @ 2018-10-25  8:17 UTC (permalink / raw)
  To: Johannes Schindelin via GitGitGadget; +Cc: git

"Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
writes:

> For a long time already, we have Git's source code continuously tested via
> Travis CI, see e.g. https://travis-ci.org/git/git/builds/421738884. It has
> served us well, and more and more developers actually pay attention and
> benefit from the testing this gives us.

What's the current status of this topic?  Has the "p4 daemon gets
left behind" one resolved to everybody's satisfaction?  I think that
one was the only large discussion on the series (aside from "do we
want to keep Travis?" subthread, which does not make this series
undesirable), modulo your "oy oy oy that is leftover debugging I
need to remove in a reroll".

The topic was marked as "On hold, monitoring discussion" and it
seems that discussion has quieted down, so the next step is to see
an updated series?

Thanks.



^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: Mirror of git.git on gitlab.com
  2018-10-22  5:41                 ` Junio C Hamano
@ 2018-10-25  8:37                   ` Jeff King
  0 siblings, 0 replies; 88+ messages in thread
From: Jeff King @ 2018-10-25  8:37 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Ævar Arnfjörð Bjarmason, Johannes Schindelin,
	Christian Couder, Nguyễn Thái Ngọc Duy,
	Johannes Schindelin via GitGitGadget, Git Mailing List

On Mon, Oct 22, 2018 at 02:41:19PM +0900, Junio C Hamano wrote:

> > As an aside, I noticed that
> > https://github.com/git/sha1collisiondetection/ has never worked in
> > combination with git.git, i.e. it's cloned at a version that pre-dates
> > the initial introduction of the sha1collisiondetection submodule. Our
> > other mirrors don't seem to have it at all relative to
> > ../sha1collisiondetection.git from their git.git.
> 
> I do not recall who cloned it or forked it there or what our longer
> term plans for that repository would be.  I think we are using this
> thing: https://github.com/cr-marcstevens/sha1collisiondetection.git
> and there is probably no reason to have our own copy.  Perhaps we
> should just get rid of it.

I poked around. It was done by you on May 23, 2017 and then never
touched after that. I couldn't find any discussion on the list. So yeah,
I think it makes sense to just delete it.

-Peff

^ permalink raw reply	[flat|nested] 88+ messages in thread

* Re: [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines
  2018-10-25  8:17   ` Junio C Hamano
@ 2018-10-26  8:37     ` Johannes Schindelin
  0 siblings, 0 replies; 88+ messages in thread
From: Johannes Schindelin @ 2018-10-26  8:37 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Johannes Schindelin via GitGitGadget, git

Hi Junio,

On Thu, 25 Oct 2018, Junio C Hamano wrote:

> "Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
> writes:
> 
> > For a long time already, we have Git's source code continuously tested via
> > Travis CI, see e.g. https://travis-ci.org/git/git/builds/421738884. It has
> > served us well, and more and more developers actually pay attention and
> > benefit from the testing this gives us.
> 
> What's the current status of this topic?  Has the "p4 daemon gets
> left behind" one resolved to everybody's satisfaction?

No. I was kind of waiting for Luke's answer, and in the alternative I
hoped to find some time to work on trying to reproduce his issues on my
system (but I failed to find said time so far).

> I think that one was the only large discussion on the series (aside from
> "do we want to keep Travis?" subthread, which does not make this series
> undesirable), modulo your "oy oy oy that is leftover debugging I need to
> remove in a reroll".
> 
> The topic was marked as "On hold, monitoring discussion" and it
> seems that discussion has quieted down, so the next step is to see
> an updated series?

I really think that I have to figure out what causes those p4d issues
before I can give you that updated. I *am* interested, to be sure, it's
just that other things seem to get in my way all the time.

One thing that keeps getting in my way, for example, is the performance
issue identified in the chain linter. And I do think that I have to take
this into consideration for another update to this here patch series, too,
by adding `--no-chain-lint` to the Windows phase. There is a similar thing
with `--with-dashes`, too.

Will keep you updated,
Dscho

> 
> Thanks.
> 
> 
> 

^ permalink raw reply	[flat|nested] 88+ messages in thread

end of thread, back to index

Thread overview: 88+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-09-03 21:10 [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Johannes Schindelin via GitGitGadget
2018-09-03 21:10 ` [PATCH 1/9] ci: rename the library of common functions Johannes Schindelin via GitGitGadget
2018-09-03 21:10 ` [PATCH 2/9] ci/lib.sh: encapsulate Travis-specific things Johannes Schindelin via GitGitGadget
2018-09-03 23:43   ` Eric Sunshine
2018-09-04 11:04     ` Johannes Schindelin
2018-09-05 18:57   ` Sebastian Schuberth
2018-09-14 19:07     ` Johannes Schindelin
2018-09-03 21:10 ` [PATCH 3/9] test-date: add a subcommand to measure times in shell scripts Johannes Schindelin via GitGitGadget
2018-09-03 21:10 ` [PATCH 4/9] tests: optionally write results as JUnit-style .xml Johannes Schindelin via GitGitGadget
2018-09-04  0:43   ` Eric Sunshine
2018-09-04 10:59     ` Johannes Schindelin
2018-09-03 21:10 ` [PATCH 5/9] ci/lib.sh: add support for VSTS CI Johannes Schindelin via GitGitGadget
2018-09-03 21:10 ` [PATCH 6/9] Add a build definition for VSTS Johannes Schindelin via GitGitGadget
2018-09-03 21:10 ` [PATCH 7/9] tests: include detailed trace logs with --write-junit-xml upon failure Johannes Schindelin via GitGitGadget
2018-09-04  4:30   ` Eric Sunshine
2018-09-04 11:09     ` Johannes Schindelin
2018-09-05  5:32       ` Luke Diamand
2018-09-05 12:39         ` Johannes Schindelin
2018-09-05 13:03           ` Luke Diamand
2018-09-14 18:46             ` Johannes Schindelin
2018-09-05 18:38           ` Eric Sunshine
2018-09-05 20:24             ` Jeff King
2018-09-14 19:04               ` Johannes Schindelin
2018-09-14 18:51             ` Johannes Schindelin
2018-09-03 21:10 ` [PATCH 8/9] tests: record more stderr with --write-junit-xml in case of failure Johannes Schindelin via GitGitGadget
2018-09-03 21:10 ` [PATCH 9/9] README: add a build badge (status of the VSTS build) Johannes Schindelin via GitGitGadget
2018-09-05 19:01 ` [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Sebastian Schuberth
2018-09-05 19:08   ` Stefan Beller
2018-10-15 10:11 ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Johannes Schindelin via GitGitGadget
2018-10-15 10:11   ` [PATCH v2 01/13] ci: rename the library of common functions Johannes Schindelin via GitGitGadget
2018-10-16  5:26     ` Junio C Hamano
2018-10-15 10:12   ` [PATCH v2 02/13] ci/lib.sh: encapsulate Travis-specific things Johannes Schindelin via GitGitGadget
2018-10-18 22:01     ` SZEDER Gábor
2018-10-19  2:06       ` Junio C Hamano
2018-10-19  8:25         ` SZEDER Gábor
2018-10-19  8:35           ` Junio C Hamano
2018-10-15 10:12   ` [PATCH v2 03/13] test-date: add a subcommand to measure times in shell scripts Johannes Schindelin via GitGitGadget
2018-10-15 10:12   ` [PATCH v2 04/13] tests: optionally write results as JUnit-style .xml Johannes Schindelin via GitGitGadget
2018-10-15 10:12   ` [PATCH v2 05/13] ci/lib.sh: add support for Azure Pipelines Johannes Schindelin via GitGitGadget
2018-10-16  9:43     ` SZEDER Gábor
2018-10-16 12:53       ` Johannes Schindelin
2018-10-15 10:12   ` [PATCH v2 06/13] Add a build definition for Azure DevOps Johannes Schindelin via GitGitGadget
2018-10-16  5:35     ` Junio C Hamano
2018-10-16  9:58       ` Johannes Schindelin
2018-10-16 19:12     ` SZEDER Gábor
2018-10-17 14:58       ` Johannes Schindelin
2018-10-15 10:12   ` [PATCH v2 07/13] tests: introduce `test_atexit` Johannes Schindelin via GitGitGadget
2018-10-21 11:20     ` SZEDER Gábor
2018-10-15 10:12   ` [PATCH v2 08/13] git-daemon: use `test_atexit` in the tests Johannes Schindelin via GitGitGadget
2018-10-15 10:12   ` [PATCH v2 09/13] git-p4: use `test_atexit` to kill the daemon Johannes Schindelin via GitGitGadget
2018-10-15 11:00     ` Luke Diamand
2018-10-15 15:02       ` Johannes Schindelin
2018-10-15 20:19         ` Luke Diamand
2018-10-16  9:39           ` Johannes Schindelin
2018-10-16  3:34     ` Eric Sunshine
2018-10-16  8:51       ` Johannes Schindelin
2018-10-17 23:29     ` SZEDER Gábor
2018-10-15 10:12   ` [PATCH v2 10/13] tests: include detailed trace logs with --write-junit-xml upon failure Johannes Schindelin via GitGitGadget
2018-10-16 10:04     ` SZEDER Gábor
2018-10-16 13:02       ` Johannes Schindelin
2018-10-16 16:03         ` SZEDER Gábor
2018-10-16 20:53           ` Johannes Schindelin
2018-10-15 10:12   ` [PATCH v2 11/13] tests: record more stderr with --write-junit-xml in case of failure Johannes Schindelin via GitGitGadget
2018-10-15 10:12   ` [PATCH v2 12/13] README: add a build badge (status of the Azure Pipelines build) Johannes Schindelin via GitGitGadget
2018-10-16  3:44     ` Eric Sunshine
2018-10-16  8:48       ` Johannes Schindelin
2018-10-15 10:12   ` [PATCH v2 13/13] travis: fix skipping tagged releases Johannes Schindelin via GitGitGadget
2018-10-16  9:40     ` SZEDER Gábor
2018-10-16 12:45       ` Johannes Schindelin
2018-10-15 14:22   ` [PATCH v2 00/13] Offer to run CI/PR builds in Azure Pipelines Taylor Blau
2018-10-15 14:55     ` Johannes Schindelin
2018-10-16  2:53       ` Taylor Blau
2018-10-16 11:55         ` Ævar Arnfjörð Bjarmason
2018-10-16 10:30     ` SZEDER Gábor
2018-10-25  8:17   ` Junio C Hamano
2018-10-26  8:37     ` Johannes Schindelin
2018-10-15 15:06 ` [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Ævar Arnfjörð Bjarmason
2018-10-15 15:42   ` Duy Nguyen
2018-10-15 16:17     ` Christian Couder
2018-10-15 18:33       ` Johannes Schindelin
2018-10-16  0:24         ` Christian Couder
2018-10-16  4:50         ` Junio C Hamano
2018-10-16  9:14           ` Ævar Arnfjörð Bjarmason
2018-10-18  2:03             ` Junio C Hamano
2018-10-19 10:40               ` Mirror of git.git on gitlab.com Ævar Arnfjörð Bjarmason
2018-10-22  5:41                 ` Junio C Hamano
2018-10-25  8:37                   ` Jeff King
2018-10-16  4:47   ` [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services Junio C Hamano

git@vger.kernel.org mailing list mirror (one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/
       or Tor2web: https://www.tor2web.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox