[PATCH 0/3] making test-suite tracing more useful

[Jeff King <peff@peff.net>]

I sometimes run git's test suite as part of an automated testing
process. I was hoping to add "-x" support to get more details when a
test fails (since failures are sometimes hard to reproduce). But I hit a
few small snags:

  - you have to run with bash, since BASH_XTRACEFD is required to avoid
    failures in some tests when we capture the stderr of shell functions
    or subshells (which get polluted with the "set -x" outupt). This
    requirement isn't a big deal for me, but it showed some other
    issues.

  - the output with BASH_XTRACEFD is a little confusing; fixed by patch
    1.

  - there's one test that _only_ fails with BASH_XTRACEFD. That's fixed
    in patch 2.

  - I wanted to use "prove" since the output of "make -j64 test" is
    unreadable. But that means using "--verbose-log", which was
    incompatible with "-x". That's patch 3.

With these patches, I can now do:

  make -j64 test \
    SHELL_PATH=/bin/bash \
    GIT_TEST_OPTS="--verbose-log -x" \
    DEFAULT_TEST_TARGET=prove \
    GIT_PROVE_OPTS=-j64 || {
          echo "Failing tests:"
          echo "--------------"
	  grep -l '[^0]' t/test-results/*.exit |
	  while read failed; do
	          base=${failed%.exit}
		  name=${base#t/test-results/}
		  echo "==> $name"
		  cat "$base.out"
	  done
	  exit 1
    }

and get fairly readable output (a nice summary from prove, and then a
dump of any failing test output).

Johannes, I've seen that you do "-x" in the tests that the
git-for-windows bot uses to comment on GitHub. You may have seen the
bogus failure in t5615, which this series should fix (you may also have
seen the "set +x" cruft at the end of each test, which is fixed here,
too).

Lars, I think with this it should be possible to turn on "-x" for the
Travis build.

  [1/3]: test-lib: silence "-x" cleanup under bash
  [2/3]: t5615: avoid re-using descriptor 4
  [3/3]: test-lib: make "-x" work with "--verbose-log"

 t/t5615-alternate-env.sh |  6 +++---
 t/test-lib.sh            | 44 ++++++++++++++++++++++++++++++++------------
 2 files changed, 35 insertions(+), 15 deletions(-)

-Peff