RE: make test

["Joachim Schmitz" <jojo@schmitz-digital.de>]

> From: Johannes Sixt [mailto:j.sixt@viscovery.net]
> Sent: Monday, October 15, 2012 12:53 PM
> To: Joachim Schmitz
> Cc: git@vger.kernel.org
> Subject: Re: make test
> 
> Am 10/15/2012 12:36, schrieb Joachim Schmitz:
> > not ok 4 - pretend we have a known breakage # TODO known breakage
> >
> >    This is expected, right?
> 
> Right.
> 
> >    the next is not though? Why might it be failing, where to check?
> >
> > not ok - 12 tests clean up even on failures
> > #
> > #               mkdir failing-cleanup &&
> > #...
> > #               test_cmp expect out
> > #               )
> > #
> 
> First thing:
> 
>   ./t0000-basic.sh -v -i

OK, I should have mentioned that I did look with -v :
not ok - 12 tests clean up even on failures
#
#               mkdir failing-cleanup &&
#               (
#               cd failing-cleanup &&
#
#               cat >failing-cleanup.sh <<-EOF &&
#               #!/bin/sh
#
#               test_description='Failing tests with cleanup commands'
#
#               # Point to the t/test-lib.sh, which isn't in ../ as usual
#               TEST_DIRECTORY="/home/jojo/git/git/t"
#               . "$TEST_DIRECTORY"/test-lib.sh
#
#               test_expect_success 'tests clean up even after a failure' '
#                       touch clean-after-failure &&
#                       test_when_finished rm clean-after-failure &&
#                       (exit 1)
#               '
#               test_expect_success 'failure to clean up causes the test to fail' '
#                       test_when_finished "(exit 2)"
#               '
#               test_done
#
#               EOF
#
#               chmod +x failing-cleanup.sh &&
#               test_must_fail ./failing-cleanup.sh >out 2>err &&
#               ! test -s err &&
#               ! test -f "trash directory.failing-cleanup/clean-after-failure" &&
#               sed -e 's/Z$//' -e 's/^> //' >expect <<-\EOF &&
#               > not ok - 1 tests clean up even after a failure
#               > #     Z
#               > #     touch clean-after-failure &&
#               > #     test_when_finished rm clean-after-failure &&
#               > #     (exit 1)
#               > #     Z
#               > not ok - 2 failure to clean up causes the test to fail
#               > #     Z
#               > #     test_when_finished "(exit 2)"
#               > #     Z
#               > # failed 2 among 2 test(s)
#               > 1..2
#               EOF
#               test_cmp expect out
#               )
#

> and if that does not give sufficient clues,
> 
>   $SHELL_PATH -x ./t0000-basic.sh -v -i

not ok - 12 tests clean up even on failures
#
#               mkdir failing-cleanup &&
#               (
#               cd failing-cleanup &&
#
#               cat >failing-cleanup.sh <<-EOF &&
#               #!/bin/sh
#
#               test_description='Failing tests with cleanup commands'
#
#               # Point to the t/test-lib.sh, which isn't in ../ as usual
#               TEST_DIRECTORY="/home/jojo/git/git/t"
#               . "$TEST_DIRECTORY"/test-lib.sh
#
#               test_expect_success 'tests clean up even after a failure' '
#                       touch clean-after-failure &&
#                       test_when_finished rm clean-after-failure &&
#                       (exit 1)
#               '
#               test_expect_success 'failure to clean up causes the test to fail' '
#                       test_when_finished "(exit 2)"
#               '
#               test_done
#
#               EOF
#
#               chmod +x failing-cleanup.sh &&
#               test_must_fail ./failing-cleanup.sh >out 2>err &&
#               ! test -s err &&
#               ! test -f "trash directory.failing-cleanup/clean-after-failure" &&
#               sed -e 's/Z$//' -e 's/^> //' >expect <<-\EOF &&
#               > not ok - 1 tests clean up even after a failure
#               > #     Z
#               > #     touch clean-after-failure &&
#               > #     test_when_finished rm clean-after-failure &&
#               > #     (exit 1)
#               > #     Z
#               > not ok - 2 failure to clean up causes the test to fail
#               > #     Z
#               > #     test_when_finished "(exit 2)"
#               > #     Z
#               > # failed 2 among 2 test(s)
#               > 1..2
#               EOF
#               test_cmp expect out
#               )
#
+ die

Looks identical, except for the "die" at the end. And still leaves me without a clue...

> (Beware, though: in some cases, the latter gives additional failures, in
> particular, when the stderr of a command is checked for with test_cmp
> instead of grep because the 'actual' results contain the shell command
> logs, which are not in the 'expected' results.)
> 
> -- Hannes,

Bye, Jojo