Re: make test

["Joachim Schmitz" <jojo@schmitz-digital.de>]

"Joachim Schmitz" <jojo@schmitz-digital.de> schrieb im Newsbeitrag news:<k5gov5$fe1$1@ger.gmane.org>...
> Hi folks
> 
> I'm trying to understand why certain tests in 'make test' fail. Here's the 
> first one
> 
> $ ../git --version
> git version 1.8.0.rc2.5.g6b89306
> $ GIT_TEST_CMP_USE_COPIED_CONTEXT=true ./t0000-basic.sh # our diff doesn't 
> understand -u
> ok 1 - .git/objects should be empty after git init in an empty repo
> ...
> ok 3 - success is reported like this
> not ok 4 - pretend we have a known breakage # TODO known breakage
> 
>     This is expected, right?
> 
> ok 5 - pretend we have fixed a known breakage (run in sub test-lib)
> ...
> ok 11 - tests clean up after themselves
> 
>     the next is not though? Why might it be failing, where to check?
> 
> not ok - 12 tests clean up even on failures
> #
> #               mkdir failing-cleanup &&
> #               (
> #               cd failing-cleanup &&
> #
> #               cat >failing-cleanup.sh <<-EOF &&
> #               #!/bin/sh
> #
> #               test_description='Failing tests with cleanup commands'
> #
> #               # Point to the t/test-lib.sh, which isn't in ../ as usual
> #               TEST_DIRECTORY="/home/jojo/git/git/t"
> #               . "$TEST_DIRECTORY"/test-lib.sh
> #
> #               test_expect_success 'tests clean up even after a failure' '
> #                       touch clean-after-failure &&
> #                       test_when_finished rm clean-after-failure &&
> #                       (exit 1)
> #               '
> #               test_expect_success 'failure to clean up causes the test to 
> fail' '
> #                       test_when_finished "(exit 2)"
> #               '
> #               test_done
> #
> #               EOF
> #
> #               chmod +x failing-cleanup.sh &&
> #               test_must_fail ./failing-cleanup.sh >out 2>err &&
> #               ! test -s err &&
> #               ! test -f "trash 
> directory.failing-cleanup/clean-after-failure" &&
> #               sed -e 's/Z$//' -e 's/^> //' >expect <<-\EOF &&
> #               > not ok - 1 tests clean up even after a failure
> #               > #     Z
> #               > #     touch clean-after-failure &&
> #               > #     test_when_finished rm clean-after-failure &&
> #               > #     (exit 1)
> #               > #     Z
> #               > not ok - 2 failure to clean up causes the test to fail
> #               > #     Z
> #               > #     test_when_finished "(exit 2)"
> #               > #     Z
> #               > # failed 2 among 2 test(s)
> #               > 1..2
> #               EOF
> #               test_cmp expect out
> #               )
> #
> ok 13 - git update-index without --add should fail adding
> ...
> ok 47 - very long name in the index handled sanely
> # still have 1 known breakage(s)
> # failed 1 among remaining 46 test(s)
> 1..47

As mentioned elsethread this works if using bash rather than the system's sh (which is a ksh)

But there are several other failures. After some investigations and experiments I found the following tests to fail with the system
provided grep (for which I had to set GIT_TEST_CMP_USE_COPIED_CONTEXT), but succeed with GNU grep:
t3308 #14, #15, #17and #19
t3310 #10, #12, #14 and #18
t4047 #38 and #39
t4050 #2 and #3
t4116 #3, #4 and #5
t5509 #2
t7401 #18

The following fail with the system provided tar, but succeed with GNU tar:
t0024 #2
t4116 #4,
t5000 #14, #16, #20, #24, #26 and #51
t5001 #2, #6, #10 and #15

The following tests fail with the system provided sh (which is a ksh really), but succeed with bash:
t0000 #12
t0001 #20
t1450 #17 and #18
(t0204 #3 and #8 succeed in sh but fail in bash. They succeed in bash too when /usr/local/bin is in PATH first though, which would
sort the diff and tar problem too, need to investigate why)
t3006 #2 and #3

t3403 #4, #5, #8 and #9
t3404 #2 - #13, #14 - #18, #20 - #41, #44, #46 - #70
t3409 #2 - #5
t3410 #2 and #3
t3411 #2 and #3
t3412 #8, #10 - #12, #15, #17, #23, #25, -26, #28, #29, #31
t3413 #3, #5 - #10, #14, #15
and many more...

The following needs bash and /usr/local/bin first in PATH ("PATH=/usr/local/bin:$PATH make test")
t0204 #3 and #8 (or just sh, see above)
t3032 #11
t3900 #24, #25
t4201 #8
t5000 #14
t5150 #6

I though "SANE_TOOL_PATH=/usr/local/bin" plus "SHELL_PATH=/usr/local/bin/bash" would to fix the all but it does not and instead
brings up some other failures too:
t5521 #2 and #5
t5526 #2, #5, #8, #10, #12, #13, #16 - #19, #21 - #25
t5702 #3
t5800 #2, #3, #5 - #14
t9001 #66

With additionally having "PATH=/usr/local/bin:$PATH" all but one work, so there must be something wrong with SANE_TOOL_PATH?.
The single failure remaining is
t0301 #12 "helper (cache --timeout=1) times out"
I don't understand this at all, neither the -v options nor running it with bash -x helps me in understanding what the issue is.

Bye, Jojo