* [PATCH v2 0/3] Make the Git tests emit TAP format
@ 2010-06-09 15:22 Ævar Arnfjörð Bjarmason
2010-06-09 15:22 ` [PATCH v2 1/3] Make test-lib.sh emit valid " Ævar Arnfjörð Bjarmason
` (3 more replies)
0 siblings, 4 replies; 20+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2010-06-09 15:22 UTC (permalink / raw
To: git; +Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason
This is a re-submission of my previous TAP patch series to apply
cleanly to the 'next' branch. 2/6 patches from the series already made
it into 'next', so I've rebased the series accordingly. I've also
removed the "Add test_harness make target for testing with prove(1)"
that allowed running `make test_harness' at the top-level sinc Junio
didn't see much point in it.
To recap on what this series is all about:
TAP, the Test Anything Protocol, is a simple text-based interface
between testing modules in a test harness. test-lib.sh's output was
already very close to being valid TAP. This change brings it all the
way there.
All this series does is slightly adjust the raw text output of our
tests so that it conforms. to the TAP standard, i.e. instead of this:
$ ./t0005-signals.sh
* ok 1: sigchain works
* passed all 1 test(s)
We get this:
$ ./t0005-signals.sh
ok 1 - sigchain works
# passed all 1 test(s)
1..1
Changing the output format like this gives us the ability to run the
Git tests with any TAP tool (like prove(1)) at no extra cost. Every
other existing way of running the tests continues to work, it's just
easier for machines to read the output now.
Ævar Arnfjörð Bjarmason (3):
Make test-lib.sh emit valid TAP format
Skip tests in a way that makes sense under TAP
We use TAP so the Perl test can run without scaffolding
t/README | 49 +++++++++++------
t/lib-git-svn.sh | 4 +-
t/lib-httpd.sh | 3 +-
t/t1304-default-acl.sh | 9 ++-
t/t1509-root-worktree.sh | 6 +-
t/t2007-checkout-symlink.sh | 2 +-
t/t3300-funny-names.sh | 2 +-
t/t3302-notes-index-expensive.sh | 2 +-
t/t3600-rm.sh | 2 +-
t/t3701-add-interactive.sh | 4 +-
t/t3902-quoted.sh | 2 +-
t/t4004-diff-rename-symlink.sh | 2 +-
t/t4011-diff-symlink.sh | 2 +-
t/t4016-diff-quote.sh | 2 +-
t/t4023-diff-rename-typechange.sh | 2 +-
t/t4114-apply-typechange.sh | 2 +-
t/t4115-apply-symlink.sh | 2 +-
t/t4122-apply-symlink-inside.sh | 2 +-
t/t5302-pack-index.sh | 2 +-
t/t5503-tagfollow.sh | 2 +-
t/t5522-pull-symlink.sh | 2 +-
t/t5540-http-push.sh | 2 +-
t/t5541-http-push.sh | 2 +-
t/t5550-http-fetch.sh | 2 +-
t/t5551-http-fetch.sh | 2 +-
t/t5561-http-backend.sh | 2 +-
t/t5705-clone-2gb.sh | 2 +-
t/t6035-merge-dir-to-symlink.sh | 2 +-
t/t7004-tag.sh | 2 +-
t/t7006-pager.sh | 2 +-
t/t7800-difftool.sh | 2 +-
t/t9001-send-email.sh | 4 +-
t/t9100-git-svn-basic.sh | 2 +-
t/t9119-git-svn-info.sh | 2 +-
t/t9129-git-svn-i18n-commitencoding.sh | 2 +-
t/t9143-git-svn-gc.sh | 2 +-
t/t9200-git-cvsexportcommit.sh | 4 +-
t/t9400-git-cvsserver-server.sh | 6 +-
t/t9401-git-cvsserver-crlf.sh | 6 +-
t/t9600-cvsimport.sh | 2 +-
t/t9700-perl-git.sh | 11 ++--
t/t9700/test.pl | 9 +++
t/test-lib.sh | 92 ++++++-------------------------
43 files changed, 119 insertions(+), 148 deletions(-)
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v2 1/3] Make test-lib.sh emit valid TAP format
2010-06-09 15:22 [PATCH v2 0/3] Make the Git tests emit TAP format Ævar Arnfjörð Bjarmason
@ 2010-06-09 15:22 ` Ævar Arnfjörð Bjarmason
2010-06-14 22:01 ` Jakub Narebski
2010-06-09 15:22 ` [PATCH v2 2/3] Skip tests in a way that makes sense under TAP Ævar Arnfjörð Bjarmason
` (2 subsequent siblings)
3 siblings, 1 reply; 20+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2010-06-09 15:22 UTC (permalink / raw
To: git; +Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason
TAP, the Test Anything Protocol, is a simple text-based interface
between testing modules in a test harness. test-lib.sh's output was
already very close to being valid TAP. This change brings it all the
way there. Before:
$ ./t0005-signals.sh
* ok 1: sigchain works
* passed all 1 test(s)
And after:
$ ./t0005-signals.sh
ok 1 - sigchain works
# passed all 1 test(s)
1..1
The advantage of using TAP is that any program that reads the format
(a "test harness") can run the tests. The most popular of these is the
prove(1) utility that comes with Perl. It can run tests in parallel,
display colored output, format the output to console, file, HTML etc.,
and much more. An example:
$ prove ./t0005-signals.sh
./t0005-signals.sh .. ok
All tests successful.
Files=1, Tests=1, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.01 cusr 0.02 csys = 0.06 CPU)
Result: PASS
prove(1) gives you human readable output without being too
verbose. Running the test suite in parallel with `make test -j15`
produces a flood of text. Running them with `prove -j 15 ./t[0-9]*.sh`
makes it easy to follow what's going on.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
t/README | 49 +++++++++++++++++++++++++++++++++----------------
t/test-lib.sh | 30 ++++++++++++++++++------------
2 files changed, 51 insertions(+), 28 deletions(-)
diff --git a/t/README b/t/README
index dcd3ebb..fc4bb04 100644
--- a/t/README
+++ b/t/README
@@ -18,25 +18,42 @@ The easiest way to run tests is to say "make". This runs all
the tests.
*** t0000-basic.sh ***
- * ok 1: .git/objects should be empty after git-init in an empty repo.
- * ok 2: .git/objects should have 256 subdirectories.
- * ok 3: git-update-index without --add should fail adding.
+ ok 1 - .git/objects should be empty after git init in an empty repo.
+ ok 2 - .git/objects should have 3 subdirectories.
+ ok 3 - success is reported like this
...
- * ok 23: no diff after checkout and git-update-index --refresh.
- * passed all 23 test(s)
- *** t0100-environment-names.sh ***
- * ok 1: using old names should issue warnings.
- * ok 2: using old names but having new names should not issue warnings.
- ...
-
-Or you can run each test individually from command line, like
-this:
-
- $ sh ./t3001-ls-files-killed.sh
- * ok 1: git-update-index --add to add various paths.
- * ok 2: git-ls-files -k to show killed files.
- * ok 3: validate git-ls-files -k output.
- * passed all 3 test(s)
+ ok 43 - very long name in the index handled sanely
+ # fixed 1 known breakage(s)
+ # still have 1 known breakage(s)
+ # passed all remaining 42 test(s)
+ 1..43
+ *** t0001-init.sh ***
+ ok 1 - plain
+ ok 2 - plain with GIT_WORK_TREE
+ ok 3 - plain bare
+
+Since the tests all output TAP (see http://testanything.org) they can
+be run with any TAP harness. Here's an example of paralell testing
+powered by a recent version of prove(1):
+
+ $ prove --timer --jobs 15 ./t[0-9]*.sh
+ [19:17:33] ./t0005-signals.sh ................................... ok 36 ms
+ [19:17:33] ./t0022-crlf-rename.sh ............................... ok 69 ms
+ [19:17:33] ./t0024-crlf-archive.sh .............................. ok 154 ms
+ [19:17:33] ./t0004-unwritable.sh ................................ ok 289 ms
+ [19:17:33] ./t0002-gitfile.sh ................................... ok 480 ms
+ ===( 102;0 25/? 6/? 5/? 16/? 1/? 4/? 2/? 1/? 3/? 1... )===
+
+You can also run each test individually from command line, like this:
+
+ $ sh ./t3010-ls-files-killed-modified.sh
+ ok 1 - git update-index --add to add various paths.
+ ok 2 - git ls-files -k to show killed files.
+ ok 3 - validate git ls-files -k output.
+ ok 4 - git ls-files -m to show modified files.
+ ok 5 - validate git ls-files -m output.
+ # passed all 5 test(s)
+ 1..5
You can pass --verbose (or -v), --debug (or -d), and --immediate
(or -i) command line argument to the test, or by setting GIT_TEST_OPTS
diff --git a/t/test-lib.sh b/t/test-lib.sh
index b23a61d..82fd0ef 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -170,7 +170,7 @@ if test -n "$color"; then
*) test -n "$quiet" && return;;
esac
shift
- printf "* %s" "$*"
+ printf "%s" "$*"
tput sgr0
echo
)
@@ -179,7 +179,7 @@ else
say_color() {
test -z "$1" && test -n "$quiet" && return
shift
- echo "* $*"
+ echo "$*"
}
fi
@@ -349,25 +349,25 @@ test_have_prereq () {
test_ok_ () {
test_success=$(($test_success + 1))
- say_color "" " ok $test_count: $@"
+ say_color "" "ok $test_count - $@"
}
test_failure_ () {
test_failure=$(($test_failure + 1))
- say_color error "FAIL $test_count: $1"
+ say_color error "not ok - $test_count $1"
shift
- echo "$@" | sed -e 's/^/ /'
+ echo "$@" | sed -e 's/^/# /'
test "$immediate" = "" || { GIT_EXIT_OK=t; exit 1; }
}
test_known_broken_ok_ () {
test_fixed=$(($test_fixed+1))
- say_color "" " FIXED $test_count: $@"
+ say_color "" "ok $test_count - $@ # TODO known breakage"
}
test_known_broken_failure_ () {
test_broken=$(($test_broken+1))
- say_color skip " still broken $test_count: $@"
+ say_color skip "not ok $test_count - $@ # TODO known breakage"
}
test_debug () {
@@ -400,7 +400,7 @@ test_skip () {
case "$to_skip" in
t)
say_color skip >&3 "skipping test: $@"
- say_color skip "skip $test_count: $1"
+ say_color skip "ok $test_count: # skip $1"
: true
;;
*)
@@ -630,18 +630,22 @@ test_done () {
if test "$test_fixed" != 0
then
- say_color pass "fixed $test_fixed known breakage(s)"
+ say_color pass "# fixed $test_fixed known breakage(s)"
fi
if test "$test_broken" != 0
then
- say_color error "still have $test_broken known breakage(s)"
+ say_color error "# still have $test_broken known breakage(s)"
msg="remaining $(($test_count-$test_broken)) test(s)"
else
msg="$test_count test(s)"
fi
case "$test_failure" in
0)
- say_color pass "passed all $msg"
+ # Maybe print SKIP message
+ [ -z "$skip_all" ] || skip_all=" # SKIP $skip_all"
+
+ say_color pass "# passed all $msg"
+ say "1..$test_count$skip_all"
test -d "$remove_trash" &&
cd "$(dirname "$remove_trash")" &&
@@ -650,7 +654,9 @@ test_done () {
exit 0 ;;
*)
- say_color error "failed $test_failure among $msg"
+ say_color error "# failed $test_failure among $msg"
+ say "1..$test_count"
+
exit 1 ;;
esac
--
1.7.1.243.gda92d6.dirty
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v2 2/3] Skip tests in a way that makes sense under TAP
2010-06-09 15:22 [PATCH v2 0/3] Make the Git tests emit TAP format Ævar Arnfjörð Bjarmason
2010-06-09 15:22 ` [PATCH v2 1/3] Make test-lib.sh emit valid " Ævar Arnfjörð Bjarmason
@ 2010-06-09 15:22 ` Ævar Arnfjörð Bjarmason
2010-06-09 15:24 ` [PATCH v2 3/3] We use TAP so the Perl test can run without scaffolding Ævar Arnfjörð Bjarmason
2010-06-14 21:49 ` [PATCH v2 0/3] Make the Git tests emit TAP format Jakub Narebski
3 siblings, 0 replies; 20+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2010-06-09 15:22 UTC (permalink / raw
To: git; +Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason
SKIP messages are now part of the TAP plan. A TAP harness now knows
why a particular test was skipped and can report that information.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
t/lib-git-svn.sh | 4 ++--
t/lib-httpd.sh | 3 +--
t/t1304-default-acl.sh | 9 ++++++---
t/t1509-root-worktree.sh | 6 +++---
t/t2007-checkout-symlink.sh | 2 +-
t/t3300-funny-names.sh | 2 +-
t/t3302-notes-index-expensive.sh | 2 +-
t/t3600-rm.sh | 2 +-
t/t3701-add-interactive.sh | 4 ++--
t/t3902-quoted.sh | 2 +-
t/t4004-diff-rename-symlink.sh | 2 +-
t/t4011-diff-symlink.sh | 2 +-
t/t4016-diff-quote.sh | 2 +-
t/t4023-diff-rename-typechange.sh | 2 +-
t/t4114-apply-typechange.sh | 2 +-
t/t4115-apply-symlink.sh | 2 +-
t/t4122-apply-symlink-inside.sh | 2 +-
t/t5302-pack-index.sh | 2 +-
t/t5503-tagfollow.sh | 2 +-
t/t5522-pull-symlink.sh | 2 +-
t/t5540-http-push.sh | 2 +-
t/t5541-http-push.sh | 2 +-
t/t5550-http-fetch.sh | 2 +-
t/t5551-http-fetch.sh | 2 +-
t/t5561-http-backend.sh | 2 +-
t/t5705-clone-2gb.sh | 2 +-
t/t6035-merge-dir-to-symlink.sh | 2 +-
t/t7004-tag.sh | 2 +-
| 2 +-
t/t7800-difftool.sh | 2 +-
t/t9001-send-email.sh | 4 ++--
t/t9100-git-svn-basic.sh | 2 +-
t/t9119-git-svn-info.sh | 2 +-
t/t9129-git-svn-i18n-commitencoding.sh | 2 +-
t/t9143-git-svn-gc.sh | 2 +-
t/t9200-git-cvsexportcommit.sh | 4 ++--
t/t9400-git-cvsserver-server.sh | 6 +++---
t/t9401-git-cvsserver-crlf.sh | 6 +++---
t/t9600-cvsimport.sh | 2 +-
t/t9700-perl-git.sh | 4 ++--
40 files changed, 56 insertions(+), 54 deletions(-)
diff --git a/t/lib-git-svn.sh b/t/lib-git-svn.sh
index 0f7f35c..344785d 100644
--- a/t/lib-git-svn.sh
+++ b/t/lib-git-svn.sh
@@ -5,11 +5,11 @@ git_svn_id=git""-svn-id
if test -n "$NO_SVN_TESTS"
then
- say 'skipping git svn tests, NO_SVN_TESTS defined'
+ skip_all='skipping git svn tests, NO_SVN_TESTS defined'
test_done
fi
if ! test_have_prereq PERL; then
- say 'skipping git svn tests, perl not available'
+ skip_all='skipping git svn tests, perl not available'
test_done
fi
diff --git a/t/lib-httpd.sh b/t/lib-httpd.sh
index da4b8d5..a0944d6 100644
--- a/t/lib-httpd.sh
+++ b/t/lib-httpd.sh
@@ -5,8 +5,7 @@
if test -z "$GIT_TEST_HTTPD"
then
- say "skipping test, network testing disabled by default"
- say "(define GIT_TEST_HTTPD to enable)"
+ skip_all="Network testing disabled (define GIT_TEST_HTTPD to enable)"
test_done
fi
diff --git a/t/t1304-default-acl.sh b/t/t1304-default-acl.sh
index 055ad00..97ab02a 100755
--- a/t/t1304-default-acl.sh
+++ b/t/t1304-default-acl.sh
@@ -15,9 +15,12 @@ umask 077
# is a good candidate: exists on all unices, and it has permission
# anyway, so we don't create a security hole running the testsuite.
-if ! setfacl -m u:root:rwx .; then
- say "Skipping ACL tests: unable to use setfacl"
- test_done
+setfacl_out="$(setfacl -m u:root:rwx . 2>&1)"
+setfacl_ret=$?
+
+if [ $setfacl_ret != 0 ]; then
+ skip_all="Skipping ACL tests: unable to use setfacl (output: '$setfacl_out'; return code: '$setfacl_ret')"
+ test_done
fi
check_perms_and_acl () {
diff --git a/t/t1509-root-worktree.sh b/t/t1509-root-worktree.sh
index 5322a3b..7f60fd0 100755
--- a/t/t1509-root-worktree.sh
+++ b/t/t1509-root-worktree.sh
@@ -99,17 +99,17 @@ test_foobar_foobar() {
}
if ! test_have_prereq POSIXPERM || ! [ -w / ]; then
- say "Dangerous test skipped. Read this test if you want to execute it"
+ skip_all="Dangerous test skipped. Read this test if you want to execute it"
test_done
fi
if [ "$IKNOWWHATIAMDOING" != "YES" ]; then
- say "You must set env var IKNOWWHATIAMDOING=YES in order to run this test"
+ skip_all="You must set env var IKNOWWHATIAMDOING=YES in order to run this test"
test_done
fi
if [ "$UID" = 0 ]; then
- say "No you can't run this with root"
+ skip_all="No you can't run this with root"
test_done
fi
diff --git a/t/t2007-checkout-symlink.sh b/t/t2007-checkout-symlink.sh
index 27e2127..05cc8fd 100755
--- a/t/t2007-checkout-symlink.sh
+++ b/t/t2007-checkout-symlink.sh
@@ -8,7 +8,7 @@ test_description='git checkout to switch between branches with symlink<->dir'
if ! test_have_prereq SYMLINKS
then
- say "symbolic links not supported - skipping tests"
+ skip_all="symbolic links not supported - skipping tests"
test_done
fi
diff --git a/t/t3300-funny-names.sh b/t/t3300-funny-names.sh
index db46d53..a99e4d8 100755
--- a/t/t3300-funny-names.sh
+++ b/t/t3300-funny-names.sh
@@ -26,7 +26,7 @@ echo 'Foo Bar Baz' >"$p2"
test -f "$p1" && cmp "$p0" "$p1" || {
# since FAT/NTFS does not allow tabs in filenames, skip this test
- say 'Your filesystem does not allow tabs in filenames, test skipped.'
+ skip_all='Your filesystem does not allow tabs in filenames, test skipped.'
test_done
}
diff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh
index ee84fc4..361a10a 100755
--- a/t/t3302-notes-index-expensive.sh
+++ b/t/t3302-notes-index-expensive.sh
@@ -8,7 +8,7 @@ test_description='Test commit notes index (expensive!)'
. ./test-lib.sh
test -z "$GIT_NOTES_TIMING_TESTS" && {
- say Skipping timing tests
+ skip_all="Skipping timing tests"
test_done
exit
}
diff --git a/t/t3600-rm.sh b/t/t3600-rm.sh
index 0aaf0ad..b514cbb 100755
--- a/t/t3600-rm.sh
+++ b/t/t3600-rm.sh
@@ -39,7 +39,7 @@ if test -f test-file
then
test_set_prereq RO_DIR
else
- say 'skipping removal failure test (perhaps running as root?)'
+ skip_all='skipping removal failure test (perhaps running as root?)'
fi
chmod 775 .
rm -f test-file
diff --git a/t/t3701-add-interactive.sh b/t/t3701-add-interactive.sh
index b6eba6a..7ad8465 100755
--- a/t/t3701-add-interactive.sh
+++ b/t/t3701-add-interactive.sh
@@ -4,7 +4,7 @@ test_description='add -i basic tests'
. ./test-lib.sh
if ! test_have_prereq PERL; then
- say 'skipping git add -i tests, perl not available'
+ skip_all='skipping git add -i tests, perl not available'
test_done
fi
@@ -154,7 +154,7 @@ rm -f .gitignore
if test "$(git config --bool core.filemode)" = false
then
- say 'skipping filemode tests (filesystem does not properly support modes)'
+ say '# skipping filemode tests (filesystem does not properly support modes)'
else
test_set_prereq FILEMODE
fi
diff --git a/t/t3902-quoted.sh b/t/t3902-quoted.sh
index 29103f6..147e634 100755
--- a/t/t3902-quoted.sh
+++ b/t/t3902-quoted.sh
@@ -17,7 +17,7 @@ DQ='"'
echo foo 2>/dev/null > "Name and an${HT}HT"
test -f "Name and an${HT}HT" || {
# since FAT/NTFS does not allow tabs in filenames, skip this test
- say 'Your filesystem does not allow tabs in filenames, test skipped.'
+ skip_all='Your filesystem does not allow tabs in filenames, test skipped.'
test_done
}
diff --git a/t/t4004-diff-rename-symlink.sh b/t/t4004-diff-rename-symlink.sh
index a4da119..1a09e8d 100755
--- a/t/t4004-diff-rename-symlink.sh
+++ b/t/t4004-diff-rename-symlink.sh
@@ -14,7 +14,7 @@ by an edit for them.
if ! test_have_prereq SYMLINKS
then
- say 'Symbolic links not supported, skipping tests.'
+ skip_all='Symbolic links not supported, skipping tests.'
test_done
fi
diff --git a/t/t4011-diff-symlink.sh b/t/t4011-diff-symlink.sh
index e12fbea..918a21a 100755
--- a/t/t4011-diff-symlink.sh
+++ b/t/t4011-diff-symlink.sh
@@ -11,7 +11,7 @@ test_description='Test diff of symlinks.
if ! test_have_prereq SYMLINKS
then
- say 'Symbolic links not supported, skipping tests.'
+ skip_all='Symbolic links not supported, skipping tests.'
test_done
fi
diff --git a/t/t4016-diff-quote.sh b/t/t4016-diff-quote.sh
index 55eb5f8..34e5144 100755
--- a/t/t4016-diff-quote.sh
+++ b/t/t4016-diff-quote.sh
@@ -14,7 +14,7 @@ P2='pathname with SP'
P3='pathname
with LF'
: 2>/dev/null >"$P1" && test -f "$P1" && rm -f "$P1" || {
- say 'Your filesystem does not allow tabs in filenames, test skipped.'
+ skip_all='Your filesystem does not allow tabs in filenames, test skipped.'
test_done
}
diff --git a/t/t4023-diff-rename-typechange.sh b/t/t4023-diff-rename-typechange.sh
index 9bdf659..40a95a1 100755
--- a/t/t4023-diff-rename-typechange.sh
+++ b/t/t4023-diff-rename-typechange.sh
@@ -6,7 +6,7 @@ test_description='typechange rename detection'
if ! test_have_prereq SYMLINKS
then
- say 'Symbolic links not supported, skipping tests.'
+ skip_all='Symbolic links not supported, skipping tests.'
test_done
fi
diff --git a/t/t4114-apply-typechange.sh b/t/t4114-apply-typechange.sh
index 99ec13d..164d58c 100755
--- a/t/t4114-apply-typechange.sh
+++ b/t/t4114-apply-typechange.sh
@@ -11,7 +11,7 @@ test_description='git apply should not get confused with type changes.
if ! test_have_prereq SYMLINKS
then
- say 'Symbolic links not supported, skipping tests.'
+ skip_all='Symbolic links not supported, skipping tests.'
test_done
fi
diff --git a/t/t4115-apply-symlink.sh b/t/t4115-apply-symlink.sh
index b852e58..aff4348 100755
--- a/t/t4115-apply-symlink.sh
+++ b/t/t4115-apply-symlink.sh
@@ -11,7 +11,7 @@ test_description='git apply symlinks and partial files
if ! test_have_prereq SYMLINKS
then
- say 'Symbolic links not supported, skipping tests.'
+ skip_all='Symbolic links not supported, skipping tests.'
test_done
fi
diff --git a/t/t4122-apply-symlink-inside.sh b/t/t4122-apply-symlink-inside.sh
index 0d3c1d5..923fcab 100755
--- a/t/t4122-apply-symlink-inside.sh
+++ b/t/t4122-apply-symlink-inside.sh
@@ -5,7 +5,7 @@ test_description='apply to deeper directory without getting fooled with symlink'
if ! test_have_prereq SYMLINKS
then
- say 'Symbolic links not supported, skipping tests.'
+ skip_all='Symbolic links not supported, skipping tests.'
test_done
fi
diff --git a/t/t5302-pack-index.sh b/t/t5302-pack-index.sh
index 4360e77..fb3a270 100755
--- a/t/t5302-pack-index.sh
+++ b/t/t5302-pack-index.sh
@@ -74,7 +74,7 @@ if msg=$(git verify-pack -v "test-3-${pack3}.pack" 2>&1) ||
then
test_set_prereq OFF64_T
else
- say "skipping tests concerning 64-bit offsets"
+ say "# skipping tests concerning 64-bit offsets"
fi
test_expect_success OFF64_T \
diff --git a/t/t5503-tagfollow.sh b/t/t5503-tagfollow.sh
index d5db75d..bab1a53 100755
--- a/t/t5503-tagfollow.sh
+++ b/t/t5503-tagfollow.sh
@@ -6,7 +6,7 @@ test_description='test automatic tag following'
case $(uname -s) in
*MINGW*)
- say "GIT_DEBUG_SEND_PACK not supported - skipping tests"
+ skip_all="GIT_DEBUG_SEND_PACK not supported - skipping tests"
test_done
esac
diff --git a/t/t5522-pull-symlink.sh b/t/t5522-pull-symlink.sh
index 7206817..298200f 100755
--- a/t/t5522-pull-symlink.sh
+++ b/t/t5522-pull-symlink.sh
@@ -6,7 +6,7 @@ test_description='pulling from symlinked subdir'
if ! test_have_prereq SYMLINKS
then
- say 'Symbolic links not supported, skipping tests.'
+ skip_all='Symbolic links not supported, skipping tests.'
test_done
fi
diff --git a/t/t5540-http-push.sh b/t/t5540-http-push.sh
index 37fe875..a266ca5 100755
--- a/t/t5540-http-push.sh
+++ b/t/t5540-http-push.sh
@@ -11,7 +11,7 @@ This test runs various sanity checks on http-push.'
if git http-push > /dev/null 2>&1 || [ $? -eq 128 ]
then
- say "skipping test, USE_CURL_MULTI is not defined"
+ skip_all="skipping test, USE_CURL_MULTI is not defined"
test_done
fi
diff --git a/t/t5541-http-push.sh b/t/t5541-http-push.sh
index 17e1bdc..504884b 100755
--- a/t/t5541-http-push.sh
+++ b/t/t5541-http-push.sh
@@ -7,7 +7,7 @@ test_description='test smart pushing over http via http-backend'
. ./test-lib.sh
if test -n "$NO_CURL"; then
- say 'skipping test, git built without http support'
+ skip_all='skipping test, git built without http support'
test_done
fi
diff --git a/t/t5550-http-fetch.sh b/t/t5550-http-fetch.sh
index fc675b5..2fb48d0 100755
--- a/t/t5550-http-fetch.sh
+++ b/t/t5550-http-fetch.sh
@@ -4,7 +4,7 @@ test_description='test dumb fetching over http via static file'
. ./test-lib.sh
if test -n "$NO_CURL"; then
- say 'skipping test, git built without http support'
+ skip_all='skipping test, git built without http support'
test_done
fi
diff --git a/t/t5551-http-fetch.sh b/t/t5551-http-fetch.sh
index 7faa31a..fd19121 100755
--- a/t/t5551-http-fetch.sh
+++ b/t/t5551-http-fetch.sh
@@ -4,7 +4,7 @@ test_description='test smart fetching over http via http-backend'
. ./test-lib.sh
if test -n "$NO_CURL"; then
- say 'skipping test, git built without http support'
+ skip_all='skipping test, git built without http support'
test_done
fi
diff --git a/t/t5561-http-backend.sh b/t/t5561-http-backend.sh
index 8c6d0b2..b5d7fbc 100755
--- a/t/t5561-http-backend.sh
+++ b/t/t5561-http-backend.sh
@@ -4,7 +4,7 @@ test_description='test git-http-backend'
. ./test-lib.sh
if test -n "$NO_CURL"; then
- say 'skipping test, git built without http support'
+ skip_all='skipping test, git built without http support'
test_done
fi
diff --git a/t/t5705-clone-2gb.sh b/t/t5705-clone-2gb.sh
index 8afbdd4..e4d1b6a 100755
--- a/t/t5705-clone-2gb.sh
+++ b/t/t5705-clone-2gb.sh
@@ -4,7 +4,7 @@ test_description='Test cloning a repository larger than 2 gigabyte'
. ./test-lib.sh
test -z "$GIT_TEST_CLONE_2GB" &&
-say "Skipping expensive 2GB clone test; enable it with GIT_TEST_CLONE_2GB=t" &&
+skip_all="Skipping expensive 2GB clone test; enable it with GIT_TEST_CLONE_2GB=t" &&
test_done &&
exit
diff --git a/t/t6035-merge-dir-to-symlink.sh b/t/t6035-merge-dir-to-symlink.sh
index 3202e1d..cd3190c 100755
--- a/t/t6035-merge-dir-to-symlink.sh
+++ b/t/t6035-merge-dir-to-symlink.sh
@@ -5,7 +5,7 @@ test_description='merging when a directory was replaced with a symlink'
if ! test_have_prereq SYMLINKS
then
- say 'Symbolic links not supported, skipping tests.'
+ skip_all='Symbolic links not supported, skipping tests.'
test_done
fi
diff --git a/t/t7004-tag.sh b/t/t7004-tag.sh
index 73dbc43..ac943f5 100755
--- a/t/t7004-tag.sh
+++ b/t/t7004-tag.sh
@@ -583,7 +583,7 @@ test_expect_success \
# subsequent tests require gpg; check if it is available
gpg --version >/dev/null 2>/dev/null
if [ $? -eq 127 ]; then
- say "gpg not found - skipping tag signing and verification tests"
+ say "# gpg not found - skipping tag signing and verification tests"
else
# As said here: http://www.gnupg.org/documentation/faqs.html#q6.19
# the gpg version 1.0.6 didn't parse trust packets correctly, so for
--git a/t/t7006-pager.sh b/t/t7006-pager.sh
index a6f3677..608f0ce 100755
--- a/t/t7006-pager.sh
+++ b/t/t7006-pager.sh
@@ -36,7 +36,7 @@ then
}
test_set_prereq TTY
else
- say no usable terminal, so skipping some tests
+ say "# no usable terminal, so skipping some tests"
fi
test_expect_success 'setup' '
diff --git a/t/t7800-difftool.sh b/t/t7800-difftool.sh
index 1de83ef..196827e 100755
--- a/t/t7800-difftool.sh
+++ b/t/t7800-difftool.sh
@@ -11,7 +11,7 @@ Testing basic diff tool invocation
. ./test-lib.sh
if ! test_have_prereq PERL; then
- say 'skipping difftool tests, perl not available'
+ skip_all='skipping difftool tests, perl not available'
test_done
fi
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 640b3d2..ddc3d8d 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -4,7 +4,7 @@ test_description='git send-email'
. ./test-lib.sh
if ! test_have_prereq PERL; then
- say 'skipping git send-email tests, perl not available'
+ skip_all='skipping git send-email tests, perl not available'
test_done
fi
@@ -58,7 +58,7 @@ test_no_confirm () {
# Exit immediately to prevent hang if a no-confirm test fails
check_no_confirm () {
test -f no_confirm_okay || {
- say 'No confirm test failed; skipping remaining tests to prevent hanging'
+ skip_all='confirm test failed; skipping remaining tests to prevent hanging'
test_done
}
}
diff --git a/t/t9100-git-svn-basic.sh b/t/t9100-git-svn-basic.sh
index 570e035..13766ab 100755
--- a/t/t9100-git-svn-basic.sh
+++ b/t/t9100-git-svn-basic.sh
@@ -15,7 +15,7 @@ case "$GIT_SVN_LC_ALL" in
test_set_prereq UTF8
;;
*)
- say "UTF-8 locale not set, some tests skipped ($GIT_SVN_LC_ALL)"
+ say "# UTF-8 locale not set, some tests skipped ($GIT_SVN_LC_ALL)"
;;
esac
diff --git a/t/t9119-git-svn-info.sh b/t/t9119-git-svn-info.sh
index a9a558d..5fb94fb 100755
--- a/t/t9119-git-svn-info.sh
+++ b/t/t9119-git-svn-info.sh
@@ -13,7 +13,7 @@ case $v in
1.[456].*)
;;
*)
- say "skipping svn-info test (SVN version: $v not supported)"
+ skip_all="skipping svn-info test (SVN version: $v not supported)"
test_done
;;
esac
diff --git a/t/t9129-git-svn-i18n-commitencoding.sh b/t/t9129-git-svn-i18n-commitencoding.sh
index 1e9a2eb..8cfdfe7 100755
--- a/t/t9129-git-svn-i18n-commitencoding.sh
+++ b/t/t9129-git-svn-i18n-commitencoding.sh
@@ -23,7 +23,7 @@ if test -n "$a_utf8_locale"
then
test_set_prereq UTF8
else
- say "UTF-8 locale not available, some tests are skipped"
+ say "# UTF-8 locale not available, some tests are skipped"
fi
compare_svn_head_with () {
diff --git a/t/t9143-git-svn-gc.sh b/t/t9143-git-svn-gc.sh
index 99f69c6..337ea59 100755
--- a/t/t9143-git-svn-gc.sh
+++ b/t/t9143-git-svn-gc.sh
@@ -43,7 +43,7 @@ then
gunzip .git/svn/refs/remotes/git-svn/unhandled.log.gz
'
else
- say "Perl Compress::Zlib unavailable, skipping gunzip test"
+ say "# Perl Compress::Zlib unavailable, skipping gunzip test"
fi
test_expect_success 'git svn gc does not change unhandled.log files' '
diff --git a/t/t9200-git-cvsexportcommit.sh b/t/t9200-git-cvsexportcommit.sh
index 61bcb8f..ee39b36 100755
--- a/t/t9200-git-cvsexportcommit.sh
+++ b/t/t9200-git-cvsexportcommit.sh
@@ -7,14 +7,14 @@ test_description='Test export of commits to CVS'
. ./test-lib.sh
if ! test_have_prereq PERL; then
- say 'skipping git cvsexportcommit tests, perl not available'
+ skip_all='skipping git cvsexportcommit tests, perl not available'
test_done
fi
cvs >/dev/null 2>&1
if test $? -ne 1
then
- say 'skipping git cvsexportcommit tests, cvs not found'
+ skip_all='skipping git cvsexportcommit tests, cvs not found'
test_done
fi
diff --git a/t/t9400-git-cvsserver-server.sh b/t/t9400-git-cvsserver-server.sh
index 8639506..36c457e 100755
--- a/t/t9400-git-cvsserver-server.sh
+++ b/t/t9400-git-cvsserver-server.sh
@@ -11,17 +11,17 @@ cvs CLI client via git-cvsserver server'
. ./test-lib.sh
if ! test_have_prereq PERL; then
- say 'skipping git cvsserver tests, perl not available'
+ skip_all='skipping git cvsserver tests, perl not available'
test_done
fi
cvs >/dev/null 2>&1
if test $? -ne 1
then
- say 'skipping git-cvsserver tests, cvs not found'
+ skip_all='skipping git-cvsserver tests, cvs not found'
test_done
fi
"$PERL_PATH" -e 'use DBI; use DBD::SQLite' >/dev/null 2>&1 || {
- say 'skipping git-cvsserver tests, Perl SQLite interface unavailable'
+ skip_all='skipping git-cvsserver tests, Perl SQLite interface unavailable'
test_done
}
diff --git a/t/t9401-git-cvsserver-crlf.sh b/t/t9401-git-cvsserver-crlf.sh
index ed7b513..925bd0f 100755
--- a/t/t9401-git-cvsserver-crlf.sh
+++ b/t/t9401-git-cvsserver-crlf.sh
@@ -41,16 +41,16 @@ not_present() {
cvs >/dev/null 2>&1
if test $? -ne 1
then
- say 'skipping git-cvsserver tests, cvs not found'
+ skip_all='skipping git-cvsserver tests, cvs not found'
test_done
fi
if ! test_have_prereq PERL
then
- say 'skipping git-cvsserver tests, perl not available'
+ skip_all='skipping git-cvsserver tests, perl not available'
test_done
fi
"$PERL_PATH" -e 'use DBI; use DBD::SQLite' >/dev/null 2>&1 || {
- say 'skipping git-cvsserver tests, Perl SQLite interface unavailable'
+ skip_all='skipping git-cvsserver tests, Perl SQLite interface unavailable'
test_done
}
diff --git a/t/t9600-cvsimport.sh b/t/t9600-cvsimport.sh
index b572ce3..2eff9cd 100755
--- a/t/t9600-cvsimport.sh
+++ b/t/t9600-cvsimport.sh
@@ -4,7 +4,7 @@ test_description='git cvsimport basic tests'
. ./lib-cvs.sh
if ! test_have_prereq PERL; then
- say 'skipping git cvsimport tests, perl not available'
+ skip_all='skipping git cvsimport tests, perl not available'
test_done
fi
diff --git a/t/t9700-perl-git.sh b/t/t9700-perl-git.sh
index 8686086..3999bf7 100755
--- a/t/t9700-perl-git.sh
+++ b/t/t9700-perl-git.sh
@@ -7,12 +7,12 @@ test_description='perl interface (Git.pm)'
. ./test-lib.sh
if ! test_have_prereq PERL; then
- say 'skipping perl interface tests, perl not available'
+ skip_all='skipping perl interface tests, perl not available'
test_done
fi
"$PERL_PATH" -MTest::More -e 0 2>/dev/null || {
- say "Perl Test::More unavailable, skipping test"
+ skip_all="Perl Test::More unavailable, skipping test"
test_done
}
--
1.7.1.243.gda92d6.dirty
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v2 3/3] We use TAP so the Perl test can run without scaffolding
2010-06-09 15:22 [PATCH v2 0/3] Make the Git tests emit TAP format Ævar Arnfjörð Bjarmason
2010-06-09 15:22 ` [PATCH v2 1/3] Make test-lib.sh emit valid " Ævar Arnfjörð Bjarmason
2010-06-09 15:22 ` [PATCH v2 2/3] Skip tests in a way that makes sense under TAP Ævar Arnfjörð Bjarmason
@ 2010-06-09 15:24 ` Ævar Arnfjörð Bjarmason
2010-06-14 21:49 ` [PATCH v2 0/3] Make the Git tests emit TAP format Jakub Narebski
3 siblings, 0 replies; 20+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2010-06-09 15:24 UTC (permalink / raw
To: git; +Cc: Junio C Hamano, Lea Wiemann,
Ævar Arnfjörð Bjarmason
This removes the test_external and test_external_without_stderr
functions added by Lea Wiemann in
fb32c410087e68d650b31f68e66b3d9cbcce4a56. Nothing else used them, and
now that we're using TAP they shouldn't be necessary.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
This patch removes functionality that I'm using for some new Perl
tests in my gettext patch series. If this gets accepted it's trivial
for me to send an update to the gettext series to use the new TAP-like
Perl tests.
t/t9700-perl-git.sh | 7 ++---
t/t9700/test.pl | 9 +++++++
t/test-lib.sh | 62 ---------------------------------------------------
3 files changed, 12 insertions(+), 66 deletions(-)
diff --git a/t/t9700-perl-git.sh b/t/t9700-perl-git.sh
index 3999bf7..6c22dbc 100755
--- a/t/t9700-perl-git.sh
+++ b/t/t9700-perl-git.sh
@@ -46,8 +46,7 @@ test_expect_success \
git config --add test.int 2k
'
-test_external_without_stderr \
- 'Perl API' \
- "$PERL_PATH" "$TEST_DIRECTORY"/t9700/test.pl
+"$PERL_PATH" "$TEST_DIRECTORY"/t9700/test.pl
-test_done
+# The Perl test finalizes the plan, so don't call test_done() here.
+GIT_EXIT_OK=t
diff --git a/t/t9700/test.pl b/t/t9700/test.pl
index 666722d..c1ac913 100755
--- a/t/t9700/test.pl
+++ b/t/t9700/test.pl
@@ -7,6 +7,13 @@ use strict;
use Test::More qw(no_plan);
+BEGIN {
+ # t9700-perl-git.sh kicks off our testing, so we have to go from
+ # there.
+ $Test::Builder::Test->{Curr_Test} = 1;
+ $Test::Builder::Test->{No_Ending} = 1;
+}
+
use Cwd;
use File::Basename;
@@ -105,3 +112,5 @@ my $last_commit = $r2->command_oneline(qw(rev-parse --verify HEAD));
like($last_commit, qr/^[0-9a-fA-F]{40}$/, 'rev-parse returned hash');
my $dir_commit = $r2->command_oneline('log', '-n1', '--pretty=format:%H', '.');
isnt($last_commit, $dir_commit, 'log . does not show last commit');
+
+printf "1..%d\n", $Test::Builder::Test->{Curr_Test};
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 82fd0ef..2355a34 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -463,68 +463,6 @@ test_expect_code () {
echo >&3 ""
}
-# test_external runs external test scripts that provide continuous
-# test output about their progress, and succeeds/fails on
-# zero/non-zero exit code. It outputs the test output on stdout even
-# in non-verbose mode, and announces the external script with "* run
-# <n>: ..." before running it. When providing relative paths, keep in
-# mind that all scripts run in "trash directory".
-# Usage: test_external description command arguments...
-# Example: test_external 'Perl API' perl ../path/to/test.pl
-test_external () {
- test "$#" = 4 && { prereq=$1; shift; } || prereq=
- test "$#" = 3 ||
- error >&5 "bug in the test script: not 3 or 4 parameters to test_external"
- descr="$1"
- shift
- if ! test_skip "$descr" "$@"
- then
- # Announce the script to reduce confusion about the
- # test output that follows.
- say_color "" " run $test_count: $descr ($*)"
- # Export TEST_DIRECTORY, TRASH_DIRECTORY and GIT_TEST_LONG
- # to be able to use them in script
- export TEST_DIRECTORY TRASH_DIRECTORY GIT_TEST_LONG
- # Run command; redirect its stderr to &4 as in
- # test_run_, but keep its stdout on our stdout even in
- # non-verbose mode.
- "$@" 2>&4
- if [ "$?" = 0 ]
- then
- test_ok_ "$descr"
- else
- test_failure_ "$descr" "$@"
- fi
- fi
-}
-
-# Like test_external, but in addition tests that the command generated
-# no output on stderr.
-test_external_without_stderr () {
- # The temporary file has no (and must have no) security
- # implications.
- tmp="$TMPDIR"; if [ -z "$tmp" ]; then tmp=/tmp; fi
- stderr="$tmp/git-external-stderr.$$.tmp"
- test_external "$@" 4> "$stderr"
- [ -f "$stderr" ] || error "Internal error: $stderr disappeared."
- descr="no stderr: $1"
- shift
- say >&3 "expecting no stderr from previous command"
- if [ ! -s "$stderr" ]; then
- rm "$stderr"
- test_ok_ "$descr"
- else
- if [ "$verbose" = t ]; then
- output=`echo; echo Stderr is:; cat "$stderr"`
- else
- output=
- fi
- # rm first in case test_failure exits.
- rm "$stderr"
- test_failure_ "$descr" "$@" "$output"
- fi
-}
-
# This is not among top-level (test_expect_success | test_expect_failure)
# but is a prefix that can be used in the test script, like:
#
--
1.7.1.243.gda92d6.dirty
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH v2 0/3] Make the Git tests emit TAP format
2010-06-09 15:22 [PATCH v2 0/3] Make the Git tests emit TAP format Ævar Arnfjörð Bjarmason
` (2 preceding siblings ...)
2010-06-09 15:24 ` [PATCH v2 3/3] We use TAP so the Perl test can run without scaffolding Ævar Arnfjörð Bjarmason
@ 2010-06-14 21:49 ` Jakub Narebski
2010-06-14 22:10 ` Ævar Arnfjörð Bjarmason
3 siblings, 1 reply; 20+ messages in thread
From: Jakub Narebski @ 2010-06-14 21:49 UTC (permalink / raw
To: Ævar Arnfjörð Bjarmason; +Cc: git, Junio C Hamano
Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> All this series does is slightly adjust the raw text output of our
> tests so that it conforms. to the TAP standard, i.e. instead of this:
>
> $ ./t0005-signals.sh
> * ok 1: sigchain works
> * passed all 1 test(s)
>
> We get this:
>
> $ ./t0005-signals.sh
> ok 1 - sigchain works
> # passed all 1 test(s)
> 1..1
>
> Changing the output format like this gives us the ability to run the
> Git tests with any TAP tool (like prove(1)) at no extra cost. Every
> other existing way of running the tests continues to work, it's just
> easier for machines to read the output now.
This doesn't tell us if the result of running test suite with
'--verbose' and/or with '--debug' changes, and if changes how? Is it
compatibile with TAP format so that TAP parsers understand it?
It doesn't tell us if (as I assume) after change git test suite still
generates summary of tests in t/test-results/. Do those results
change, and if change then how?
It doesn't tell us if (as I assume) we still have the same color
output as we had before, and what we should do to have the same color
output e.g. from TAP tests in Perl.
+1
In my opinion it is a good change, because there are many tools that
understand TAP output format (and being able to output tests in Perl
via standard Test::Simple / Test::More is also nice). Those include
e.g. Smolder and prove(1).
It would be nice to have something like CPAN Testers
(http://cpantesters.org) but for git test suite. Junio would be able
to submit new version of git and check if it breaks on some uncommon
architecture...
P.S. I wonder who was original designer of git tests output format,
ans why it was chosen this way (and e.g. why not TAP)...
--
Jakub Narebski
Poland
ShadeHawk on #git
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v2 1/3] Make test-lib.sh emit valid TAP format
2010-06-09 15:22 ` [PATCH v2 1/3] Make test-lib.sh emit valid " Ævar Arnfjörð Bjarmason
@ 2010-06-14 22:01 ` Jakub Narebski
2010-06-14 22:29 ` Ævar Arnfjörð Bjarmason
0 siblings, 1 reply; 20+ messages in thread
From: Jakub Narebski @ 2010-06-14 22:01 UTC (permalink / raw
To: Ævar Arnfjörð Bjarmason; +Cc: git, Junio C Hamano
Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> TAP, the Test Anything Protocol, is a simple text-based interface
> between testing modules in a test harness. test-lib.sh's output was
> already very close to being valid TAP. This change brings it all the
> way there. Before:
>
> $ ./t0005-signals.sh
> * ok 1: sigchain works
> * passed all 1 test(s)
>
> And after:
>
> $ ./t0005-signals.sh
> ok 1 - sigchain works
> # passed all 1 test(s)
> 1..1
How failing test looks like before and after the change?
How test_expect_failure, i.e. known breakage (TODO test) looks like
(both in known broken, and in accidentally fixed version) before and
after the change?
How does '--verbose' and '--debug' output looks like before and after
the change?
--
Jakub Narebski
Poland
ShadeHawk on #git
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v2 0/3] Make the Git tests emit TAP format
2010-06-14 21:49 ` [PATCH v2 0/3] Make the Git tests emit TAP format Jakub Narebski
@ 2010-06-14 22:10 ` Ævar Arnfjörð Bjarmason
2010-06-14 23:16 ` Ævar Arnfjörð Bjarmason
2010-06-15 15:17 ` Ævar Arnfjörð Bjarmason
0 siblings, 2 replies; 20+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2010-06-14 22:10 UTC (permalink / raw
To: Jakub Narebski; +Cc: git, Junio C Hamano
On Mon, Jun 14, 2010 at 21:49, Jakub Narebski <jnareb@gmail.com> wrote:
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>> All this series does is slightly adjust the raw text output of our
>> tests so that it conforms. to the TAP standard, i.e. instead of this:
>>
>> $ ./t0005-signals.sh
>> * ok 1: sigchain works
>> * passed all 1 test(s)
>>
>> We get this:
>>
>> $ ./t0005-signals.sh
>> ok 1 - sigchain works
>> # passed all 1 test(s)
>> 1..1
>>
>> Changing the output format like this gives us the ability to run the
>> Git tests with any TAP tool (like prove(1)) at no extra cost. Every
>> other existing way of running the tests continues to work, it's just
>> easier for machines to read the output now.
I'll re-submit a version of the patch with a better commit message
which addresses all of the below.
> This doesn't tell us if the result of running test suite with
> '--verbose' and/or with '--debug' changes, and if changes how? Is it
> compatibile with TAP format so that TAP parsers understand it?
It just changes in that the lines that previously said e.g. "* ok 1:
sigchain works" now say "ok 1 - sigchain works".
TAP parsers still understand it, since ignoring unknown garbage is
part of the TAP standard.
> It doesn't tell us if (as I assume) after change git test suite still
> generates summary of tests in t/test-results/. Do those results
> change, and if change then how?
The semantics of t/test-results don't change at all since that's
aggregated internally by test-lib.sh guts that aren't changed in this
series.
> It doesn't tell us if (as I assume) we still have the same color
> output as we had before, and what we should do to have the same color
> output e.g. from TAP tests in Perl.
The color output is still there if you run the tests under `make test'
or manually.
Under TAP consumers like prove(1) the tests won't output color since
the TAP Harness will set a dumb terminal.
That's a good thing, since we shouldn't emit terminal escape codes
with TAP. The output could be used for e.g. a HTML page instead of a
terminal.
> +1
>
> In my opinion it is a good change, because there are many tools that
> understand TAP output format (and being able to output tests in Perl
> via standard Test::Simple / Test::More is also nice). Those include
> e.g. Smolder and prove(1).
>
> It would be nice to have something like CPAN Testers
> (http://cpantesters.org) but for git test suite. Junio would be able
> to submit new version of git and check if it breaks on some uncommon
> architecture...
There's been a lot of good changes to the CPAN Testers infrastructure
lately. We could probably just set up a copy of their software.
Then anyone could run a smoker on some odd OS/architechture and send
in test results.
> P.S. I wonder who was original designer of git tests output format,
> ans why it was chosen this way (and e.g. why not TAP)...
It was originally discussed here:
http://thread.gmane.org/gmane.comp.version-control.git/2901/focus=3076
It seems that the people involved weren't aware of TAP at the time, at
least it wasn't brought up.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v2 1/3] Make test-lib.sh emit valid TAP format
2010-06-14 22:01 ` Jakub Narebski
@ 2010-06-14 22:29 ` Ævar Arnfjörð Bjarmason
0 siblings, 0 replies; 20+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2010-06-14 22:29 UTC (permalink / raw
To: Jakub Narebski; +Cc: git, Junio C Hamano
On Mon, Jun 14, 2010 at 22:01, Jakub Narebski <jnareb@gmail.com> wrote:
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>> TAP, the Test Anything Protocol, is a simple text-based interface
>> between testing modules in a test harness. test-lib.sh's output was
>> already very close to being valid TAP. This change brings it all the
>> way there. Before:
>>
>> $ ./t0005-signals.sh
>> * ok 1: sigchain works
>> * passed all 1 test(s)
>>
>> And after:
>>
>> $ ./t0005-signals.sh
>> ok 1 - sigchain works
>> # passed all 1 test(s)
>> 1..1
>
> How failing test looks like before and after the change?
>
> How test_expect_failure, i.e. known breakage (TODO test) looks like
> (both in known broken, and in accidentally fixed version) before and
> after the change?
>
> How does '--verbose' and '--debug' output looks like before and after
> the change?
I'll submit a version with an updated commit message. The gist of it:
* It's now ok/not ok instead of ok/failed
* --debug and --verbose work just like before
* Other things look like before, but may be changed to make TAP like
it (e.g. prefixing comments with #)
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v2 0/3] Make the Git tests emit TAP format
2010-06-14 22:10 ` Ævar Arnfjörð Bjarmason
@ 2010-06-14 23:16 ` Ævar Arnfjörð Bjarmason
2010-06-15 3:08 ` Junio C Hamano
2010-06-15 15:17 ` Ævar Arnfjörð Bjarmason
1 sibling, 1 reply; 20+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2010-06-14 23:16 UTC (permalink / raw
To: Jakub Narebski; +Cc: git, Junio C Hamano
On Mon, Jun 14, 2010 at 22:10, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> On Mon, Jun 14, 2010 at 21:49, Jakub Narebski <jnareb@gmail.com> wrote:
> I'll re-submit a version of the patch with a better commit message
> which addresses all of the below.
Since the series just made it into pu is that still wanted or needed?
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v2 0/3] Make the Git tests emit TAP format
2010-06-14 23:16 ` Ævar Arnfjörð Bjarmason
@ 2010-06-15 3:08 ` Junio C Hamano
2010-06-15 3:10 ` Ævar Arnfjörð Bjarmason
0 siblings, 1 reply; 20+ messages in thread
From: Junio C Hamano @ 2010-06-15 3:08 UTC (permalink / raw
To: Ævar Arnfjörð Bjarmason; +Cc: Jakub Narebski, git
Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> On Mon, Jun 14, 2010 at 22:10, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>> On Mon, Jun 14, 2010 at 21:49, Jakub Narebski <jnareb@gmail.com> wrote:
>> I'll re-submit a version of the patch with a better commit message
>> which addresses all of the below.
>
> Since the series just made it into pu is that still wanted or needed?
It very much is. There isn't that much difference between being in 'pu'
and being in the mailing list archive. Queuing in 'pu' is just to help me
not to forget about the series, nothing more. Depending on how further
discussions go, the series can be replaced with an improvement or even can
be dropped as a whole as unusable.
In this case, the explanation in the commit messages and/or documentation
are found to be lacking, and you already know what the improved version
should look like, so I think it makes sense to replace the series with a
new version (v3?)
Thanks.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v2 0/3] Make the Git tests emit TAP format
2010-06-15 3:08 ` Junio C Hamano
@ 2010-06-15 3:10 ` Ævar Arnfjörð Bjarmason
0 siblings, 0 replies; 20+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2010-06-15 3:10 UTC (permalink / raw
To: Junio C Hamano; +Cc: Jakub Narebski, git
On Tue, Jun 15, 2010 at 03:08, Junio C Hamano <gitster@pobox.com> wrote:
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>> On Mon, Jun 14, 2010 at 22:10, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>>> On Mon, Jun 14, 2010 at 21:49, Jakub Narebski <jnareb@gmail.com> wrote:
>>> I'll re-submit a version of the patch with a better commit message
>>> which addresses all of the below.
>>
>> Since the series just made it into pu is that still wanted or needed?
>
> It very much is. There isn't that much difference between being in 'pu'
> and being in the mailing list archive. Queuing in 'pu' is just to help me
> not to forget about the series, nothing more. Depending on how further
> discussions go, the series can be replaced with an improvement or even can
> be dropped as a whole as unusable.
>
> In this case, the explanation in the commit messages and/or documentation
> are found to be lacking, and you already know what the improved version
> should look like, so I think it makes sense to replace the series with a
> new version (v3?)
Thanks for the explanation. I'll submit an better version of it soon.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v2 0/3] Make the Git tests emit TAP format
2010-06-14 22:10 ` Ævar Arnfjörð Bjarmason
2010-06-14 23:16 ` Ævar Arnfjörð Bjarmason
@ 2010-06-15 15:17 ` Ævar Arnfjörð Bjarmason
2010-06-15 16:42 ` Andreas Ericsson
` (6 more replies)
1 sibling, 7 replies; 20+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2010-06-15 15:17 UTC (permalink / raw
To: Jakub Narebski; +Cc: git, Junio C Hamano
On Mon, Jun 14, 2010 at 22:10, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> On Mon, Jun 14, 2010 at 21:49, Jakub Narebski <jnareb@gmail.com> wrote:
>> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>>
>>> All this series does is slightly adjust the raw text output of our
>>> tests so that it conforms. to the TAP standard, i.e. instead of this:
>>>
>>> $ ./t0005-signals.sh
>>> * ok 1: sigchain works
>>> * passed all 1 test(s)
>>>
>>> We get this:
>>>
>>> $ ./t0005-signals.sh
>>> ok 1 - sigchain works
>>> # passed all 1 test(s)
>>> 1..1
>>>
>>> Changing the output format like this gives us the ability to run the
>>> Git tests with any TAP tool (like prove(1)) at no extra cost. Every
>>> other existing way of running the tests continues to work, it's just
>>> easier for machines to read the output now.
>
> I'll re-submit a version of the patch with a better commit message
> which addresses all of the below.
>
>> This doesn't tell us if the result of running test suite with
>> '--verbose' and/or with '--debug' changes, and if changes how? Is it
>> compatibile with TAP format so that TAP parsers understand it?
>
> It just changes in that the lines that previously said e.g. "* ok 1:
> sigchain works" now say "ok 1 - sigchain works".
>
> TAP parsers still understand it, since ignoring unknown garbage is
> part of the TAP standard.
Actually it doesn't work for all the tests. The following crops up on
prove -j 10 ./t[0-9]*.sh :: --verbose:
./t1007-hash-object.sh (Wstat: 0
Tests: 19 Failed: 0)
Parse errors: Tests out of sequence. Found (12) but expected (11)
Tests out of sequence. Found (13) but expected (12)
Tests out of sequence. Found (14) but expected (13)
Tests out of sequence. Found (15) but expected (14)
Tests out of sequence. Found (16) but expected (15)
Displayed the first 5 of 10 TAP syntax errors.
Re-run prove with the -p option to see them all.
./t3411-rebase-preserve-around-merges.sh (Wstat: 0
Tests: 1 Failed: 0)
Parse errors: Bad plan. You planned 3 tests but ran 1.
./t3410-rebase-preserve-dropped-merges.sh (Wstat: 0
Tests: 1 Failed: 0)
Parse errors: Bad plan. You planned 3 tests but ran 1.
./t3413-rebase-hook.sh (Wstat: 0
Tests: 10 Failed: 0)
Parse errors: Tests out of sequence. Found (4) but expected (3)
Tests out of sequence. Found (5) but expected (4)
Tests out of sequence. Found (6) but expected (5)
Tests out of sequence. Found (7) but expected (6)
Tests out of sequence. Found (11) but expected (7)
Displayed the first 5 of 9 TAP syntax errors.
Re-run prove with the -p option to see them all.
./t3409-rebase-preserve-merges.sh (Wstat: 0
Tests: 1 Failed: 0)
Parse errors: Bad plan. You planned 3 tests but ran 1.
./t3414-rebase-preserve-onto.sh (Wstat: 0
Tests: 1 Failed: 0)
Parse errors: Bad plan. You planned 4 tests but ran 1.
./t3415-rebase-autosquash.sh (Wstat: 0
Tests: 1 Failed: 0)
Parse errors: Bad plan. You planned 4 tests but ran 1.
./t3416-rebase-onto-threedots.sh (Wstat: 0
Tests: 5 Failed: 0)
Parse errors: Tests out of sequence. Found (7) but expected (5)
Bad plan. You planned 7 tests but ran 5.
./t3412-rebase-root.sh (Wstat: 0
Tests: 24 Failed: 0)
./t3404-rebase-interactive.sh (Wstat: 0
Tests: 23 Failed: 0)
Parse errors: Tests out of sequence. Found (6) but expected (2)
Tests out of sequence. Found (8) but expected (3)
Tests out of sequence. Found (9) but expected (4)
Tests out of sequence. Found (10) but expected (5)
Tests out of sequence. Found (11) but expected (6)
Displayed the first 5 of 23 TAP syntax errors.
Re-run prove with the -p option to see them all.
./t5407-post-rewrite-hook.sh (Wstat: 0
Tests: 9 Failed: 0)
Parse errors: Tests out of sequence. Found (10) but expected (8)
Tests out of sequence. Found (11) but expected (9)
Bad plan. You planned 12 tests but ran 9.
./t7402-submodule-rebase.sh (Wstat: 0
Tests: 4 Failed: 0)
Parse errors: Tests out of sequence. Found (4) but expected (3)
Tests out of sequence. Found (5) but expected (4)
Bad plan. You planned 5 tests but ran 4.
./t7003-filter-branch.sh (Wstat: 0
Tests: 30 Failed: 0)
Parse errors: Tests out of sequence. Found (8) but expected (7)
Tests out of sequence. Found (9) but expected (8)
Tests out of sequence. Found (10) but expected (9)
Tests out of sequence. Found (11) but expected (10)
Tests out of sequence. Found (12) but expected (11)
Displayed the first 5 of 25 TAP syntax errors.
Re-run prove with the -p option to see them all.
./t9001-send-email.sh (Wstat: 0
Tests: 63 Failed: 0)
Parse errors: Tests out of sequence. Found (45) but expected (44)
Tests out of sequence. Found (46) but expected (45)
Tests out of sequence. Found (47) but expected (46)
Tests out of sequence. Found (48) but expected (47)
Tests out of sequence. Found (49) but expected (48)
The problem is that these tests all have code outputthat goes
something like this:
ok 2 - rebase
expecting success:
git checkout test &&
git reset --hard side &&
EDITOR=true git rebase -i master &&
test "z$(cat git)" = zworld
HEAD is now at c847452 side
Rebasing (1/1)^Mok 3 - rebase -i
expecting success:
Caused by tests like these that don't end in a newline:
test_expect_success 'rebase -i' '
git checkout test &&
git reset --hard side &&
EDITOR=true git rebase -i master &&
test "z$(cat git)" = zworld'
'
This was of course also broken before TAP, it just revealed the issue:
Applying: side
* ok 2: rebase
* expecting success:
git checkout test &&
git reset --hard side &&
EDITOR=true git rebase -i master &&
test "z$(cat git)" = zworld
HEAD is now at c847452 side
Rebasing (1/1)^M* ok 3: rebase -i
I propose to fix it like this:
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 37987d7..86a46bf 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -369,6 +369,9 @@ test_run_ () {
eval >&3 2>&4 "$1"
eval_ret=$?
eval >&3 2>&4 "$test_cleanup"
+ if test "$verbose" = "t" && test -n "$HARNESS_ACTIVE"; then
+ echo ""
+ fi
return 0
}
That makes all tests a bit more verbose when run under --verbose and a
TAP::Harness, but fixes the issue everywhere and in any future code by
ensuring that there's a newline before /^(?:not )?ok/ lines.
The alternative would be to change all the tests in question so that
they end with && echo, or to munge the `eval >&3 2>&4 "$1"' part above
(maybe with tempfiles?) so that a newline is only added to the end if
there isn't one already.
Aside from that, making sure that nothing in the test suite itself
says "ok" on an otherwise empty line is also required:
t/t1020-subdirectory.sh | 12 ++++++------
t/t2102-update-index-symlinks.sh | 2 +-
t/t3700-add.sh | 12 ++++++------
3 files changed, 13 insertions(+), 13 deletions(-)
diff --git a/t/t1020-subdirectory.sh b/t/t1020-subdirectory.sh
index 210e594..5687499 100755
--- a/t/t1020-subdirectory.sh
+++ b/t/t1020-subdirectory.sh
@@ -24,18 +24,18 @@ test_expect_success 'update-index and ls-files' '
cd "$HERE" &&
git update-index --add one &&
case "`git ls-files`" in
- one) echo ok one ;;
+ one) echo pass one ;;
*) echo bad one; exit 1 ;;
esac &&
cd dir &&
git update-index --add two &&
case "`git ls-files`" in
- two) echo ok two ;;
+ two) echo pass two ;;
*) echo bad two; exit 1 ;;
esac &&
cd .. &&
case "`git ls-files`" in
- dir/two"$LF"one) echo ok both ;;
+ dir/two"$LF"one) echo pass both ;;
*) echo bad; exit 1 ;;
esac
'
@@ -58,17 +58,17 @@ test_expect_success 'diff-files' '
echo a >>one &&
echo d >>dir/two &&
case "`git diff-files --name-only`" in
- dir/two"$LF"one) echo ok top ;;
+ dir/two"$LF"one) echo pass top ;;
*) echo bad top; exit 1 ;;
esac &&
# diff should not omit leading paths
cd dir &&
case "`git diff-files --name-only`" in
- dir/two"$LF"one) echo ok subdir ;;
+ dir/two"$LF"one) echo pass subdir ;;
*) echo bad subdir; exit 1 ;;
esac &&
case "`git diff-files --name-only .`" in
- dir/two) echo ok subdir limited ;;
+ dir/two) echo pass subdir limited ;;
*) echo bad subdir limited; exit 1 ;;
esac
'
diff --git a/t/t2102-update-index-symlinks.sh
b/t/t2102-update-index-symlinks.sh
index 1ed44ee..4d0d0a3 100755
--- a/t/t2102-update-index-symlinks.sh
+++ b/t/t2102-update-index-symlinks.sh
@@ -24,7 +24,7 @@ git update-index symlink'
test_expect_success \
'the index entry must still be a symbolic link' '
case "`git ls-files --stage --cached symlink`" in
-120000" "*symlink) echo ok;;
+120000" "*symlink) echo pass;;
*) echo fail; git ls-files --stage --cached symlink; (exit 1);;
esac'
diff --git a/t/t3700-add.sh b/t/t3700-add.sh
index 525c9a8..6f031af 100755
--- a/t/t3700-add.sh
+++ b/t/t3700-add.sh
@@ -26,7 +26,7 @@ test_expect_success \
chmod 755 xfoo1 &&
git add xfoo1 &&
case "`git ls-files --stage xfoo1`" in
- 100644" "*xfoo1) echo ok;;
+ 100644" "*xfoo1) echo pass;;
*) echo fail; git ls-files --stage xfoo1; (exit 1);;
esac'
@@ -35,7 +35,7 @@ test_expect_success SYMLINKS 'git add:
filemode=0 should not get confused by sym
ln -s foo xfoo1 &&
git add xfoo1 &&
case "`git ls-files --stage xfoo1`" in
- 120000" "*xfoo1) echo ok;;
+ 120000" "*xfoo1) echo pass;;
*) echo fail; git ls-files --stage xfoo1; (exit 1);;
esac
'
@@ -47,7 +47,7 @@ test_expect_success \
chmod 755 xfoo2 &&
git update-index --add xfoo2 &&
case "`git ls-files --stage xfoo2`" in
- 100644" "*xfoo2) echo ok;;
+ 100644" "*xfoo2) echo pass;;
*) echo fail; git ls-files --stage xfoo2; (exit 1);;
esac'
@@ -56,7 +56,7 @@ test_expect_success SYMLINKS 'git add:
filemode=0 should not get confused by sym
ln -s foo xfoo2 &&
git update-index --add xfoo2 &&
case "`git ls-files --stage xfoo2`" in
- 120000" "*xfoo2) echo ok;;
+ 120000" "*xfoo2) echo pass;;
*) echo fail; git ls-files --stage xfoo2; (exit 1);;
esac
'
@@ -67,7 +67,7 @@ test_expect_success SYMLINKS \
ln -s xfoo2 xfoo3 &&
git update-index --add xfoo3 &&
case "`git ls-files --stage xfoo3`" in
- 120000" "*xfoo3) echo ok;;
+ 120000" "*xfoo3) echo pass;;
*) echo fail; git ls-files --stage xfoo3; (exit 1);;
esac'
@@ -172,7 +172,7 @@ test_expect_success 'git add --refresh' '
test -z "`git diff-index HEAD -- foo`" &&
git read-tree HEAD &&
case "`git diff-index HEAD -- foo`" in
- :100644" "*"M foo") echo ok;;
+ :100644" "*"M foo") echo pass;;
*) echo fail; (exit 1);;
esac &&
git add --refresh -- foo &&
Another way to solve all of these would be to apply s/^/# / to all the
--verbose output. This is what the Perl tools do:
$ perl -MTest::More=tests,2 -E 'pass "a test";
diag("hello\nthere\noutput\n"); pass "another test"'
1..2
ok 1 - a test
# hello
# there
# output
ok 2 - another test
Then the verbose output would only be printed under prove(1) if its
--verbose option was supplied (if lines aren't comments they're
printed as-is). That'd also require something like the eval + tempfile
hack suggested above.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v2 0/3] Make the Git tests emit TAP format
2010-06-15 15:17 ` Ævar Arnfjörð Bjarmason
@ 2010-06-15 16:42 ` Andreas Ericsson
2010-06-15 16:49 ` Ævar Arnfjörð Bjarmason
2010-06-15 22:32 ` [PATCH v3 0/5] TAP support for Git Ævar Arnfjörð Bjarmason
` (5 subsequent siblings)
6 siblings, 1 reply; 20+ messages in thread
From: Andreas Ericsson @ 2010-06-15 16:42 UTC (permalink / raw
To: Ævar Arnfjörð Bjarmason
Cc: Jakub Narebski, git, Junio C Hamano
On 06/15/2010 05:17 PM, Ævar Arnfjörð Bjarmason wrote:
> On Mon, Jun 14, 2010 at 22:10, Ævar Arnfjörð Bjarmason<avarab@gmail.com> wrote:
>> On Mon, Jun 14, 2010 at 21:49, Jakub Narebski<jnareb@gmail.com> wrote:
>>> Ævar Arnfjörð Bjarmason<avarab@gmail.com> writes:
>>>
>>>> All this series does is slightly adjust the raw text output of our
>>>> tests so that it conforms. to the TAP standard, i.e. instead of this:
>>>>
>>>> $ ./t0005-signals.sh
>>>> * ok 1: sigchain works
>>>> * passed all 1 test(s)
>>>>
>>>> We get this:
>>>>
>>>> $ ./t0005-signals.sh
>>>> ok 1 - sigchain works
>>>> # passed all 1 test(s)
>>>> 1..1
>>>>
>>>> Changing the output format like this gives us the ability to run the
>>>> Git tests with any TAP tool (like prove(1)) at no extra cost. Every
>>>> other existing way of running the tests continues to work, it's just
>>>> easier for machines to read the output now.
>>
>> I'll re-submit a version of the patch with a better commit message
>> which addresses all of the below.
>>
>>> This doesn't tell us if the result of running test suite with
>>> '--verbose' and/or with '--debug' changes, and if changes how? Is it
>>> compatibile with TAP format so that TAP parsers understand it?
>>
>> It just changes in that the lines that previously said e.g. "* ok 1:
>> sigchain works" now say "ok 1 - sigchain works".
>>
>> TAP parsers still understand it, since ignoring unknown garbage is
>> part of the TAP standard.
>
> Actually it doesn't work for all the tests. The following crops up on
> prove -j 10 ./t[0-9]*.sh :: --verbose:
>
> ./t1007-hash-object.sh (Wstat: 0
> Tests: 19 Failed: 0)
> Parse errors: Tests out of sequence. Found (12) but expected (11)
> Tests out of sequence. Found (13) but expected (12)
> Tests out of sequence. Found (14) but expected (13)
> Tests out of sequence. Found (15) but expected (14)
> Tests out of sequence. Found (16) but expected (15)
Why are you using a plan at all? I absolutely detest using planned tap
series, and last I heard was that the majority of TAP-fanatics actually
agreed that using a plan to run tests was an extraordinarily lousy idea,
since it makes it harder to add tests properly.
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v2 0/3] Make the Git tests emit TAP format
2010-06-15 16:42 ` Andreas Ericsson
@ 2010-06-15 16:49 ` Ævar Arnfjörð Bjarmason
0 siblings, 0 replies; 20+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2010-06-15 16:49 UTC (permalink / raw
To: Andreas Ericsson; +Cc: Jakub Narebski, git, Junio C Hamano
On Tue, Jun 15, 2010 at 16:42, Andreas Ericsson <ae@op5.se> wrote:
> Why are you using a plan at all? I absolutely detest using planned tap
> series, and last I heard was that the majority of TAP-fanatics actually
> agreed that using a plan to run tests was an extraordinarily lousy idea,
> since it makes it harder to add tests properly.
I'm not using a plan before the tests start (although I've considered
adding support for it). But these errors have nothing to do with TAP
plans, they happen because the tests are out of order.
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v3 0/5] TAP support for Git
2010-06-15 15:17 ` Ævar Arnfjörð Bjarmason
2010-06-15 16:42 ` Andreas Ericsson
@ 2010-06-15 22:32 ` Ævar Arnfjörð Bjarmason
2010-06-15 22:32 ` [PATCH v3 1/5] Make test-lib.sh emit valid TAP format Ævar Arnfjörð Bjarmason
` (4 subsequent siblings)
6 siblings, 0 replies; 20+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2010-06-15 22:32 UTC (permalink / raw
To: git; +Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason
Here's v3 of the TAP support for Git series, which replaces the one
already in pu.
Changes since v2:
* The tests now run cleanly under:
prove -v ./t0005-signals.sh :: --verbose --debug
This required changing 'ok' to 'pass' in a few tests, and
emitting a newline before ok/not ok under HARNESS_ACTIVE=1 as I
suggested on-list.
* The Perl tests now properly report failure if they fail via their
exit code, previously a failure from them wouldn't be propagated.
* Extra docs in t/README hinting at prove's --state option, in
particular:
# Repeat until no more failures
$ prove -j 15 --state=failed,save ./t[0-9]*.sh
* diff --check clean (wasn't before because I didn't want to change
existing lines, but they deserve to to be changed)
Changes since v2 in diff --stat format:
t/README | 10 ++++++++--
t/t1020-subdirectory.sh | 12 ++++++------
t/t2102-update-index-symlinks.sh | 2 +-
t/t3700-add.sh | 12 ++++++------
t/t9700-perl-git.sh | 7 +++----
t/t9700/test.pl | 2 ++
t/test-lib.sh | 3 +++
7 files changed, 29 insertions(+), 19 deletions(-)
Ævar Arnfjörð Bjarmason (5):
Make test-lib.sh emit valid TAP format
Skip tests in a way that makes sense under TAP
We use TAP so the Perl test can run without scaffolding
TAP: Say "pass" rather than "ok" on an empty line
TAP: Make sure there's a newline before "ok" under harness
t/README | 55 +++++++++++++-----
t/lib-git-svn.sh | 4 +-
t/lib-httpd.sh | 3 +-
t/t1020-subdirectory.sh | 12 ++--
t/t1304-default-acl.sh | 9 ++-
t/t1509-root-worktree.sh | 6 +-
t/t2007-checkout-symlink.sh | 2 +-
t/t2102-update-index-symlinks.sh | 2 +-
t/t3300-funny-names.sh | 2 +-
t/t3302-notes-index-expensive.sh | 2 +-
t/t3600-rm.sh | 2 +-
t/t3700-add.sh | 12 ++--
t/t3701-add-interactive.sh | 4 +-
t/t3902-quoted.sh | 2 +-
t/t4004-diff-rename-symlink.sh | 2 +-
t/t4011-diff-symlink.sh | 2 +-
t/t4016-diff-quote.sh | 2 +-
t/t4023-diff-rename-typechange.sh | 2 +-
t/t4114-apply-typechange.sh | 2 +-
t/t4115-apply-symlink.sh | 2 +-
t/t4122-apply-symlink-inside.sh | 2 +-
t/t5302-pack-index.sh | 2 +-
t/t5503-tagfollow.sh | 2 +-
t/t5522-pull-symlink.sh | 2 +-
t/t5540-http-push.sh | 2 +-
t/t5541-http-push.sh | 2 +-
t/t5550-http-fetch.sh | 2 +-
t/t5551-http-fetch.sh | 2 +-
t/t5561-http-backend.sh | 2 +-
t/t5705-clone-2gb.sh | 2 +-
t/t6035-merge-dir-to-symlink.sh | 2 +-
t/t7004-tag.sh | 2 +-
t/t7006-pager.sh | 2 +-
t/t7800-difftool.sh | 2 +-
t/t9001-send-email.sh | 4 +-
t/t9100-git-svn-basic.sh | 2 +-
t/t9119-git-svn-info.sh | 2 +-
t/t9129-git-svn-i18n-commitencoding.sh | 2 +-
t/t9143-git-svn-gc.sh | 2 +-
t/t9200-git-cvsexportcommit.sh | 4 +-
t/t9400-git-cvsserver-server.sh | 6 +-
t/t9401-git-cvsserver-crlf.sh | 6 +-
t/t9600-cvsimport.sh | 2 +-
t/t9700-perl-git.sh | 12 ++--
t/t9700/test.pl | 11 ++++
t/test-lib.sh | 95 +++++++-------------------------
46 files changed, 143 insertions(+), 162 deletions(-)
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v3 1/5] Make test-lib.sh emit valid TAP format
2010-06-15 15:17 ` Ævar Arnfjörð Bjarmason
2010-06-15 16:42 ` Andreas Ericsson
2010-06-15 22:32 ` [PATCH v3 0/5] TAP support for Git Ævar Arnfjörð Bjarmason
@ 2010-06-15 22:32 ` Ævar Arnfjörð Bjarmason
2010-06-15 22:32 ` [PATCH v3 2/5] Skip tests in a way that makes sense under TAP Ævar Arnfjörð Bjarmason
` (3 subsequent siblings)
6 siblings, 0 replies; 20+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2010-06-15 22:32 UTC (permalink / raw
To: git; +Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason
TAP, the Test Anything Protocol, is a simple text-based interface
between testing modules in a test harness. test-lib.sh's output was
already very close to being valid TAP. This change brings it all the
way there. Before:
$ ./t0005-signals.sh
* ok 1: sigchain works
* passed all 1 test(s)
And after:
$ ./t0005-signals.sh
ok 1 - sigchain works
# passed all 1 test(s)
1..1
The advantage of using TAP is that any program that reads the format
(a "test harness") can run the tests. The most popular of these is the
prove(1) utility that comes with Perl. It can run tests in parallel,
display colored output, format the output to console, file, HTML etc.,
and much more. An example:
$ prove ./t0005-signals.sh
./t0005-signals.sh .. ok
All tests successful.
Files=1, Tests=1, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.01 cusr 0.02 csys = 0.06 CPU)
Result: PASS
prove(1) gives you human readable output without being too
verbose. Running the test suite in parallel with `make test -j15`
produces a flood of text. Running them with `prove -j 15 ./t[0-9]*.sh`
makes it easy to follow what's going on.
All this patch does is re-arrange the output a bit so that it conforms
with the TAP spec, everything that the test suite did before continues
to work. That includes aggregating results in t/test-results/, the
--verbose, --debug and other options for tests, and the test color
output.
TAP harnesses ignore everything that they don't know about, so running
the tests with --verbose works:
$ prove ./t0005-signals.sh :: --verbose --debug
./t0005-signals.sh .. Terminated
./t0005-signals.sh .. ok
All tests successful.
Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.01 sys + 0.01 cusr 0.01 csys = 0.05 CPU)
Result: PASS
Just supply the -v option to prove itself to get all the verbose
output that it suppresses:
$ prove -v ./t0005-signals.sh :: --verbose --debug
./t0005-signals.sh ..
Initialized empty Git repository in /home/avar/g/git/t/trash directory.t0005-signals/.git/
expecting success:
test-sigchain >actual
case "$?" in
143) true ;; # POSIX w/ SIGTERM=15
3) true ;; # Windows
*) false ;;
esac &&
test_cmp expect actual
Terminated
ok 1 - sigchain works
# passed all 1 test(s)
1..1
ok
All tests successful.
Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.01 cusr 0.01 csys = 0.04 CPU)
Result: PASS
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
t/README | 55 +++++++++++++++++++++++++++++++++++++++----------------
t/test-lib.sh | 30 ++++++++++++++++++------------
2 files changed, 57 insertions(+), 28 deletions(-)
diff --git a/t/README b/t/README
index 0e4e8d8..aa4ed28 100644
--- a/t/README
+++ b/t/README
@@ -18,25 +18,48 @@ The easiest way to run tests is to say "make". This runs all
the tests.
*** t0000-basic.sh ***
- * ok 1: .git/objects should be empty after git-init in an empty repo.
- * ok 2: .git/objects should have 256 subdirectories.
- * ok 3: git-update-index without --add should fail adding.
+ ok 1 - .git/objects should be empty after git init in an empty repo.
+ ok 2 - .git/objects should have 3 subdirectories.
+ ok 3 - success is reported like this
...
- * ok 23: no diff after checkout and git-update-index --refresh.
- * passed all 23 test(s)
- *** t0100-environment-names.sh ***
- * ok 1: using old names should issue warnings.
- * ok 2: using old names but having new names should not issue warnings.
- ...
-
-Or you can run each test individually from command line, like
-this:
-
- $ sh ./t3001-ls-files-killed.sh
- * ok 1: git-update-index --add to add various paths.
- * ok 2: git-ls-files -k to show killed files.
- * ok 3: validate git-ls-files -k output.
- * passed all 3 test(s)
+ ok 43 - very long name in the index handled sanely
+ # fixed 1 known breakage(s)
+ # still have 1 known breakage(s)
+ # passed all remaining 42 test(s)
+ 1..43
+ *** t0001-init.sh ***
+ ok 1 - plain
+ ok 2 - plain with GIT_WORK_TREE
+ ok 3 - plain bare
+
+Since the tests all output TAP (see http://testanything.org) they can
+be run with any TAP harness. Here's an example of paralell testing
+powered by a recent version of prove(1):
+
+ $ prove --timer --jobs 15 ./t[0-9]*.sh
+ [19:17:33] ./t0005-signals.sh ................................... ok 36 ms
+ [19:17:33] ./t0022-crlf-rename.sh ............................... ok 69 ms
+ [19:17:33] ./t0024-crlf-archive.sh .............................. ok 154 ms
+ [19:17:33] ./t0004-unwritable.sh ................................ ok 289 ms
+ [19:17:33] ./t0002-gitfile.sh ................................... ok 480 ms
+ ===( 102;0 25/? 6/? 5/? 16/? 1/? 4/? 2/? 1/? 3/? 1... )===
+
+prove and other harnesses come with a lot of useful options. The
+--state option in particular is very useful:
+
+ # Repeat until no more failures
+ $ prove -j 15 --state=failed,save ./t[0-9]*.sh
+
+You can also run each test individually from command line, like this:
+
+ $ sh ./t3010-ls-files-killed-modified.sh
+ ok 1 - git update-index --add to add various paths.
+ ok 2 - git ls-files -k to show killed files.
+ ok 3 - validate git ls-files -k output.
+ ok 4 - git ls-files -m to show modified files.
+ ok 5 - validate git ls-files -m output.
+ # passed all 5 test(s)
+ 1..5
You can pass --verbose (or -v), --debug (or -d), and --immediate
(or -i) command line argument to the test, or by setting GIT_TEST_OPTS
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 367f053..e97645c 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -160,7 +160,7 @@ if test -n "$color"; then
*) test -n "$quiet" && return;;
esac
shift
- printf "* %s" "$*"
+ printf "%s" "$*"
tput sgr0
echo
)
@@ -169,7 +169,7 @@ else
say_color() {
test -z "$1" && test -n "$quiet" && return
shift
- echo "* $*"
+ echo "$*"
}
fi
@@ -339,25 +339,25 @@ test_have_prereq () {
test_ok_ () {
test_success=$(($test_success + 1))
- say_color "" " ok $test_count: $@"
+ say_color "" "ok $test_count - $@"
}
test_failure_ () {
test_failure=$(($test_failure + 1))
- say_color error "FAIL $test_count: $1"
+ say_color error "not ok - $test_count $1"
shift
- echo "$@" | sed -e 's/^/ /'
+ echo "$@" | sed -e 's/^/# /'
test "$immediate" = "" || { GIT_EXIT_OK=t; exit 1; }
}
test_known_broken_ok_ () {
test_fixed=$(($test_fixed+1))
- say_color "" " FIXED $test_count: $@"
+ say_color "" "ok $test_count - $@ # TODO known breakage"
}
test_known_broken_failure_ () {
test_broken=$(($test_broken+1))
- say_color skip " still broken $test_count: $@"
+ say_color skip "not ok $test_count - $@ # TODO known breakage"
}
test_debug () {
@@ -390,7 +390,7 @@ test_skip () {
case "$to_skip" in
t)
say_color skip >&3 "skipping test: $@"
- say_color skip "skip $test_count: $1"
+ say_color skip "ok $test_count: # skip $1"
: true
;;
*)
@@ -620,18 +620,22 @@ test_done () {
if test "$test_fixed" != 0
then
- say_color pass "fixed $test_fixed known breakage(s)"
+ say_color pass "# fixed $test_fixed known breakage(s)"
fi
if test "$test_broken" != 0
then
- say_color error "still have $test_broken known breakage(s)"
+ say_color error "# still have $test_broken known breakage(s)"
msg="remaining $(($test_count-$test_broken)) test(s)"
else
msg="$test_count test(s)"
fi
case "$test_failure" in
0)
- say_color pass "passed all $msg"
+ # Maybe print SKIP message
+ [ -z "$skip_all" ] || skip_all=" # SKIP $skip_all"
+
+ say_color pass "# passed all $msg"
+ say "1..$test_count$skip_all"
test -d "$remove_trash" &&
cd "$(dirname "$remove_trash")" &&
@@ -640,7 +644,9 @@ test_done () {
exit 0 ;;
*)
- say_color error "failed $test_failure among $msg"
+ say_color error "# failed $test_failure among $msg"
+ say "1..$test_count"
+
exit 1 ;;
esac
--
1.7.1.251.g92a7
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v3 2/5] Skip tests in a way that makes sense under TAP
2010-06-15 15:17 ` Ævar Arnfjörð Bjarmason
` (2 preceding siblings ...)
2010-06-15 22:32 ` [PATCH v3 1/5] Make test-lib.sh emit valid TAP format Ævar Arnfjörð Bjarmason
@ 2010-06-15 22:32 ` Ævar Arnfjörð Bjarmason
2010-06-15 22:32 ` [PATCH v3 3/5] We use TAP so the Perl test can run without scaffolding Ævar Arnfjörð Bjarmason
` (2 subsequent siblings)
6 siblings, 0 replies; 20+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2010-06-15 22:32 UTC (permalink / raw
To: git; +Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason
SKIP messages are now part of the TAP plan. A TAP harness now knows
why a particular test was skipped and can report that information.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
t/lib-git-svn.sh | 4 ++--
t/lib-httpd.sh | 3 +--
t/t1304-default-acl.sh | 9 ++++++---
t/t1509-root-worktree.sh | 6 +++---
t/t2007-checkout-symlink.sh | 2 +-
t/t3300-funny-names.sh | 2 +-
t/t3302-notes-index-expensive.sh | 2 +-
t/t3600-rm.sh | 2 +-
t/t3701-add-interactive.sh | 4 ++--
t/t3902-quoted.sh | 2 +-
t/t4004-diff-rename-symlink.sh | 2 +-
t/t4011-diff-symlink.sh | 2 +-
t/t4016-diff-quote.sh | 2 +-
t/t4023-diff-rename-typechange.sh | 2 +-
t/t4114-apply-typechange.sh | 2 +-
t/t4115-apply-symlink.sh | 2 +-
t/t4122-apply-symlink-inside.sh | 2 +-
t/t5302-pack-index.sh | 2 +-
t/t5503-tagfollow.sh | 2 +-
t/t5522-pull-symlink.sh | 2 +-
t/t5540-http-push.sh | 2 +-
t/t5541-http-push.sh | 2 +-
t/t5550-http-fetch.sh | 2 +-
t/t5551-http-fetch.sh | 2 +-
t/t5561-http-backend.sh | 2 +-
t/t5705-clone-2gb.sh | 2 +-
t/t6035-merge-dir-to-symlink.sh | 2 +-
t/t7004-tag.sh | 2 +-
| 2 +-
t/t7800-difftool.sh | 2 +-
t/t9001-send-email.sh | 4 ++--
t/t9100-git-svn-basic.sh | 2 +-
t/t9119-git-svn-info.sh | 2 +-
t/t9129-git-svn-i18n-commitencoding.sh | 2 +-
t/t9143-git-svn-gc.sh | 2 +-
t/t9200-git-cvsexportcommit.sh | 4 ++--
t/t9400-git-cvsserver-server.sh | 6 +++---
t/t9401-git-cvsserver-crlf.sh | 6 +++---
t/t9600-cvsimport.sh | 2 +-
t/t9700-perl-git.sh | 4 ++--
40 files changed, 56 insertions(+), 54 deletions(-)
diff --git a/t/lib-git-svn.sh b/t/lib-git-svn.sh
index 0f7f35c..344785d 100644
--- a/t/lib-git-svn.sh
+++ b/t/lib-git-svn.sh
@@ -5,11 +5,11 @@ git_svn_id=git""-svn-id
if test -n "$NO_SVN_TESTS"
then
- say 'skipping git svn tests, NO_SVN_TESTS defined'
+ skip_all='skipping git svn tests, NO_SVN_TESTS defined'
test_done
fi
if ! test_have_prereq PERL; then
- say 'skipping git svn tests, perl not available'
+ skip_all='skipping git svn tests, perl not available'
test_done
fi
diff --git a/t/lib-httpd.sh b/t/lib-httpd.sh
index da4b8d5..a0944d6 100644
--- a/t/lib-httpd.sh
+++ b/t/lib-httpd.sh
@@ -5,8 +5,7 @@
if test -z "$GIT_TEST_HTTPD"
then
- say "skipping test, network testing disabled by default"
- say "(define GIT_TEST_HTTPD to enable)"
+ skip_all="Network testing disabled (define GIT_TEST_HTTPD to enable)"
test_done
fi
diff --git a/t/t1304-default-acl.sh b/t/t1304-default-acl.sh
index 055ad00..97ab02a 100755
--- a/t/t1304-default-acl.sh
+++ b/t/t1304-default-acl.sh
@@ -15,9 +15,12 @@ umask 077
# is a good candidate: exists on all unices, and it has permission
# anyway, so we don't create a security hole running the testsuite.
-if ! setfacl -m u:root:rwx .; then
- say "Skipping ACL tests: unable to use setfacl"
- test_done
+setfacl_out="$(setfacl -m u:root:rwx . 2>&1)"
+setfacl_ret=$?
+
+if [ $setfacl_ret != 0 ]; then
+ skip_all="Skipping ACL tests: unable to use setfacl (output: '$setfacl_out'; return code: '$setfacl_ret')"
+ test_done
fi
check_perms_and_acl () {
diff --git a/t/t1509-root-worktree.sh b/t/t1509-root-worktree.sh
index 5322a3b..7f60fd0 100755
--- a/t/t1509-root-worktree.sh
+++ b/t/t1509-root-worktree.sh
@@ -99,17 +99,17 @@ test_foobar_foobar() {
}
if ! test_have_prereq POSIXPERM || ! [ -w / ]; then
- say "Dangerous test skipped. Read this test if you want to execute it"
+ skip_all="Dangerous test skipped. Read this test if you want to execute it"
test_done
fi
if [ "$IKNOWWHATIAMDOING" != "YES" ]; then
- say "You must set env var IKNOWWHATIAMDOING=YES in order to run this test"
+ skip_all="You must set env var IKNOWWHATIAMDOING=YES in order to run this test"
test_done
fi
if [ "$UID" = 0 ]; then
- say "No you can't run this with root"
+ skip_all="No you can't run this with root"
test_done
fi
diff --git a/t/t2007-checkout-symlink.sh b/t/t2007-checkout-symlink.sh
index 27e2127..05cc8fd 100755
--- a/t/t2007-checkout-symlink.sh
+++ b/t/t2007-checkout-symlink.sh
@@ -8,7 +8,7 @@ test_description='git checkout to switch between branches with symlink<->dir'
if ! test_have_prereq SYMLINKS
then
- say "symbolic links not supported - skipping tests"
+ skip_all="symbolic links not supported - skipping tests"
test_done
fi
diff --git a/t/t3300-funny-names.sh b/t/t3300-funny-names.sh
index db46d53..a99e4d8 100755
--- a/t/t3300-funny-names.sh
+++ b/t/t3300-funny-names.sh
@@ -26,7 +26,7 @@ echo 'Foo Bar Baz' >"$p2"
test -f "$p1" && cmp "$p0" "$p1" || {
# since FAT/NTFS does not allow tabs in filenames, skip this test
- say 'Your filesystem does not allow tabs in filenames, test skipped.'
+ skip_all='Your filesystem does not allow tabs in filenames, test skipped.'
test_done
}
diff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh
index ee84fc4..361a10a 100755
--- a/t/t3302-notes-index-expensive.sh
+++ b/t/t3302-notes-index-expensive.sh
@@ -8,7 +8,7 @@ test_description='Test commit notes index (expensive!)'
. ./test-lib.sh
test -z "$GIT_NOTES_TIMING_TESTS" && {
- say Skipping timing tests
+ skip_all="Skipping timing tests"
test_done
exit
}
diff --git a/t/t3600-rm.sh b/t/t3600-rm.sh
index 0aaf0ad..b514cbb 100755
--- a/t/t3600-rm.sh
+++ b/t/t3600-rm.sh
@@ -39,7 +39,7 @@ if test -f test-file
then
test_set_prereq RO_DIR
else
- say 'skipping removal failure test (perhaps running as root?)'
+ skip_all='skipping removal failure test (perhaps running as root?)'
fi
chmod 775 .
rm -f test-file
diff --git a/t/t3701-add-interactive.sh b/t/t3701-add-interactive.sh
index b6eba6a..7ad8465 100755
--- a/t/t3701-add-interactive.sh
+++ b/t/t3701-add-interactive.sh
@@ -4,7 +4,7 @@ test_description='add -i basic tests'
. ./test-lib.sh
if ! test_have_prereq PERL; then
- say 'skipping git add -i tests, perl not available'
+ skip_all='skipping git add -i tests, perl not available'
test_done
fi
@@ -154,7 +154,7 @@ rm -f .gitignore
if test "$(git config --bool core.filemode)" = false
then
- say 'skipping filemode tests (filesystem does not properly support modes)'
+ say '# skipping filemode tests (filesystem does not properly support modes)'
else
test_set_prereq FILEMODE
fi
diff --git a/t/t3902-quoted.sh b/t/t3902-quoted.sh
index 29103f6..147e634 100755
--- a/t/t3902-quoted.sh
+++ b/t/t3902-quoted.sh
@@ -17,7 +17,7 @@ DQ='"'
echo foo 2>/dev/null > "Name and an${HT}HT"
test -f "Name and an${HT}HT" || {
# since FAT/NTFS does not allow tabs in filenames, skip this test
- say 'Your filesystem does not allow tabs in filenames, test skipped.'
+ skip_all='Your filesystem does not allow tabs in filenames, test skipped.'
test_done
}
diff --git a/t/t4004-diff-rename-symlink.sh b/t/t4004-diff-rename-symlink.sh
index a4da119..1a09e8d 100755
--- a/t/t4004-diff-rename-symlink.sh
+++ b/t/t4004-diff-rename-symlink.sh
@@ -14,7 +14,7 @@ by an edit for them.
if ! test_have_prereq SYMLINKS
then
- say 'Symbolic links not supported, skipping tests.'
+ skip_all='Symbolic links not supported, skipping tests.'
test_done
fi
diff --git a/t/t4011-diff-symlink.sh b/t/t4011-diff-symlink.sh
index e12fbea..918a21a 100755
--- a/t/t4011-diff-symlink.sh
+++ b/t/t4011-diff-symlink.sh
@@ -11,7 +11,7 @@ test_description='Test diff of symlinks.
if ! test_have_prereq SYMLINKS
then
- say 'Symbolic links not supported, skipping tests.'
+ skip_all='Symbolic links not supported, skipping tests.'
test_done
fi
diff --git a/t/t4016-diff-quote.sh b/t/t4016-diff-quote.sh
index 55eb5f8..34e5144 100755
--- a/t/t4016-diff-quote.sh
+++ b/t/t4016-diff-quote.sh
@@ -14,7 +14,7 @@ P2='pathname with SP'
P3='pathname
with LF'
: 2>/dev/null >"$P1" && test -f "$P1" && rm -f "$P1" || {
- say 'Your filesystem does not allow tabs in filenames, test skipped.'
+ skip_all='Your filesystem does not allow tabs in filenames, test skipped.'
test_done
}
diff --git a/t/t4023-diff-rename-typechange.sh b/t/t4023-diff-rename-typechange.sh
index 9bdf659..40a95a1 100755
--- a/t/t4023-diff-rename-typechange.sh
+++ b/t/t4023-diff-rename-typechange.sh
@@ -6,7 +6,7 @@ test_description='typechange rename detection'
if ! test_have_prereq SYMLINKS
then
- say 'Symbolic links not supported, skipping tests.'
+ skip_all='Symbolic links not supported, skipping tests.'
test_done
fi
diff --git a/t/t4114-apply-typechange.sh b/t/t4114-apply-typechange.sh
index 99ec13d..164d58c 100755
--- a/t/t4114-apply-typechange.sh
+++ b/t/t4114-apply-typechange.sh
@@ -11,7 +11,7 @@ test_description='git apply should not get confused with type changes.
if ! test_have_prereq SYMLINKS
then
- say 'Symbolic links not supported, skipping tests.'
+ skip_all='Symbolic links not supported, skipping tests.'
test_done
fi
diff --git a/t/t4115-apply-symlink.sh b/t/t4115-apply-symlink.sh
index b852e58..aff4348 100755
--- a/t/t4115-apply-symlink.sh
+++ b/t/t4115-apply-symlink.sh
@@ -11,7 +11,7 @@ test_description='git apply symlinks and partial files
if ! test_have_prereq SYMLINKS
then
- say 'Symbolic links not supported, skipping tests.'
+ skip_all='Symbolic links not supported, skipping tests.'
test_done
fi
diff --git a/t/t4122-apply-symlink-inside.sh b/t/t4122-apply-symlink-inside.sh
index 0d3c1d5..923fcab 100755
--- a/t/t4122-apply-symlink-inside.sh
+++ b/t/t4122-apply-symlink-inside.sh
@@ -5,7 +5,7 @@ test_description='apply to deeper directory without getting fooled with symlink'
if ! test_have_prereq SYMLINKS
then
- say 'Symbolic links not supported, skipping tests.'
+ skip_all='Symbolic links not supported, skipping tests.'
test_done
fi
diff --git a/t/t5302-pack-index.sh b/t/t5302-pack-index.sh
index 4360e77..fb3a270 100755
--- a/t/t5302-pack-index.sh
+++ b/t/t5302-pack-index.sh
@@ -74,7 +74,7 @@ if msg=$(git verify-pack -v "test-3-${pack3}.pack" 2>&1) ||
then
test_set_prereq OFF64_T
else
- say "skipping tests concerning 64-bit offsets"
+ say "# skipping tests concerning 64-bit offsets"
fi
test_expect_success OFF64_T \
diff --git a/t/t5503-tagfollow.sh b/t/t5503-tagfollow.sh
index d5db75d..bab1a53 100755
--- a/t/t5503-tagfollow.sh
+++ b/t/t5503-tagfollow.sh
@@ -6,7 +6,7 @@ test_description='test automatic tag following'
case $(uname -s) in
*MINGW*)
- say "GIT_DEBUG_SEND_PACK not supported - skipping tests"
+ skip_all="GIT_DEBUG_SEND_PACK not supported - skipping tests"
test_done
esac
diff --git a/t/t5522-pull-symlink.sh b/t/t5522-pull-symlink.sh
index 7206817..298200f 100755
--- a/t/t5522-pull-symlink.sh
+++ b/t/t5522-pull-symlink.sh
@@ -6,7 +6,7 @@ test_description='pulling from symlinked subdir'
if ! test_have_prereq SYMLINKS
then
- say 'Symbolic links not supported, skipping tests.'
+ skip_all='Symbolic links not supported, skipping tests.'
test_done
fi
diff --git a/t/t5540-http-push.sh b/t/t5540-http-push.sh
index 37fe875..a266ca5 100755
--- a/t/t5540-http-push.sh
+++ b/t/t5540-http-push.sh
@@ -11,7 +11,7 @@ This test runs various sanity checks on http-push.'
if git http-push > /dev/null 2>&1 || [ $? -eq 128 ]
then
- say "skipping test, USE_CURL_MULTI is not defined"
+ skip_all="skipping test, USE_CURL_MULTI is not defined"
test_done
fi
diff --git a/t/t5541-http-push.sh b/t/t5541-http-push.sh
index 17e1bdc..504884b 100755
--- a/t/t5541-http-push.sh
+++ b/t/t5541-http-push.sh
@@ -7,7 +7,7 @@ test_description='test smart pushing over http via http-backend'
. ./test-lib.sh
if test -n "$NO_CURL"; then
- say 'skipping test, git built without http support'
+ skip_all='skipping test, git built without http support'
test_done
fi
diff --git a/t/t5550-http-fetch.sh b/t/t5550-http-fetch.sh
index fc675b5..2fb48d0 100755
--- a/t/t5550-http-fetch.sh
+++ b/t/t5550-http-fetch.sh
@@ -4,7 +4,7 @@ test_description='test dumb fetching over http via static file'
. ./test-lib.sh
if test -n "$NO_CURL"; then
- say 'skipping test, git built without http support'
+ skip_all='skipping test, git built without http support'
test_done
fi
diff --git a/t/t5551-http-fetch.sh b/t/t5551-http-fetch.sh
index 7faa31a..fd19121 100755
--- a/t/t5551-http-fetch.sh
+++ b/t/t5551-http-fetch.sh
@@ -4,7 +4,7 @@ test_description='test smart fetching over http via http-backend'
. ./test-lib.sh
if test -n "$NO_CURL"; then
- say 'skipping test, git built without http support'
+ skip_all='skipping test, git built without http support'
test_done
fi
diff --git a/t/t5561-http-backend.sh b/t/t5561-http-backend.sh
index 8c6d0b2..b5d7fbc 100755
--- a/t/t5561-http-backend.sh
+++ b/t/t5561-http-backend.sh
@@ -4,7 +4,7 @@ test_description='test git-http-backend'
. ./test-lib.sh
if test -n "$NO_CURL"; then
- say 'skipping test, git built without http support'
+ skip_all='skipping test, git built without http support'
test_done
fi
diff --git a/t/t5705-clone-2gb.sh b/t/t5705-clone-2gb.sh
index 8afbdd4..e4d1b6a 100755
--- a/t/t5705-clone-2gb.sh
+++ b/t/t5705-clone-2gb.sh
@@ -4,7 +4,7 @@ test_description='Test cloning a repository larger than 2 gigabyte'
. ./test-lib.sh
test -z "$GIT_TEST_CLONE_2GB" &&
-say "Skipping expensive 2GB clone test; enable it with GIT_TEST_CLONE_2GB=t" &&
+skip_all="Skipping expensive 2GB clone test; enable it with GIT_TEST_CLONE_2GB=t" &&
test_done &&
exit
diff --git a/t/t6035-merge-dir-to-symlink.sh b/t/t6035-merge-dir-to-symlink.sh
index 3202e1d..cd3190c 100755
--- a/t/t6035-merge-dir-to-symlink.sh
+++ b/t/t6035-merge-dir-to-symlink.sh
@@ -5,7 +5,7 @@ test_description='merging when a directory was replaced with a symlink'
if ! test_have_prereq SYMLINKS
then
- say 'Symbolic links not supported, skipping tests.'
+ skip_all='Symbolic links not supported, skipping tests.'
test_done
fi
diff --git a/t/t7004-tag.sh b/t/t7004-tag.sh
index 73dbc43..ac943f5 100755
--- a/t/t7004-tag.sh
+++ b/t/t7004-tag.sh
@@ -583,7 +583,7 @@ test_expect_success \
# subsequent tests require gpg; check if it is available
gpg --version >/dev/null 2>/dev/null
if [ $? -eq 127 ]; then
- say "gpg not found - skipping tag signing and verification tests"
+ say "# gpg not found - skipping tag signing and verification tests"
else
# As said here: http://www.gnupg.org/documentation/faqs.html#q6.19
# the gpg version 1.0.6 didn't parse trust packets correctly, so for
--git a/t/t7006-pager.sh b/t/t7006-pager.sh
index a6f3677..608f0ce 100755
--- a/t/t7006-pager.sh
+++ b/t/t7006-pager.sh
@@ -36,7 +36,7 @@ then
}
test_set_prereq TTY
else
- say no usable terminal, so skipping some tests
+ say "# no usable terminal, so skipping some tests"
fi
test_expect_success 'setup' '
diff --git a/t/t7800-difftool.sh b/t/t7800-difftool.sh
index 1de83ef..196827e 100755
--- a/t/t7800-difftool.sh
+++ b/t/t7800-difftool.sh
@@ -11,7 +11,7 @@ Testing basic diff tool invocation
. ./test-lib.sh
if ! test_have_prereq PERL; then
- say 'skipping difftool tests, perl not available'
+ skip_all='skipping difftool tests, perl not available'
test_done
fi
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 640b3d2..ddc3d8d 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -4,7 +4,7 @@ test_description='git send-email'
. ./test-lib.sh
if ! test_have_prereq PERL; then
- say 'skipping git send-email tests, perl not available'
+ skip_all='skipping git send-email tests, perl not available'
test_done
fi
@@ -58,7 +58,7 @@ test_no_confirm () {
# Exit immediately to prevent hang if a no-confirm test fails
check_no_confirm () {
test -f no_confirm_okay || {
- say 'No confirm test failed; skipping remaining tests to prevent hanging'
+ skip_all='confirm test failed; skipping remaining tests to prevent hanging'
test_done
}
}
diff --git a/t/t9100-git-svn-basic.sh b/t/t9100-git-svn-basic.sh
index 570e035..13766ab 100755
--- a/t/t9100-git-svn-basic.sh
+++ b/t/t9100-git-svn-basic.sh
@@ -15,7 +15,7 @@ case "$GIT_SVN_LC_ALL" in
test_set_prereq UTF8
;;
*)
- say "UTF-8 locale not set, some tests skipped ($GIT_SVN_LC_ALL)"
+ say "# UTF-8 locale not set, some tests skipped ($GIT_SVN_LC_ALL)"
;;
esac
diff --git a/t/t9119-git-svn-info.sh b/t/t9119-git-svn-info.sh
index a9a558d..5fb94fb 100755
--- a/t/t9119-git-svn-info.sh
+++ b/t/t9119-git-svn-info.sh
@@ -13,7 +13,7 @@ case $v in
1.[456].*)
;;
*)
- say "skipping svn-info test (SVN version: $v not supported)"
+ skip_all="skipping svn-info test (SVN version: $v not supported)"
test_done
;;
esac
diff --git a/t/t9129-git-svn-i18n-commitencoding.sh b/t/t9129-git-svn-i18n-commitencoding.sh
index 1e9a2eb..8cfdfe7 100755
--- a/t/t9129-git-svn-i18n-commitencoding.sh
+++ b/t/t9129-git-svn-i18n-commitencoding.sh
@@ -23,7 +23,7 @@ if test -n "$a_utf8_locale"
then
test_set_prereq UTF8
else
- say "UTF-8 locale not available, some tests are skipped"
+ say "# UTF-8 locale not available, some tests are skipped"
fi
compare_svn_head_with () {
diff --git a/t/t9143-git-svn-gc.sh b/t/t9143-git-svn-gc.sh
index 99f69c6..337ea59 100755
--- a/t/t9143-git-svn-gc.sh
+++ b/t/t9143-git-svn-gc.sh
@@ -43,7 +43,7 @@ then
gunzip .git/svn/refs/remotes/git-svn/unhandled.log.gz
'
else
- say "Perl Compress::Zlib unavailable, skipping gunzip test"
+ say "# Perl Compress::Zlib unavailable, skipping gunzip test"
fi
test_expect_success 'git svn gc does not change unhandled.log files' '
diff --git a/t/t9200-git-cvsexportcommit.sh b/t/t9200-git-cvsexportcommit.sh
index 61bcb8f..ee39b36 100755
--- a/t/t9200-git-cvsexportcommit.sh
+++ b/t/t9200-git-cvsexportcommit.sh
@@ -7,14 +7,14 @@ test_description='Test export of commits to CVS'
. ./test-lib.sh
if ! test_have_prereq PERL; then
- say 'skipping git cvsexportcommit tests, perl not available'
+ skip_all='skipping git cvsexportcommit tests, perl not available'
test_done
fi
cvs >/dev/null 2>&1
if test $? -ne 1
then
- say 'skipping git cvsexportcommit tests, cvs not found'
+ skip_all='skipping git cvsexportcommit tests, cvs not found'
test_done
fi
diff --git a/t/t9400-git-cvsserver-server.sh b/t/t9400-git-cvsserver-server.sh
index 8639506..36c457e 100755
--- a/t/t9400-git-cvsserver-server.sh
+++ b/t/t9400-git-cvsserver-server.sh
@@ -11,17 +11,17 @@ cvs CLI client via git-cvsserver server'
. ./test-lib.sh
if ! test_have_prereq PERL; then
- say 'skipping git cvsserver tests, perl not available'
+ skip_all='skipping git cvsserver tests, perl not available'
test_done
fi
cvs >/dev/null 2>&1
if test $? -ne 1
then
- say 'skipping git-cvsserver tests, cvs not found'
+ skip_all='skipping git-cvsserver tests, cvs not found'
test_done
fi
"$PERL_PATH" -e 'use DBI; use DBD::SQLite' >/dev/null 2>&1 || {
- say 'skipping git-cvsserver tests, Perl SQLite interface unavailable'
+ skip_all='skipping git-cvsserver tests, Perl SQLite interface unavailable'
test_done
}
diff --git a/t/t9401-git-cvsserver-crlf.sh b/t/t9401-git-cvsserver-crlf.sh
index ed7b513..925bd0f 100755
--- a/t/t9401-git-cvsserver-crlf.sh
+++ b/t/t9401-git-cvsserver-crlf.sh
@@ -41,16 +41,16 @@ not_present() {
cvs >/dev/null 2>&1
if test $? -ne 1
then
- say 'skipping git-cvsserver tests, cvs not found'
+ skip_all='skipping git-cvsserver tests, cvs not found'
test_done
fi
if ! test_have_prereq PERL
then
- say 'skipping git-cvsserver tests, perl not available'
+ skip_all='skipping git-cvsserver tests, perl not available'
test_done
fi
"$PERL_PATH" -e 'use DBI; use DBD::SQLite' >/dev/null 2>&1 || {
- say 'skipping git-cvsserver tests, Perl SQLite interface unavailable'
+ skip_all='skipping git-cvsserver tests, Perl SQLite interface unavailable'
test_done
}
diff --git a/t/t9600-cvsimport.sh b/t/t9600-cvsimport.sh
index b572ce3..2eff9cd 100755
--- a/t/t9600-cvsimport.sh
+++ b/t/t9600-cvsimport.sh
@@ -4,7 +4,7 @@ test_description='git cvsimport basic tests'
. ./lib-cvs.sh
if ! test_have_prereq PERL; then
- say 'skipping git cvsimport tests, perl not available'
+ skip_all='skipping git cvsimport tests, perl not available'
test_done
fi
diff --git a/t/t9700-perl-git.sh b/t/t9700-perl-git.sh
index 8686086..3999bf7 100755
--- a/t/t9700-perl-git.sh
+++ b/t/t9700-perl-git.sh
@@ -7,12 +7,12 @@ test_description='perl interface (Git.pm)'
. ./test-lib.sh
if ! test_have_prereq PERL; then
- say 'skipping perl interface tests, perl not available'
+ skip_all='skipping perl interface tests, perl not available'
test_done
fi
"$PERL_PATH" -MTest::More -e 0 2>/dev/null || {
- say "Perl Test::More unavailable, skipping test"
+ skip_all="Perl Test::More unavailable, skipping test"
test_done
}
--
1.7.1.251.g92a7
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v3 3/5] We use TAP so the Perl test can run without scaffolding
2010-06-15 15:17 ` Ævar Arnfjörð Bjarmason
` (3 preceding siblings ...)
2010-06-15 22:32 ` [PATCH v3 2/5] Skip tests in a way that makes sense under TAP Ævar Arnfjörð Bjarmason
@ 2010-06-15 22:32 ` Ævar Arnfjörð Bjarmason
2010-06-15 22:32 ` [PATCH v3 4/5] TAP: Say "pass" rather than "ok" on an empty line Ævar Arnfjörð Bjarmason
2010-06-15 22:32 ` [PATCH v3 5/5] TAP: Make sure there's a newline before "ok" under harness Ævar Arnfjörð Bjarmason
6 siblings, 0 replies; 20+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2010-06-15 22:32 UTC (permalink / raw
To: git; +Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason
Before TAP we ran the Perl test and assumed that it failed if nothing
was printed on STDERR. Now we pass control over to it and rely on its
exit code, which is only non-zero if a test fails.
Running the Perl test with --verbose now gives meaningful output.
This removes the test_external and test_external_without_stderr
functions added by Lea Wiemann in
fb32c410087e68d650b31f68e66b3d9cbcce4a56. Nothing else used them, and
now that we're using TAP they shouldn't be necessary.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
t/t9700-perl-git.sh | 8 ++----
t/t9700/test.pl | 11 +++++++++
t/test-lib.sh | 62 ---------------------------------------------------
3 files changed, 14 insertions(+), 67 deletions(-)
diff --git a/t/t9700-perl-git.sh b/t/t9700-perl-git.sh
index 3999bf7..17252ab 100755
--- a/t/t9700-perl-git.sh
+++ b/t/t9700-perl-git.sh
@@ -46,8 +46,6 @@ test_expect_success \
git config --add test.int 2k
'
-test_external_without_stderr \
- 'Perl API' \
- "$PERL_PATH" "$TEST_DIRECTORY"/t9700/test.pl
-
-test_done
+"$PERL_PATH" "$TEST_DIRECTORY"/t9700/test.pl &&
+ # The Perl test finalizes the plan, so don't call test_done() here.
+ GIT_EXIT_OK=t
diff --git a/t/t9700/test.pl b/t/t9700/test.pl
index 666722d..e5d4b03 100755
--- a/t/t9700/test.pl
+++ b/t/t9700/test.pl
@@ -7,6 +7,13 @@ use strict;
use Test::More qw(no_plan);
+BEGIN {
+ # t9700-perl-git.sh kicks off our testing, so we have to go from
+ # there.
+ $Test::Builder::Test->{Curr_Test} = 1;
+ $Test::Builder::Test->{No_Ending} = 1;
+}
+
use Cwd;
use File::Basename;
@@ -105,3 +112,7 @@ my $last_commit = $r2->command_oneline(qw(rev-parse --verify HEAD));
like($last_commit, qr/^[0-9a-fA-F]{40}$/, 'rev-parse returned hash');
my $dir_commit = $r2->command_oneline('log', '-n1', '--pretty=format:%H', '.');
isnt($last_commit, $dir_commit, 'log . does not show last commit');
+
+printf "1..%d\n", $Test::Builder::Test->{Curr_Test};
+
+exit($Test::Builder::Test->{Is_Passing} ? 0 : 1);
diff --git a/t/test-lib.sh b/t/test-lib.sh
index e97645c..37987d7 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -453,68 +453,6 @@ test_expect_code () {
echo >&3 ""
}
-# test_external runs external test scripts that provide continuous
-# test output about their progress, and succeeds/fails on
-# zero/non-zero exit code. It outputs the test output on stdout even
-# in non-verbose mode, and announces the external script with "* run
-# <n>: ..." before running it. When providing relative paths, keep in
-# mind that all scripts run in "trash directory".
-# Usage: test_external description command arguments...
-# Example: test_external 'Perl API' perl ../path/to/test.pl
-test_external () {
- test "$#" = 4 && { prereq=$1; shift; } || prereq=
- test "$#" = 3 ||
- error >&5 "bug in the test script: not 3 or 4 parameters to test_external"
- descr="$1"
- shift
- if ! test_skip "$descr" "$@"
- then
- # Announce the script to reduce confusion about the
- # test output that follows.
- say_color "" " run $test_count: $descr ($*)"
- # Export TEST_DIRECTORY, TRASH_DIRECTORY and GIT_TEST_LONG
- # to be able to use them in script
- export TEST_DIRECTORY TRASH_DIRECTORY GIT_TEST_LONG
- # Run command; redirect its stderr to &4 as in
- # test_run_, but keep its stdout on our stdout even in
- # non-verbose mode.
- "$@" 2>&4
- if [ "$?" = 0 ]
- then
- test_ok_ "$descr"
- else
- test_failure_ "$descr" "$@"
- fi
- fi
-}
-
-# Like test_external, but in addition tests that the command generated
-# no output on stderr.
-test_external_without_stderr () {
- # The temporary file has no (and must have no) security
- # implications.
- tmp="$TMPDIR"; if [ -z "$tmp" ]; then tmp=/tmp; fi
- stderr="$tmp/git-external-stderr.$$.tmp"
- test_external "$@" 4> "$stderr"
- [ -f "$stderr" ] || error "Internal error: $stderr disappeared."
- descr="no stderr: $1"
- shift
- say >&3 "expecting no stderr from previous command"
- if [ ! -s "$stderr" ]; then
- rm "$stderr"
- test_ok_ "$descr"
- else
- if [ "$verbose" = t ]; then
- output=`echo; echo Stderr is:; cat "$stderr"`
- else
- output=
- fi
- # rm first in case test_failure exits.
- rm "$stderr"
- test_failure_ "$descr" "$@" "$output"
- fi
-}
-
# This is not among top-level (test_expect_success | test_expect_failure)
# but is a prefix that can be used in the test script, like:
#
--
1.7.1.251.g92a7
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v3 4/5] TAP: Say "pass" rather than "ok" on an empty line
2010-06-15 15:17 ` Ævar Arnfjörð Bjarmason
` (4 preceding siblings ...)
2010-06-15 22:32 ` [PATCH v3 3/5] We use TAP so the Perl test can run without scaffolding Ævar Arnfjörð Bjarmason
@ 2010-06-15 22:32 ` Ævar Arnfjörð Bjarmason
2010-06-15 22:32 ` [PATCH v3 5/5] TAP: Make sure there's a newline before "ok" under harness Ævar Arnfjörð Bjarmason
6 siblings, 0 replies; 20+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2010-06-15 22:32 UTC (permalink / raw
To: git; +Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason
Lines that begin with "ok" confuse the TAP harness because it can't
distinguish them from a test counter. Work around the issue by saying
"pass" instead, which isn't a reserved TAP word.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
t/t1020-subdirectory.sh | 12 ++++++------
t/t2102-update-index-symlinks.sh | 2 +-
t/t3700-add.sh | 12 ++++++------
3 files changed, 13 insertions(+), 13 deletions(-)
diff --git a/t/t1020-subdirectory.sh b/t/t1020-subdirectory.sh
index 210e594..5687499 100755
--- a/t/t1020-subdirectory.sh
+++ b/t/t1020-subdirectory.sh
@@ -24,18 +24,18 @@ test_expect_success 'update-index and ls-files' '
cd "$HERE" &&
git update-index --add one &&
case "`git ls-files`" in
- one) echo ok one ;;
+ one) echo pass one ;;
*) echo bad one; exit 1 ;;
esac &&
cd dir &&
git update-index --add two &&
case "`git ls-files`" in
- two) echo ok two ;;
+ two) echo pass two ;;
*) echo bad two; exit 1 ;;
esac &&
cd .. &&
case "`git ls-files`" in
- dir/two"$LF"one) echo ok both ;;
+ dir/two"$LF"one) echo pass both ;;
*) echo bad; exit 1 ;;
esac
'
@@ -58,17 +58,17 @@ test_expect_success 'diff-files' '
echo a >>one &&
echo d >>dir/two &&
case "`git diff-files --name-only`" in
- dir/two"$LF"one) echo ok top ;;
+ dir/two"$LF"one) echo pass top ;;
*) echo bad top; exit 1 ;;
esac &&
# diff should not omit leading paths
cd dir &&
case "`git diff-files --name-only`" in
- dir/two"$LF"one) echo ok subdir ;;
+ dir/two"$LF"one) echo pass subdir ;;
*) echo bad subdir; exit 1 ;;
esac &&
case "`git diff-files --name-only .`" in
- dir/two) echo ok subdir limited ;;
+ dir/two) echo pass subdir limited ;;
*) echo bad subdir limited; exit 1 ;;
esac
'
diff --git a/t/t2102-update-index-symlinks.sh b/t/t2102-update-index-symlinks.sh
index 1ed44ee..4d0d0a3 100755
--- a/t/t2102-update-index-symlinks.sh
+++ b/t/t2102-update-index-symlinks.sh
@@ -24,7 +24,7 @@ git update-index symlink'
test_expect_success \
'the index entry must still be a symbolic link' '
case "`git ls-files --stage --cached symlink`" in
-120000" "*symlink) echo ok;;
+120000" "*symlink) echo pass;;
*) echo fail; git ls-files --stage --cached symlink; (exit 1);;
esac'
diff --git a/t/t3700-add.sh b/t/t3700-add.sh
index 525c9a8..6f031af 100755
--- a/t/t3700-add.sh
+++ b/t/t3700-add.sh
@@ -26,7 +26,7 @@ test_expect_success \
chmod 755 xfoo1 &&
git add xfoo1 &&
case "`git ls-files --stage xfoo1`" in
- 100644" "*xfoo1) echo ok;;
+ 100644" "*xfoo1) echo pass;;
*) echo fail; git ls-files --stage xfoo1; (exit 1);;
esac'
@@ -35,7 +35,7 @@ test_expect_success SYMLINKS 'git add: filemode=0 should not get confused by sym
ln -s foo xfoo1 &&
git add xfoo1 &&
case "`git ls-files --stage xfoo1`" in
- 120000" "*xfoo1) echo ok;;
+ 120000" "*xfoo1) echo pass;;
*) echo fail; git ls-files --stage xfoo1; (exit 1);;
esac
'
@@ -47,7 +47,7 @@ test_expect_success \
chmod 755 xfoo2 &&
git update-index --add xfoo2 &&
case "`git ls-files --stage xfoo2`" in
- 100644" "*xfoo2) echo ok;;
+ 100644" "*xfoo2) echo pass;;
*) echo fail; git ls-files --stage xfoo2; (exit 1);;
esac'
@@ -56,7 +56,7 @@ test_expect_success SYMLINKS 'git add: filemode=0 should not get confused by sym
ln -s foo xfoo2 &&
git update-index --add xfoo2 &&
case "`git ls-files --stage xfoo2`" in
- 120000" "*xfoo2) echo ok;;
+ 120000" "*xfoo2) echo pass;;
*) echo fail; git ls-files --stage xfoo2; (exit 1);;
esac
'
@@ -67,7 +67,7 @@ test_expect_success SYMLINKS \
ln -s xfoo2 xfoo3 &&
git update-index --add xfoo3 &&
case "`git ls-files --stage xfoo3`" in
- 120000" "*xfoo3) echo ok;;
+ 120000" "*xfoo3) echo pass;;
*) echo fail; git ls-files --stage xfoo3; (exit 1);;
esac'
@@ -172,7 +172,7 @@ test_expect_success 'git add --refresh' '
test -z "`git diff-index HEAD -- foo`" &&
git read-tree HEAD &&
case "`git diff-index HEAD -- foo`" in
- :100644" "*"M foo") echo ok;;
+ :100644" "*"M foo") echo pass;;
*) echo fail; (exit 1);;
esac &&
git add --refresh -- foo &&
--
1.7.1.251.g92a7
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v3 5/5] TAP: Make sure there's a newline before "ok" under harness
2010-06-15 15:17 ` Ævar Arnfjörð Bjarmason
` (5 preceding siblings ...)
2010-06-15 22:32 ` [PATCH v3 4/5] TAP: Say "pass" rather than "ok" on an empty line Ævar Arnfjörð Bjarmason
@ 2010-06-15 22:32 ` Ævar Arnfjörð Bjarmason
6 siblings, 0 replies; 20+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2010-06-15 22:32 UTC (permalink / raw
To: git; +Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason
Some tests in the testsuite will emit a line that doesn't end with a
newline, right before we're about to output "ok" or "not ok". This
breaks the TAP output with "Tests out of sequence" errors since a TAP
harness can't understand this:
ok 1 - A test
[some output here]ok 2 - Another test
ok 3 - Yet another test
Work around it by emitting an empty line before we're about to say
"ok" or "not ok", but only if we're running under --verbose and
HARNESS_ACTIVE=1 is set, which'll only be the case when running under
a harnesses like prove(1).
I think it's better to do this than fix each tests by adding `&& echo'
everywhere. More tests might be added that break TAP in the future,
and a human isn't going to look at the extra whitespace, since
HARNESS_ACTIVE=1 always means a harness is reading it.
The tests that had issues were:
t1007, t3410, t3413, t3409, t3414, t3415, t3416, t3412, t3404,
t5407, t7402, t7003, t9001
With this workaround the entire test suite runs without errors under:
prove -j 10 ./t[0-9]*.sh :: --verbose
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
t/test-lib.sh | 3 +++
1 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 37987d7..86a46bf 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -369,6 +369,9 @@ test_run_ () {
eval >&3 2>&4 "$1"
eval_ret=$?
eval >&3 2>&4 "$test_cleanup"
+ if test "$verbose" = "t" && test -n "$HARNESS_ACTIVE"; then
+ echo ""
+ fi
return 0
}
--
1.7.1.251.g92a7
^ permalink raw reply related [flat|nested] 20+ messages in thread
end of thread, other threads:[~2010-06-15 22:34 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-06-09 15:22 [PATCH v2 0/3] Make the Git tests emit TAP format Ævar Arnfjörð Bjarmason
2010-06-09 15:22 ` [PATCH v2 1/3] Make test-lib.sh emit valid " Ævar Arnfjörð Bjarmason
2010-06-14 22:01 ` Jakub Narebski
2010-06-14 22:29 ` Ævar Arnfjörð Bjarmason
2010-06-09 15:22 ` [PATCH v2 2/3] Skip tests in a way that makes sense under TAP Ævar Arnfjörð Bjarmason
2010-06-09 15:24 ` [PATCH v2 3/3] We use TAP so the Perl test can run without scaffolding Ævar Arnfjörð Bjarmason
2010-06-14 21:49 ` [PATCH v2 0/3] Make the Git tests emit TAP format Jakub Narebski
2010-06-14 22:10 ` Ævar Arnfjörð Bjarmason
2010-06-14 23:16 ` Ævar Arnfjörð Bjarmason
2010-06-15 3:08 ` Junio C Hamano
2010-06-15 3:10 ` Ævar Arnfjörð Bjarmason
2010-06-15 15:17 ` Ævar Arnfjörð Bjarmason
2010-06-15 16:42 ` Andreas Ericsson
2010-06-15 16:49 ` Ævar Arnfjörð Bjarmason
2010-06-15 22:32 ` [PATCH v3 0/5] TAP support for Git Ævar Arnfjörð Bjarmason
2010-06-15 22:32 ` [PATCH v3 1/5] Make test-lib.sh emit valid TAP format Ævar Arnfjörð Bjarmason
2010-06-15 22:32 ` [PATCH v3 2/5] Skip tests in a way that makes sense under TAP Ævar Arnfjörð Bjarmason
2010-06-15 22:32 ` [PATCH v3 3/5] We use TAP so the Perl test can run without scaffolding Ævar Arnfjörð Bjarmason
2010-06-15 22:32 ` [PATCH v3 4/5] TAP: Say "pass" rather than "ok" on an empty line Ævar Arnfjörð Bjarmason
2010-06-15 22:32 ` [PATCH v3 5/5] TAP: Make sure there's a newline before "ok" under harness Ævar Arnfjörð Bjarmason
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).