[PATCH] git-notes: Run partial expensive test everywhere

[PATCH] git-notes: Run partial expensive test everywhere

[Ævar Arnfjörð Bjarmason]

The git-notes expensive timing test is only expensive because it
either did 1k iterations or nothing. Change it to do 10 by default,
with an option to run the expensive version with the old
GIT_NOTES_TIMING_TESTS=ZomgYesPlease variable.

Since nobody was ostensibly running this test under TAP the code had
bitrotted so that it emitted invalid TAP. This change fixes that.

The old version would also mysteriously fail on systems without
/usr/bin/time, there's now a check for that using the test
prerequisite facility.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t3302-notes-index-expensive.sh |   39 ++++++++++++++++++++++++++-----------
 1 files changed, 27 insertions(+), 12 deletions(-)

diff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh
index 361a10a..3b8313c 100755
--- a/t/t3302-notes-index-expensive.sh
+++ b/t/t3302-notes-index-expensive.sh
@@ -7,11 +7,16 @@ test_description='Test commit notes index (expensive!)'
 
 . ./test-lib.sh
 
-test -z "$GIT_NOTES_TIMING_TESTS" && {
-	skip_all="Skipping timing tests"
-	test_done
-	exit
-}
+test_set_prereq NOT_EXPENSIVE
+test -n "$GIT_NOTES_TIMING_TESTS" && test_set_prereq EXPENSIVE
+
+if test -x /usr/bin/time
+then
+	# Hack around multiple test prerequisites not supporting AND-ing
+	# of terms
+	test_set_prereq USR_BIN_TIME+NOT_EXPENSIVE
+	test_have_prereq EXPENSIVE && test_set_prereq USR_BIN_TIME+EXPENSIVE
+fi
 
 create_repo () {
 	number_of_commits=$1
@@ -102,17 +107,27 @@ time_notes () {
 	done
 }
 
-for count in 10 100 1000 10000; do
+do_tests () {
+	pr=$1
+	count=$2
+
+	test_expect_success $pr 'setup / mkdir' '
+		mkdir $count &&
+		cd $count
+	'
 
-	mkdir $count
-	(cd $count;
+	test_expect_success $pr "setup $count" "create_repo $count"
 
-	test_expect_success "setup $count" "create_repo $count"
+	test_expect_success $pr 'notes work' "test_notes $count"
 
-	test_expect_success 'notes work' "test_notes $count"
+	test_expect_success USR_BIN_TIME+$pr 'notes timing with /usr/bin/time' "time_notes 100"
+
+	test_expect_success $pr 'teardown / cd ..' 'cd ..'
+}
 
-	test_expect_success 'notes timing' "time_notes 100"
-	)
+do_tests NOT_EXPENSIVE 10
+for count in 100 1000 10000; do
+	do_tests EXPENSIVE $count
 done
 
 test_done
-- 
1.7.2.1.295.gd03d

Re: [PATCH] git-notes: Run partial expensive test everywhere

[Sverre Rabbelier]

Heya,

On Tue, Aug 10, 2010 at 14:56, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> The git-notes expensive timing test is only expensive because it
> either did 1k iterations or nothing. Change it to do 10 by default,
> with an option to run the expensive version with the old
> GIT_NOTES_TIMING_TESTS=ZomgYesPlease variable.

Nice, why 10 though? Any motivation for that particular value?

> Since nobody was ostensibly running this test under TAP the code had
> bitrotted so that it emitted invalid TAP. This change fixes that.

Nice catch.

> The old version would also mysteriously fail on systems without
> /usr/bin/time, there's now a check for that using the test
> prerequisite facility.

Should this patch be split up?

-- 
Cheers,

Sverre Rabbelier

Re: [PATCH] git-notes: Run partial expensive test everywhere

[Ævar Arnfjörð Bjarmason]

On Tue, Aug 10, 2010 at 20:29, Sverre Rabbelier <srabbelier@gmail.com> wrote:
> Heya,
>
> On Tue, Aug 10, 2010 at 14:56, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>> The git-notes expensive timing test is only expensive because it
>> either did 1k iterations or nothing. Change it to do 10 by default,
>> with an option to run the expensive version with the old
>> GIT_NOTES_TIMING_TESTS=ZomgYesPlease variable.
>
> Nice, why 10 though? Any motivation for that particular value?

The old version had "for count in 10 100 1000 10000; do". Mine has 10
as non-expensive, and "for count in 100 1000 10000; do" as expensive.

I.e. I'm running the first test batch from the old tests.

I have no idea whether it actually needs to run 10..10k times, I
didn't try to grok the actual test code.

>> The old version would also mysteriously fail on systems without
>> /usr/bin/time, there's now a check for that using the test
>> prerequisite facility.
>
> Should this patch be split up?

It all touched the same bits, it'd be nastier to split it up IMO.

Re: [PATCH] git-notes: Run partial expensive test everywhere

[Ævar Arnfjörð Bjarmason]

On Tue, Aug 10, 2010 at 19:56, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> +       # Hack around multiple test prerequisites not supporting AND-ing
> +       # of terms
> +       test_set_prereq USR_BIN_TIME+NOT_EXPENSIVE
> +       test_have_prereq EXPENSIVE && test_set_prereq USR_BIN_TIME+EXPENSIVE
> +fi

In retrospect this may have been some brainfried code, I'll check it
out tomorrow.

[PATCH v2] git-notes: Run partial expensive test everywhere

[Ævar Arnfjörð Bjarmason]

The git-notes expensive timing test is only expensive because it
either did 10,100,1k and 10k iterations or nothing.

Change it to do 10 by default, with an option to run the expensive
version with the old GIT_NOTES_TIMING_TESTS=ZomgYesPlease variable.

Since nobody was ostensibly running this test under TAP the code had
bitrotted so that it emitted invalid TAP. This change fixes that.

The old version would also mysteriously fail on systems without
/usr/bin/time, there's now a check for that using the multiple test
prerequisite facility.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---

On Tue, Aug 10, 2010 at 21:56, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> On Tue, Aug 10, 2010 at 19:56, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>> +       # Hack around multiple test prerequisites not supporting AND-ing
>> +       # of terms
>> +       test_set_prereq USR_BIN_TIME+NOT_EXPENSIVE
>> +       test_have_prereq EXPENSIVE && test_set_prereq USR_BIN_TIME+EXPENSIVE
>> +fi
>
> In retrospect this may have been some brainfried code, I'll check it
> out tomorrow.

Here's a patch that's not crazy. In v1 I was hacking around not having
a facility I already added to the test-lib (tired).

This patch goes on top of my "test-lib: Multi-prereq support only
checked the last prereq" patch, which fixes up the test prereq
facility so that it actually works.

 t/t3302-notes-index-expensive.sh |   32 ++++++++++++++++++++------------
 1 files changed, 20 insertions(+), 12 deletions(-)

diff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh
index 361a10a..7c08e99 100755
--- a/t/t3302-notes-index-expensive.sh
+++ b/t/t3302-notes-index-expensive.sh
@@ -7,11 +7,9 @@ test_description='Test commit notes index (expensive!)'
 
 . ./test-lib.sh
 
-test -z "$GIT_NOTES_TIMING_TESTS" && {
-	skip_all="Skipping timing tests"
-	test_done
-	exit
-}
+test_set_prereq NOT_EXPENSIVE
+test -n "$GIT_NOTES_TIMING_TESTS" && test_set_prereq EXPENSIVE
+test -x /usr/bin/time && test_set_prereq USR_BIN_TIME
 
 create_repo () {
 	number_of_commits=$1
@@ -102,17 +100,27 @@ time_notes () {
 	done
 }
 
-for count in 10 100 1000 10000; do
+do_tests () {
+	pr=$1
+	count=$2
+
+	test_expect_success $pr 'setup / mkdir' '
+		mkdir $count &&
+		cd $count
+	'
 
-	mkdir $count
-	(cd $count;
+	test_expect_success $pr "setup $count" "create_repo $count"
 
-	test_expect_success "setup $count" "create_repo $count"
+	test_expect_success $pr 'notes work' "test_notes $count"
 
-	test_expect_success 'notes work' "test_notes $count"
+	test_expect_success USR_BIN_TIME,$pr 'notes timing with /usr/bin/time' "time_notes 100"
+
+	test_expect_success $pr 'teardown / cd ..' 'cd ..'
+}
 
-	test_expect_success 'notes timing' "time_notes 100"
-	)
+do_tests NOT_EXPENSIVE 10
+for count in 100 1000 10000; do
+	do_tests EXPENSIVE $count
 done
 
 test_done
-- 
1.7.2.1.295.gd03d

Re: [PATCH v2] git-notes: Run partial expensive test everywhere

[Ævar Arnfjörð Bjarmason]

On Tue, Aug 10, 2010 at 23:37, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:

> +test -x /usr/bin/time && test_set_prereq USR_BIN_TIME

It turns out that this fails on Solaris because its /usr/bin/time is different.

Maybe this should just check if /usr/bin/time --version is ^GNU ?

Re: [PATCH v2] git-notes: Run partial expensive test everywhere

[Jonathan Nieder]

Ævar Arnfjörð Bjarmason wrote:

> It turns out that this fails on Solaris because its /usr/bin/time is different.

Odd.  Different how?  As far as I can tell, all that test asks
of time is to execv() its arguments and pass on a 0 exit status.

Ah, maybe this is it: perhaps /usr/bin/time sh runs /bin/sh.  Does the
following help?

Patch is against next.  Untested except on Linux where it wouldn't
make a difference.
-- 8< --
Subject: t3302 (notes): Port to Solaris

The time_notes script, which uses POSIX shell features, is
currently sometimes run with a non-POSIX /bin/sh.

Reported-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
diff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh
index 7c08e99..e35d781 100755
--- a/t/t3302-notes-index-expensive.sh
+++ b/t/t3302-notes-index-expensive.sh
@@ -96,7 +96,7 @@ time_notes () {
 	for mode in no-notes notes
 	do
 		echo $mode
-		/usr/bin/time sh ../time_notes $mode $1
+		/usr/bin/time "$SHELL_PATH" ../time_notes $mode $1
 	done
 }
 
-- 

Re: [PATCH v2] git-notes: Run partial expensive test everywhere

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> Odd.  Different how?  As far as I can tell, all that test asks
> of time is to execv() its arguments and pass on a 0 exit status.
>
> Ah, maybe this is it: perhaps /usr/bin/time sh runs /bin/sh.  Does the
> following help?
>
> Patch is against next.  Untested except on Linux where it wouldn't
> make a difference.

Makes sense, although I don't have access to Solaris boxes right now...

> -- 8< --
> Subject: t3302 (notes): Port to Solaris
>
> The time_notes script, which uses POSIX shell features, is
> currently sometimes run with a non-POSIX /bin/sh.
>
> Reported-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
> ---
> diff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh
> index 7c08e99..e35d781 100755
> --- a/t/t3302-notes-index-expensive.sh
> +++ b/t/t3302-notes-index-expensive.sh
> @@ -96,7 +96,7 @@ time_notes () {
>  	for mode in no-notes notes
>  	do
>  		echo $mode
> -		/usr/bin/time sh ../time_notes $mode $1
> +		/usr/bin/time "$SHELL_PATH" ../time_notes $mode $1
>  	done
>  }
>  
> -- 

Re: [PATCH v2] git-notes: Run partial expensive test everywhere

[Ævar Arnfjörð Bjarmason]

On Tue, Aug 24, 2010 at 16:46, Junio C Hamano <gitster@pobox.com> wrote:
> Jonathan Nieder <jrnieder@gmail.com> writes:
>
>> Odd.  Different how?  As far as I can tell, all that test asks
>> of time is to execv() its arguments and pass on a 0 exit status.
>>
>> Ah, maybe this is it: perhaps /usr/bin/time sh runs /bin/sh.  Does the
>> following help?
>>
>> Patch is against next.  Untested except on Linux where it wouldn't
>> make a difference.
>
> Makes sense, although I don't have access to Solaris boxes right now...

I tested it FWIW. It works, thanks both.