git 2.0.0 PROFILE=BUILD check-phase problems with ./t5561-http-backend.sh; GIT_TEST_HTTPD=false problems with t5537-fetch-shallow.sh

git 2.0.0 PROFILE=BUILD check-phase problems with ./t5561-http-backend.sh; GIT_TEST_HTTPD=false problems with t5537-fetch-shallow.sh

[Nix]

I observe test failures with git 2.0.0 which are attributable to the
change to run network tests by default. I'm lumping them both together
into one report because I'm lazy and I've blown too much time on this
already.

I've got Apache 2.2.24 on this box, and t5551-http-fetch-smart.sh fails
in a peculiar fashion during the PROFILE=BUILD stage:

--- exp 2014-05-29 22:42:50.221599297 +0000
+++ act 2014-05-29 22:42:50.231598452 +0000
@@ -12,10 +12,10 @@
 GET  /smart/repo.git/objects/info/http-alternates HTTP/1.1 200 -
 GET  /smart/repo.git/objects/41/57d6f47fc8b7cb455fbf6f18d6f47fed49a6a5 HTTP/1.1 200
 GET  /smart/repo.git/objects/pack/pack-2cb3186872e8768852fcb6b22cdf2182e12f7490.pack HTTP/1.1 200
-GET  /smart/repo.git/objects/pack/pack-2cb3186872e8768852fcb6b22cdf2182e12f7490.idx HTTP/1.1 200

 ###  no git-daemon-export-ok
 ###
+GET  /smart/repo.git/objects/pack/pack-2cb3186872e8768852fcb6b22cdf2182e12f7490.idx HTTP/1.1 200
 GET  /smart_noexport/repo.git/HEAD HTTP/1.1 404 -
 GET  /smart_noexport/repo.git/info/refs HTTP/1.1 404 -
 GET  /smart_noexport/repo.git/objects/info/packs HTTP/1.1 404 -
@@ -34,10 +34,10 @@
 GET  /smart_noexport/repo.git/objects/info/http-alternates HTTP/1.1 200 -
 GET  /smart_noexport/repo.git/objects/41/57d6f47fc8b7cb455fbf6f18d6f47fed49a6a5 HTTP/1.1 200
 GET  /smart_noexport/repo.git/objects/pack/pack-2cb3186872e8768852fcb6b22cdf2182e12f7490.pack HTTP/1.1 200
-GET  /smart_noexport/repo.git/objects/pack/pack-2cb3186872e8768852fcb6b22cdf2182e12f7490.idx HTTP/1.1 200

 ###  getanyfile true
 ###
+GET  /smart_noexport/repo.git/objects/pack/pack-2cb3186872e8768852fcb6b22cdf2182e12f7490.idx HTTP/1.1 200
 GET  /smart/repo.git/HEAD HTTP/1.1 200
 GET  /smart/repo.git/info/refs HTTP/1.1 200
 GET  /smart/repo.git/objects/info/packs HTTP/1.1 200
@@ -45,10 +45,10 @@
 GET  /smart/repo.git/objects/info/http-alternates HTTP/1.1 200 -
 GET  /smart/repo.git/objects/41/57d6f47fc8b7cb455fbf6f18d6f47fed49a6a5 HTTP/1.1 200
 GET  /smart/repo.git/objects/pack/pack-2cb3186872e8768852fcb6b22cdf2182e12f7490.pack HTTP/1.1 200
-GET  /smart/repo.git/objects/pack/pack-2cb3186872e8768852fcb6b22cdf2182e12f7490.idx HTTP/1.1 200

 ###  getanyfile false
 ###
+GET  /smart/repo.git/objects/pack/pack-2cb3186872e8768852fcb6b22cdf2182e12f7490.idx HTTP/1.1 200
 GET  /smart/repo.git/HEAD HTTP/1.1 403 -
 GET  /smart/repo.git/info/refs HTTP/1.1 403 -
 GET  /smart/repo.git/objects/info/packs HTTP/1.1 403 -
not ok 14 - server request log matches test results
#
#               sed -e "
#                       s/^.* \"//
#                       s/\"//
#                       s/ [1-9][0-9]*\$//
#                       s/^GET /GET  /
#               " >act <"$HTTPD_ROOT_PATH"/access.log &&
#               test_cmp exp act
#

# failed 1 among 14 test(s)

It appears that the Apache daemon is writing to the log slowly enough
that its log lines only get there after the testsuite has written its
separator, so a bunch of log lines appear to be attached to the wrong
test, and the comparison fails.

Curiously, I can't make this happen in a conventional 'make check', even
though the only relevant components would seem to be bash and httpd: if
anything, you'd expect the gcovvery to slow down git and thus make it
*more* likely that any race between httpd log syncing and testsuite
framework output to the same logfile would be hit...

Attempting to work around this by building with GIT_TEST_HTTPD=false
doesn't work either:

*** t5537-fetch-shallow.sh ***
ok 1 - setup
ok 2 - setup shallow clone
ok 3 - clone from shallow clone
ok 4 - fetch from shallow clone
ok 5 - fetch --depth from shallow clone
ok 6 - fetch --unshallow from shallow clone
ok 7 - fetch something upstream has but hidden by clients shallow boundaries
ok 8 - fetch that requires changes in .git/shallow is filtered
ok 9 - fetch --update-shallow
error: Can't use skip_all after running some tests
Makefile:43: recipe for target 't5537-fetch-shallow.sh' failed
make[3]: *** [t5537-fetch-shallow.sh] Error 1

since this is trying to run the httpd halfway through the test, which
will never work if it's skipping it. Moving the httpd sourcing to the
top of the test isn't going to work either, because that would skip
*everything*, when we want to skip only the httpd bits. Maybe splitting
the httpd bits into a separate test is best here? I'm not sure.

-- 
NULL && (void)

Re: git 2.0.0 PROFILE=BUILD check-phase problems with ./t5561-http-backend.sh; GIT_TEST_HTTPD=false problems with t5537-fetch-shallow.sh

[Jeff King]

On Thu, May 29, 2014 at 11:44:37PM +0100, Nix wrote:

> I observe test failures with git 2.0.0 which are attributable to the
> change to run network tests by default. I'm lumping them both together
> into one report because I'm lazy and I've blown too much time on this
> already.

Weird. I also see a strange failure on t5310 when building with
PROFILE=BUILD. We get a segfault when reading jgit-produced bitmaps.
Tracking it down, we're getting inexplicably bogus data from an mmap'd
file (!). Compiling without PROFILE=BUILD, the test passes fine (even
with valgrind).

If I instrument it like this:

diff --git a/ewah/ewah_io.c b/ewah/ewah_io.c
index f7f700e..8cafacf 100644
--- a/ewah/ewah_io.c
+++ b/ewah/ewah_io.c
@@ -119,6 +119,7 @@ int ewah_read_mmap(struct ewah_bitmap *self, void *map, size_t len)
 	ptr += sizeof(uint32_t);
 
 	self->buffer_size = self->alloc_size = get_be32(ptr);
+	warning("got buffer_size of %lu from %lu", self->buffer_size, *(uint32_t *)ptr);
 	ptr += sizeof(uint32_t);
 
 	self->buffer = ewah_realloc(self->buffer,

a regular test-run reads:

  warning: got buffer_size of 2 from 33554432
  warning: got buffer_size of 2 from 33554432
  warning: got buffer_size of 3 from 50331648
  warning: got buffer_size of 1 from 16777216
  warning: got buffer_size of 2 from 33554432

and a PROFILE=BUILD one reads:

  warning: got buffer_size of 2 from 33554432
  warning: got buffer_size of 2 from 33554432
  warning: got buffer_size of 3 from 50331648
  warning: got buffer_size of 1 from 16777216
  warning: got buffer_size of 131072 from 512

I'm willing to believe we're doing something weird that violates the C
standard there, but I really can't see it.

Anyway, that's a side note to your problem...

> It appears that the Apache daemon is writing to the log slowly enough
> that its log lines only get there after the testsuite has written its
> separator, so a bunch of log lines appear to be attached to the wrong
> test, and the comparison fails.

Yeah, that looks to me like what is happening, too. If I put a 'sleep
1' into log_div, it passes. I would think apache would write the log
before serving the file, but perhaps not. And like you, I would expect
gcov to make things slower, not faster. Could there be something in the
environment that

I'm not sure what the best fix is. We could check the logfiles after
each test instead of at the end, but that will just end up with the same
race: we may check them before apache has written them.

> Attempting to work around this by building with GIT_TEST_HTTPD=false
> doesn't work either:
> 
> *** t5537-fetch-shallow.sh ***
> ok 1 - setup
> ok 2 - setup shallow clone
> ok 3 - clone from shallow clone
> ok 4 - fetch from shallow clone
> ok 5 - fetch --depth from shallow clone
> ok 6 - fetch --unshallow from shallow clone
> ok 7 - fetch something upstream has but hidden by clients shallow boundaries
> ok 8 - fetch that requires changes in .git/shallow is filtered
> ok 9 - fetch --update-shallow
> error: Can't use skip_all after running some tests
> Makefile:43: recipe for target 't5537-fetch-shallow.sh' failed
> make[3]: *** [t5537-fetch-shallow.sh] Error 1

Hrm. This already came up, and we dealt with it in 0232852 (t5537: move
http tests out to t5539, 2014-02-13). Which is in v2.0.0.

But somehow the code is back in v2.0.0 Presumably this is the result of a
mis-merge. I'll send a patch.

-Peff

[PATCH] t5537: re-drop http tests

[Jeff King]

These were originally removed by 0232852 (t5537: move
http tests out to t5539, 2014-02-13). However, they were
accidentally re-added in 1ddb4d7 (Merge branch
'nd/upload-pack-shallow', 2014-03-21).

This looks like an error in manual conflict resolution.
Here's what happened:

  1. v1.9.0 shipped with the http tests in t5537.

  2. We realized that this caused problems, and built
     0232852 on top to move the tests to their own file.
     This fix made it into v1.9.1.

  3. We later had another fix in nd/upload-pack-shallow that
     also touched t5537. It was built directly on v1.9.0.

When we merged nd/upload-pack-shallow to master, we got a
conflict; it was built on a version with the http tests, but
we had since removed them. The correct resolution was to
drop the http tests and keep the new ones, but instead we
kept everything.

Signed-off-by: Jeff King <peff@peff.net>
---
This is a regression in running "make test" in v2.0.0 for people who do
not have apache installed, so it would be nice to hit maint for 2.0.1.

I had a very hard time finding the merge to blame, because the
combined-diff code is convinced this isn't an interesting hunk (because
we kept no content from the other side, as there was none to keep!). I
eventually tracked it down with "git log --first-parent -m t/t5537*".

And a final side note. If you retry the merge by:

  m=1ddb4d7
  git checkout $m^1
  git merge $m^2

you can see the resulting conflict is quite tricky to understand. You
can tell from the blank "ours" side that _something_ was removed
(otherwise there would not have been a conflict), but there is no way to
tell what, or that the same something is part of the "theirs" side.
Viewing it with diff3 conflict-style makes it much more obvious.

 t/t5537-fetch-shallow.sh | 28 ----------------------------
 1 file changed, 28 deletions(-)

diff --git a/t/t5537-fetch-shallow.sh b/t/t5537-fetch-shallow.sh
index be951a4..a980574 100755
--- a/t/t5537-fetch-shallow.sh
+++ b/t/t5537-fetch-shallow.sh
@@ -173,33 +173,6 @@ EOF
 	)
 '
 
-if test -n "$NO_CURL" -o -z "$GIT_TEST_HTTPD"; then
-	say 'skipping remaining tests, git built without http support'
-	test_done
-fi
-
-LIB_HTTPD_PORT=${LIB_HTTPD_PORT-'5537'}
-. "$TEST_DIRECTORY"/lib-httpd.sh
-start_httpd
-
-test_expect_success 'clone http repository' '
-	git clone --bare --no-local shallow "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
-	git clone $HTTPD_URL/smart/repo.git clone &&
-	(
-	cd clone &&
-	git fsck &&
-	git log --format=%s origin/master >actual &&
-	cat <<EOF >expect &&
-7
-6
-5
-4
-3
-EOF
-	test_cmp expect actual
-	)
-'
-
 test_expect_success POSIXPERM,SANITY 'shallow fetch from a read-only repo' '
 	cp -R .git read-only.git &&
 	find read-only.git -print | xargs chmod -w &&
@@ -213,5 +186,4 @@ EOF
 	test_cmp expect actual
 '
 
-stop_httpd
 test_done
-- 
2.0.0.rc1.436.g03cb729

Re: git 2.0.0 PROFILE=BUILD check-phase problems with ./t5561-http-backend.sh; GIT_TEST_HTTPD=false problems with t5537-fetch-shallow.sh

[Nix]

On 30 May 2014, Jeff King told this:

> On Thu, May 29, 2014 at 11:44:37PM +0100, Nix wrote:
>
>> I observe test failures with git 2.0.0 which are attributable to the
>> change to run network tests by default. I'm lumping them both together
>> into one report because I'm lazy and I've blown too much time on this
>> already.
>
> Weird. I also see a strange failure on t5310 when building with
> PROFILE=BUILD. We get a segfault when reading jgit-produced bitmaps.

I don't see that, but this may be a compiler difference, or
environmental difference, or something.

> Tracking it down, we're getting inexplicably bogus data from an mmap'd
> file (!). Compiling without PROFILE=BUILD, the test passes fine (even
> with valgrind).
>
> If I instrument it like this:
>
> diff --git a/ewah/ewah_io.c b/ewah/ewah_io.c
> index f7f700e..8cafacf 100644
> --- a/ewah/ewah_io.c
> +++ b/ewah/ewah_io.c
> @@ -119,6 +119,7 @@ int ewah_read_mmap(struct ewah_bitmap *self, void *map, size_t len)
>  	ptr += sizeof(uint32_t);
>  
>  	self->buffer_size = self->alloc_size = get_be32(ptr);
> +	warning("got buffer_size of %lu from %lu", self->buffer_size, *(uint32_t *)ptr);
>  	ptr += sizeof(uint32_t);
>  
>  	self->buffer = ewah_realloc(self->buffer,
>
> a regular test-run reads:
>
>   warning: got buffer_size of 2 from 33554432
>   warning: got buffer_size of 2 from 33554432
>   warning: got buffer_size of 3 from 50331648
>   warning: got buffer_size of 1 from 16777216
>   warning: got buffer_size of 2 from 33554432
>
> and a PROFILE=BUILD one reads:
>
>   warning: got buffer_size of 2 from 33554432
>   warning: got buffer_size of 2 from 33554432
>   warning: got buffer_size of 3 from 50331648
>   warning: got buffer_size of 1 from 16777216
>   warning: got buffer_size of 131072 from 512
>
> I'm willing to believe we're doing something weird that violates the C
> standard there, but I really can't see it.

It's doubly strange because you're not building with -flto at all, all
callers of ewah_read_mmap() are in a separate file, and there are no
conditionals or loops above the warning() locus. Maybe the error is in a
different file, and is causing a nonsense 'len' to get passed in?

But I think this *may* be a false alarm. *(uint32_t *)ptr on a uint8_t
(in your logging line) is an aliasing violation, and the nearly-
uninitialized '512' is pretty much what I'd expect GCC to do now and
then, given that. Just a guess.

(I'm surprised this doesn't cause a problem *always*, though.)

>> It appears that the Apache daemon is writing to the log slowly enough
>> that its log lines only get there after the testsuite has written its
>> separator, so a bunch of log lines appear to be attached to the wrong
>> test, and the comparison fails.
>
> Yeah, that looks to me like what is happening, too. If I put a 'sleep
> 1' into log_div, it passes. I would think apache would write the log
> before serving the file, but perhaps not.

It does a straight write() via APR, which should work, I'd have thought.
This is on a tmpfs, not a network filesystem or anything.

(Aside: the flushing semantics are quite tangled, but shouldn't matter,
given that we're just doing a write(). Apache calls apr_file_flush() on
the logfile, but that's just flushing apr's analogue of stdio buffers:
it does nothing if the file is unbuffered -- and the error log file is
unbuffered. It does not call apr_file_sync(), but you shouldn't need a
sync() for one process to see another process's output.)

>                                           And like you, I would expect
> gcov to make things slower, not faster. Could there be something in the
> environment that

I'm more inclined to blame the kernel or the compiler, but I *always*
blame the kernel or the compiler. :P

> I'm not sure what the best fix is. We could check the logfiles after
> each test instead of at the end, but that will just end up with the same
> race: we may check them before apache has written them.

Yeah, that seems less than useful.

-- 
NULL && (void)

[RFC PATCH] t5538: move http push tests out to t5542

[Nix]

As 0232852b, but for the push tests instead: this avoids a start_httpd
in the middle of the file, which fails under GIT_TEST_HTTPD=false.

Signed-off-by: Nick Alcock <nick.alcock@oracle.com>
---
On 30 May 2014, Jeff King said:
> diff --git a/t/t5537-fetch-shallow.sh b/t/t5537-fetch-shallow.sh
> index be951a4..a980574 100755
> --- a/t/t5537-fetch-shallow.sh
> +++ b/t/t5537-fetch-shallow.sh

btw, t5538-push-shallow.sh has the same bug but was not fixed up that I
can see. (Though I can't see the original fix in the history of master
either: was it squashed away somehow?)

This new test fails if the patch is applied as-is because I haven't done
any ref adjustment (I don't honestly know what results are expected, and
the actual results I'm seeing, with a ref named '1s0' intermittently
replacing either 1 or 10 and 1 both, are so bizarre and inconsistent
that I suspect I'm hitting another local config problem of some sort.)

But something like this is needed.

(signed off with my work email address for paranoia's sake.)

 t/t5538-push-shallow.sh      |  59 -------------------------
 t/t5542-push-http-shallow.sh | 101 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 101 insertions(+), 59 deletions(-)
 create mode 100755 t/t5542-push-http-shallow.sh

diff --git a/t/t5538-push-shallow.sh b/t/t5538-push-shallow.sh
index 8e54ac5..ceee95b 100755
--- a/t/t5538-push-shallow.sh
+++ b/t/t5538-push-shallow.sh
@@ -120,63 +120,4 @@ EOF
 	git cat-file blob `echo 1|git hash-object --stdin` >/dev/null
 	)
 '
-
-if test -n "$NO_CURL" -o -z "$GIT_TEST_HTTPD"; then
-	say 'skipping remaining tests, git built without http support'
-	test_done
-fi
-
-. "$TEST_DIRECTORY"/lib-httpd.sh
-start_httpd
-
-test_expect_success 'push to shallow repo via http' '
-	git clone --bare --no-local shallow "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
-	(
-	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
-	git config http.receivepack true
-	) &&
-	(
-	cd full &&
-	commit 9 &&
-	git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master
-	) &&
-	(
-	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
-	git fsck &&
-	git log --format=%s top/master >actual &&
-	cat <<EOF >expect &&
-9
-4
-3
-EOF
-	test_cmp expect actual
-	)
-'
-
-test_expect_success 'push from shallow repo via http' '
-	mv "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" shallow-upstream.git &&
-	git clone --bare --no-local full "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
-	(
-	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
-	git config http.receivepack true
-	) &&
-	commit 10 &&
-	git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master &&
-	(
-	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
-	git fsck &&
-	git log --format=%s top/master >actual &&
-	cat <<EOF >expect &&
-10
-1
-4
-3
-2
-1
-EOF
-	test_cmp expect actual
-	)
-'
-
-stop_httpd
 test_done
diff --git a/t/t5542-push-http-shallow.sh b/t/t5542-push-http-shallow.sh
new file mode 100755
index 0000000..61a7281
--- /dev/null
+++ b/t/t5542-push-http-shallow.sh
@@ -0,0 +1,101 @@
+#!/bin/sh
+
+test_description='push from/to a shallow clone over http'
+
+. ./test-lib.sh
+
+if test -n "$NO_CURL" -o -z "$GIT_TEST_HTTPD"; then
+	say 'skipping test, git built without http support'
+	test_done
+fi
+
+. "$TEST_DIRECTORY"/lib-httpd.sh
+start_httpd
+
+commit() {
+	echo "$1" >tracked &&
+	git add tracked &&
+	git commit -m "$1"
+}
+
+test_expect_success 'setup' '
+	git config --global transfer.fsckObjects true &&
+	commit 1 &&
+	commit 2 &&
+	commit 3 &&
+	commit 4 &&
+	git clone . full &&
+	(
+	git init full-abc &&
+	cd full-abc &&
+	commit a &&
+	commit b &&
+	commit c
+	) &&
+	git clone --no-local --depth=2 .git shallow &&
+	git --git-dir=shallow/.git log --format=%s >actual &&
+	cat <<EOF >expect &&
+4
+3
+EOF
+	test_cmp expect actual &&
+	git clone --no-local --depth=2 full-abc/.git shallow2 &&
+	git --git-dir=shallow2/.git log --format=%s >actual &&
+	cat <<EOF >expect &&
+c
+b
+EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'push to shallow repo via http' '
+	git clone --bare --no-local shallow "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+	(
+	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+	git config http.receivepack true
+	) &&
+	(
+	cd full &&
+	commit 9 &&
+	git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master
+	) &&
+	(
+	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+	git fsck &&
+	git log --format=%s top/master >actual &&
+	cat <<EOF >expect &&
+9
+4
+3
+EOF
+	test_cmp expect actual
+	)
+'
+
+test_expect_success 'push from shallow repo via http' '
+	mv "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" shallow-upstream.git &&
+	git clone --bare --no-local full "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+	(
+	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+	git config http.receivepack true
+	) &&
+	commit 1s0 &&
+	git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master &&
+	(
+	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+	git fsck &&
+	git log --format=%s top/master >actual &&
+	cat <<EOF >expect &&
+10
+1
+4
+3
+2
+1
+EOF
+	test_cmp expect actual
+	)
+'
+
+stop_httpd
+test_done
-- 
1.9.2.172.gbe1f7c7

Re: [PATCH] t5537: re-drop http tests

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> And a final side note. If you retry the merge by:
>
>   m=1ddb4d7
>   git checkout $m^1
>   git merge $m^2
>
> you can see the resulting conflict is quite tricky to understand....
> Viewing it with diff3 conflict-style makes it much more obvious.

Yes, that was a clear mismerge on my side; thanks for catching it.

>
>  t/t5537-fetch-shallow.sh | 28 ----------------------------
>  1 file changed, 28 deletions(-)
>
> diff --git a/t/t5537-fetch-shallow.sh b/t/t5537-fetch-shallow.sh
> index be951a4..a980574 100755
> --- a/t/t5537-fetch-shallow.sh
> +++ b/t/t5537-fetch-shallow.sh
> @@ -173,33 +173,6 @@ EOF
>  	)
>  '
>  
> -if test -n "$NO_CURL" -o -z "$GIT_TEST_HTTPD"; then
> -	say 'skipping remaining tests, git built without http support'
> -	test_done
> -fi
> -
> -LIB_HTTPD_PORT=${LIB_HTTPD_PORT-'5537'}
> -. "$TEST_DIRECTORY"/lib-httpd.sh
> -start_httpd
> -
> -test_expect_success 'clone http repository' '
> -	git clone --bare --no-local shallow "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
> -	git clone $HTTPD_URL/smart/repo.git clone &&
> -	(
> -	cd clone &&
> -	git fsck &&
> -	git log --format=%s origin/master >actual &&
> -	cat <<EOF >expect &&
> -7
> -6
> -5
> -4
> -3
> -EOF
> -	test_cmp expect actual
> -	)
> -'
> -
>  test_expect_success POSIXPERM,SANITY 'shallow fetch from a read-only repo' '
>  	cp -R .git read-only.git &&
>  	find read-only.git -print | xargs chmod -w &&
> @@ -213,5 +186,4 @@ EOF
>  	test_cmp expect actual
>  '
>  
> -stop_httpd
>  test_done

Re: [RFC PATCH] t5538: move http push tests out to t5542

[Jeff King]

On Fri, May 30, 2014 at 11:29:10AM +0100, Nix wrote:

> As 0232852b, but for the push tests instead: this avoids a start_httpd
> in the middle of the file, which fails under GIT_TEST_HTTPD=false.
> 
> Signed-off-by: Nick Alcock <nick.alcock@oracle.com>

Thanks, this makes sense. I'm really not sure why we didn't notice this
at the same time as t5537.

I'm cc-ing Duy, who wrote the original test, to give input on the new
test setup.

> ---
> On 30 May 2014, Jeff King said:
> > diff --git a/t/t5537-fetch-shallow.sh b/t/t5537-fetch-shallow.sh
> > index be951a4..a980574 100755
> > --- a/t/t5537-fetch-shallow.sh
> > +++ b/t/t5537-fetch-shallow.sh
> 
> btw, t5538-push-shallow.sh has the same bug but was not fixed up that I
> can see. (Though I can't see the original fix in the history of master
> either: was it squashed away somehow?)

I don't think so. As far as I can tell, it was never fixed.

> +test_description='push from/to a shallow clone over http'
> +
> +. ./test-lib.sh
> +
> +if test -n "$NO_CURL" -o -z "$GIT_TEST_HTTPD"; then
> +	say 'skipping test, git built without http support'
> +	test_done
> +fi

Ah, this may be why we missed it. We should leave the handling of
$GIT_TEST_HTTPD to lib-httpd.sh. I think the topic to refactor lib-httpd
setup and this shallow topic were developed in isolation, and we never
noticed the semantic conflict during the merge, because there's no
textual conflict.

So when I tested with a broken apache, we silently skipped this test
(but setting GIT_TEST_HTTPD=false does not trigger the above skip).

> +test_expect_success 'push from shallow repo via http' '
> +	mv "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" shallow-upstream.git &&
> +	git clone --bare --no-local full "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
> +	(
> +	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
> +	git config http.receivepack true
> +	) &&
> +	commit 1s0 &&

This is a typo, no? The original is just "commit 10".

With that fix, the test still fails:

    --- expect      2014-05-30 17:18:08.638496558 +0000
    +++ actual      2014-05-30 17:18:08.638496558 +0000
    @@ -1,5 +1,4 @@
     10
    -1
     4
     3
     2
    not ok 3 - push from shallow repo via http

I think we can just drop that "1" from the expected output, though. It
was created in the original by the "push from full to shallow" test (and
is rather confusing, anyway, as it shares a name with the original "1").

-Peff

Re: [RFC PATCH] t5538: move http push tests out to t5542

[Junio C Hamano]

Nix <nix@esperi.org.uk> writes:

> As 0232852b, but for the push tests instead: this avoids a start_httpd
> in the middle of the file, which fails under GIT_TEST_HTTPD=false.
>
> Signed-off-by: Nick Alcock <nick.alcock@oracle.com>
> ---
> On 30 May 2014, Jeff King said:
>> diff --git a/t/t5537-fetch-shallow.sh b/t/t5537-fetch-shallow.sh
>> index be951a4..a980574 100755
>> --- a/t/t5537-fetch-shallow.sh
>> +++ b/t/t5537-fetch-shallow.sh
>
> btw, t5538-push-shallow.sh has the same bug but was not fixed up that I
> can see. (Though I can't see the original fix in the history of master
> either: was it squashed away somehow?)

What do you mean by "the original fix"?

0232852b (t5537: move http tests out to t5539, 2014-02-13) you cited
in your proposed log message?  That is v1.9.1~12^2 and certainly is
reachable from the master branch as master~185^2.

> But something like this is needed.

Thanks for marking the patch not to be applied / still broken.  I
agree that moving http bits into a separate test would be a good way
to go.

> (signed off with my work email address for paranoia's sake.)

As long as your employer is fine with your patch contributed to us,
and assuming that the patch was done on your employer's time, it
would be the right thing to do ;-).  Just FYI, if your From: address
on your e-mail does not match that address, you can start your
message body with "From: Nick Alcock <nick.alcock@oracle.com>", a
blank line and then the true body of your message.

>  t/t5538-push-shallow.sh      |  59 -------------------------
>  t/t5542-push-http-shallow.sh | 101 +++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 101 insertions(+), 59 deletions(-)
>  create mode 100755 t/t5542-push-http-shallow.sh
>
> diff --git a/t/t5538-push-shallow.sh b/t/t5538-push-shallow.sh
> index 8e54ac5..ceee95b 100755
> --- a/t/t5538-push-shallow.sh
> +++ b/t/t5538-push-shallow.sh
> @@ -120,63 +120,4 @@ EOF
>  	git cat-file blob `echo 1|git hash-object --stdin` >/dev/null
>  	)
>  '
> -
> -if test -n "$NO_CURL" -o -z "$GIT_TEST_HTTPD"; then
> -	say 'skipping remaining tests, git built without http support'
> -	test_done
> -fi
> -
> -. "$TEST_DIRECTORY"/lib-httpd.sh
> -start_httpd
> -
> -test_expect_success 'push to shallow repo via http' '
> -	git clone --bare --no-local shallow "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
> -	(
> -	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
> -	git config http.receivepack true
> -	) &&
> -	(
> -	cd full &&
> -	commit 9 &&
> -	git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master
> -	) &&
> -	(
> -	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
> -	git fsck &&
> -	git log --format=%s top/master >actual &&
> -	cat <<EOF >expect &&
> -9
> -4
> -3
> -EOF
> -	test_cmp expect actual
> -	)
> -'
> -
> -test_expect_success 'push from shallow repo via http' '
> -	mv "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" shallow-upstream.git &&
> -	git clone --bare --no-local full "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
> -	(
> -	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
> -	git config http.receivepack true
> -	) &&
> -	commit 10 &&
> -	git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master &&
> -	(
> -	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
> -	git fsck &&
> -	git log --format=%s top/master >actual &&
> -	cat <<EOF >expect &&
> -10
> -1
> -4
> -3
> -2
> -1
> -EOF
> -	test_cmp expect actual
> -	)
> -'
> -
> -stop_httpd
>  test_done
> diff --git a/t/t5542-push-http-shallow.sh b/t/t5542-push-http-shallow.sh
> new file mode 100755
> index 0000000..61a7281
> --- /dev/null
> +++ b/t/t5542-push-http-shallow.sh
> @@ -0,0 +1,101 @@
> +#!/bin/sh
> +
> +test_description='push from/to a shallow clone over http'
> +
> +. ./test-lib.sh
> +
> +if test -n "$NO_CURL" -o -z "$GIT_TEST_HTTPD"; then
> +	say 'skipping test, git built without http support'
> +	test_done
> +fi
> +
> +. "$TEST_DIRECTORY"/lib-httpd.sh
> +start_httpd
> +
> +commit() {
> +	echo "$1" >tracked &&
> +	git add tracked &&
> +	git commit -m "$1"
> +}
> +
> +test_expect_success 'setup' '
> +	git config --global transfer.fsckObjects true &&
> +	commit 1 &&
> +	commit 2 &&
> +	commit 3 &&
> +	commit 4 &&
> +	git clone . full &&
> +	(
> +	git init full-abc &&
> +	cd full-abc &&
> +	commit a &&
> +	commit b &&
> +	commit c
> +	) &&
> +	git clone --no-local --depth=2 .git shallow &&
> +	git --git-dir=shallow/.git log --format=%s >actual &&
> +	cat <<EOF >expect &&
> +4
> +3
> +EOF
> +	test_cmp expect actual &&
> +	git clone --no-local --depth=2 full-abc/.git shallow2 &&
> +	git --git-dir=shallow2/.git log --format=%s >actual &&
> +	cat <<EOF >expect &&
> +c
> +b
> +EOF
> +	test_cmp expect actual
> +'
> +
> +test_expect_success 'push to shallow repo via http' '
> +	git clone --bare --no-local shallow "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
> +	(
> +	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
> +	git config http.receivepack true
> +	) &&
> +	(
> +	cd full &&
> +	commit 9 &&
> +	git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master
> +	) &&
> +	(
> +	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
> +	git fsck &&
> +	git log --format=%s top/master >actual &&
> +	cat <<EOF >expect &&
> +9
> +4
> +3
> +EOF
> +	test_cmp expect actual
> +	)
> +'
> +
> +test_expect_success 'push from shallow repo via http' '
> +	mv "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" shallow-upstream.git &&
> +	git clone --bare --no-local full "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
> +	(
> +	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
> +	git config http.receivepack true
> +	) &&
> +	commit 1s0 &&
> +	git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master &&
> +	(
> +	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
> +	git fsck &&
> +	git log --format=%s top/master >actual &&
> +	cat <<EOF >expect &&
> +10
> +1
> +4
> +3
> +2
> +1
> +EOF
> +	test_cmp expect actual
> +	)
> +'
> +
> +stop_httpd
> +test_done

[PATCH v2] t5538: move http push tests out to t5542

[Jeff King]

From: Nick Alcock <nick.alcock@oracle.com>

As 0232852b, but for the push tests instead: this avoids a start_httpd
in the middle of the file, which fails under GIT_TEST_HTTPD=false.

Note that we have to munge the test in a few ways while
moving it:

  1. We drop the `test -z "$GIT_TEST_HTTPD"` check; this is
     too simplistic since 83d842d, and we should let
     lib-httpd.sh handle it.

  2. We have to port over some of the old setup from t5538.

  3. In the final test, we no longer expect the extra commit
     "1" built on top of "4". This was a side effect from an
     earlier test in t5538 which was not ported over.

Signed-off-by: Nick Alcock <nick.alcock@oracle.com>
Signed-off-by: Jeff King <peff@peff.net>
---
Here it is with the fixups I suggested in the last email. This passes
for me, but I'd love to have a sanity check from Duy.

 t/t5538-push-shallow.sh      |  59 -------------------------
 t/t5542-push-http-shallow.sh | 100 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 100 insertions(+), 59 deletions(-)
 create mode 100755 t/t5542-push-http-shallow.sh

diff --git a/t/t5538-push-shallow.sh b/t/t5538-push-shallow.sh
index 8e54ac5..ceee95b 100755
--- a/t/t5538-push-shallow.sh
+++ b/t/t5538-push-shallow.sh
@@ -120,63 +120,4 @@ EOF
 	git cat-file blob `echo 1|git hash-object --stdin` >/dev/null
 	)
 '
-
-if test -n "$NO_CURL" -o -z "$GIT_TEST_HTTPD"; then
-	say 'skipping remaining tests, git built without http support'
-	test_done
-fi
-
-. "$TEST_DIRECTORY"/lib-httpd.sh
-start_httpd
-
-test_expect_success 'push to shallow repo via http' '
-	git clone --bare --no-local shallow "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
-	(
-	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
-	git config http.receivepack true
-	) &&
-	(
-	cd full &&
-	commit 9 &&
-	git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master
-	) &&
-	(
-	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
-	git fsck &&
-	git log --format=%s top/master >actual &&
-	cat <<EOF >expect &&
-9
-4
-3
-EOF
-	test_cmp expect actual
-	)
-'
-
-test_expect_success 'push from shallow repo via http' '
-	mv "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" shallow-upstream.git &&
-	git clone --bare --no-local full "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
-	(
-	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
-	git config http.receivepack true
-	) &&
-	commit 10 &&
-	git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master &&
-	(
-	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
-	git fsck &&
-	git log --format=%s top/master >actual &&
-	cat <<EOF >expect &&
-10
-1
-4
-3
-2
-1
-EOF
-	test_cmp expect actual
-	)
-'
-
-stop_httpd
 test_done
diff --git a/t/t5542-push-http-shallow.sh b/t/t5542-push-http-shallow.sh
new file mode 100755
index 0000000..2a691e0
--- /dev/null
+++ b/t/t5542-push-http-shallow.sh
@@ -0,0 +1,100 @@
+#!/bin/sh
+
+test_description='push from/to a shallow clone over http'
+
+. ./test-lib.sh
+
+if test -n "$NO_CURL"; then
+	say 'skipping test, git built without http support'
+	test_done
+fi
+
+. "$TEST_DIRECTORY"/lib-httpd.sh
+start_httpd
+
+commit() {
+	echo "$1" >tracked &&
+	git add tracked &&
+	git commit -m "$1"
+}
+
+test_expect_success 'setup' '
+	git config --global transfer.fsckObjects true &&
+	commit 1 &&
+	commit 2 &&
+	commit 3 &&
+	commit 4 &&
+	git clone . full &&
+	(
+	git init full-abc &&
+	cd full-abc &&
+	commit a &&
+	commit b &&
+	commit c
+	) &&
+	git clone --no-local --depth=2 .git shallow &&
+	git --git-dir=shallow/.git log --format=%s >actual &&
+	cat <<EOF >expect &&
+4
+3
+EOF
+	test_cmp expect actual &&
+	git clone --no-local --depth=2 full-abc/.git shallow2 &&
+	git --git-dir=shallow2/.git log --format=%s >actual &&
+	cat <<EOF >expect &&
+c
+b
+EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'push to shallow repo via http' '
+	git clone --bare --no-local shallow "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+	(
+	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+	git config http.receivepack true
+	) &&
+	(
+	cd full &&
+	commit 9 &&
+	git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master
+	) &&
+	(
+	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+	git fsck &&
+	git log --format=%s top/master >actual &&
+	cat <<EOF >expect &&
+9
+4
+3
+EOF
+	test_cmp expect actual
+	)
+'
+
+test_expect_success 'push from shallow repo via http' '
+	mv "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" shallow-upstream.git &&
+	git clone --bare --no-local full "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+	(
+	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+	git config http.receivepack true
+	) &&
+	commit 10 &&
+	git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master &&
+	(
+	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+	git fsck &&
+	git log --format=%s top/master >actual &&
+	cat <<EOF >expect &&
+10
+4
+3
+2
+1
+EOF
+	test_cmp expect actual
+	)
+'
+
+stop_httpd
+test_done
-- 
2.0.0.rc1.436.g03cb729

Re: [RFC PATCH] t5538: move http push tests out to t5542

[Nix]

On 30 May 2014, Jeff King said:

> On Fri, May 30, 2014 at 11:29:10AM +0100, Nix wrote:
>
>> +test_expect_success 'push from shallow repo via http' '
>> +	mv "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" shallow-upstream.git &&
>> +	git clone --bare --no-local full "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
>> +	(
>> +	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
>> +	git config http.receivepack true
>> +	) &&
>> +	commit 1s0 &&
>
> This is a typo, no? The original is just "commit 10".

I have no *idea* where that came from. vi-wreckage I suspect.

> I think we can just drop that "1" from the expected output, though. It
> was created in the original by the "push from full to shallow" test (and
> is rather confusing, anyway, as it shares a name with the original "1").

Right.

-- 
NULL && (void)

Re: [RFC PATCH] t5538: move http push tests out to t5542

[Nix]

On 30 May 2014, Junio C. Hamano said:

> Nix <nix@esperi.org.uk> writes:
>
>> (signed off with my work email address for paranoia's sake.)
>
> As long as your employer is fine with your patch contributed to us,

My employer encourages it.

> and assuming that the patch was done on your employer's time, it

That is hard to define, but I use this copy of git for work purposes as
well as non-work purposes, so I arbitrarily define it as done on their
time.

> would be the right thing to do ;-).  Just FYI, if your From: address
> on your e-mail does not match that address, you can start your
> message body with "From: Nick Alcock <nick.alcock@oracle.com>", a
> blank line and then the true body of your message.

Yeah, I oscillated on that and picked the wrong option in the end. :)

Re: [PATCH v2] t5538: move http push tests out to t5542

[Duy Nguyen]

On Sat, May 31, 2014 at 12:36 AM, Jeff King <peff@peff.net> wrote:
> From: Nick Alcock <nick.alcock@oracle.com>
>
> As 0232852b, but for the push tests instead: this avoids a start_httpd
> in the middle of the file, which fails under GIT_TEST_HTTPD=false.
>
> Note that we have to munge the test in a few ways while
> moving it:
>
>   1. We drop the `test -z "$GIT_TEST_HTTPD"` check; this is
>      too simplistic since 83d842d, and we should let
>      lib-httpd.sh handle it.
>
>   2. We have to port over some of the old setup from t5538.
>
>   3. In the final test, we no longer expect the extra commit
>      "1" built on top of "4". This was a side effect from an
>      earlier test in t5538 which was not ported over.
>
> Signed-off-by: Nick Alcock <nick.alcock@oracle.com>
> Signed-off-by: Jeff King <peff@peff.net>
> ---
> Here it is with the fixups I suggested in the last email. This passes
> for me, but I'd love to have a sanity check from Duy.

Looks good.
-- 
Duy