[regression] `make profile-install` fails in 2.10.1

[regression] `make profile-install` fails in 2.10.1

[Jan Keromnes]

Hello,

TL;DR - Probably a regression of a previously reported bug. [0]

I'm trying to `profile-install` Git from source on Ubuntu 16.04, to
have the latest stable Git optimized for my machine.

However, this often fails, because the profile build runs all Git
tests (to get an idea of how Git behaves on my hardware, and optimize
for it), but it bails out if there are any test failures (for me, this
has happened on most Git version upgrades this year, see also [0] and
[1]).

- Problem: Is there a way to `make profile-install` but ignore
occasional test failures, as these are not critical to get a useful
hardware profile? (Note: In a previous thread, Dennis Kaarsemaker
mentioned this is fixing a symptom, not the root cause, but it would
still be great to get a working profile in spite of occasional test
failures.)

- Related problem: `t3700-add.sh` fails again in 2.10.1 for me. More
details below, and I can provide further debug information if you
don't already know the problem.

Thanks,
Jan

[0] "`make profile-install` fails in 2.9.3" -
https://marc.info/?l=git&m=147274608823171&w=2

[1] "Fwd: Git 2.8.1 fails test 32 of t7300-clean.sh, breaks profile
build" - https://marc.info/?l=git&m=146193443529229&w=2

---

Steps to reproduce:

    curl https://www.kernel.org/pub/software/scm/git/git-2.10.1.tar.xz
| tar xJ \
     && cd git-2.10.1 \
     && make prefix=/usr profile-install install-man -j18

Expected result:

    - runs all tests to get a profile (ignoring occasional failures)
    - rebuilds Git with the profile
    - installs Git

Actual result:

    - runs all tests to get a profile
    - at least one test fails, interrupting the whole process
    - Git is not installed

Failure log:

    (snip)
    *** t3700-add.sh ***
    ok 1 - Test of git add
    ok 2 - Post-check that foo is in the index
    ok 3 - Test that "git add -- -q" works
    ok 4 - git add: Test that executable bit is not used if core.filemode=0
    ok 5 - git add: filemode=0 should not get confused by symlink
    (snip)
    ok 38 - git add --chmod=[+-]x stages correctly
    ok 39 - git add --chmod=+x with symlinks
    not ok 40 - git add --chmod=[+-]x changes index with already added file
    #
    #               echo foo >foo3 &&
    #               git add foo3 &&
    #               git add --chmod=+x foo3 &&
    #               test_mode_in_index 100755 foo3 &&
    #               echo foo >xfoo3 &&
    #               chmod 755 xfoo3 &&
    #               git add xfoo3 &&
    #               git add --chmod=-x xfoo3 &&
    #               test_mode_in_index 100644 xfoo3
    #
    ok 41 - git add --chmod=[+-]x does not change the working tree
    ok 42 - no file status change if no pathspec is given
    ok 43 - no file status change if no pathspec is given in subdir
    ok 44 - all statuses changed in folder if . is given
    # failed 1 among 44 test(s)
    1..44
    Makefile:43: recipe for target 't3700-add.sh' failed
    make[3]: Leaving directory '/tmp/git/git-2.10.1/t'
    make[3]: *** [t3700-add.sh] Error 1
    Makefile:36: recipe for target 'test' failed
    make[2]: Leaving directory '/tmp/git/git-2.10.1/t'
    make[2]: *** [test] Error 2
    Makefile:2273: recipe for target 'test' failed
    make[1]: *** [test] Error 2
    make[1]: Leaving directory '/tmp/git/git-2.10.1'
    Makefile:1679: recipe for target 'profile' failed
    make: *** [profile] Error 2
    The command '/bin/sh -c mkdir /tmp/git  && cd /tmp/git  && curl
https://www.kernel.org/pub/software/scm/git/git-2.10.1.tar.xz | tar xJ
 && cd git-2.10.1  && make prefix=/usr profile-install install-man
-j18  && rm -rf /tmp/git' returned a non-zero code: 2

Re: [regression] `make profile-install` fails in 2.10.1

[Jeff King]

On Wed, Oct 19, 2016 at 01:15:56PM +0200, Jan Keromnes wrote:

> - Problem: Is there a way to `make profile-install` but ignore
> occasional test failures, as these are not critical to get a useful
> hardware profile? (Note: In a previous thread, Dennis Kaarsemaker
> mentioned this is fixing a symptom, not the root cause, but it would
> still be great to get a working profile in spite of occasional test
> failures.)

No, there isn't a way currently. I agree with Dennis that we do want to
ultimately fix the flaky tests, but at the same time, it does seem
reasonable to use a partial test run for your profile results.

I suspect you'd need to switch the call to:

  $(MAKE) PROFILE=GEN -j1 -k test || true

to make it best-effort.

> - Related problem: `t3700-add.sh` fails again in 2.10.1 for me. More
> details below, and I can provide further debug information if you
> don't already know the problem.

I can't reproduce any problems with raciness there, but there is a known
problem with running the script as root (which I guess you might be
doing from your "make prefix=/usr" call). There's some discussion in
http://public-inbox.org/git/20161010035756.38408-1-jeremyhu@apple.com/T/#u,
but it looks like the patch stalled.

-Peff

Re: [regression] `make profile-install` fails in 2.10.1

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> I can't reproduce any problems with raciness there, but there is a known
> problem with running the script as root (which I guess you might be
> doing from your "make prefix=/usr" call). There's some discussion in
> http://public-inbox.org/git/20161010035756.38408-1-jeremyhu@apple.com/T/#u,
> but it looks like the patch stalled.

Ouch.  Thanks for a reminder.  How about doing this for now?

-- >8 --
From: Junio C Hamano <gitster@pobox.com>
Date: Mon, 10 Oct 2016 10:41:51 -0700
Subject: [PATCH] t3700: fix broken test under !SANITY

An "add --chmod=+x" test recently added by 610d55af0f ("add: modify
already added files when --chmod is given", 2016-09-14) used "xfoo3"
as a test file.  The paths xfoo[1-3] were used by earlier tests for
symbolic links but they were expected to have been removed by the
test script reached this new test.

The removal with "git reset --hard" however happened in tests that
are protected by POSIXPERM,SANITY prerequisites.  Platforms and test
environments that lacked these would have seen xfoo3 as a leftover
symbolic link, pointing somewhere else, and chmod test would have
given a wrong result.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/t3700-add.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/t/t3700-add.sh b/t/t3700-add.sh
index 924a266126..53c0cb6dea 100755
--- a/t/t3700-add.sh
+++ b/t/t3700-add.sh
@@ -350,6 +350,7 @@ test_expect_success POSIXPERM,SYMLINKS 'git add --chmod=+x with symlinks' '
 '
 
 test_expect_success 'git add --chmod=[+-]x changes index with already added file' '
+	rm -f foo3 xfoo3 &&
 	echo foo >foo3 &&
 	git add foo3 &&
 	git add --chmod=+x foo3 &&
-- 
2.10.1-633-g7f0e449216

Re: [regression] `make profile-install` fails in 2.10.1

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Ouch.  Thanks for a reminder.  How about doing this for now?

And the hack I used to quickly test it looks like this:

    $ cd t
    $ GIT_I_AM_INSANE=Yes sh ./t3700-add.sh

We may want a more general 

    GIT_OVERRIDE_PREREQ='!SANITY,!POSIXPERM,MINGW' make test

or something like that, though.

 t/test-lib.sh | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/t/test-lib.sh b/t/test-lib.sh
index 0055ebba46..9c5bcd9d1d 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -89,6 +89,7 @@ unset VISUAL EMAIL LANGUAGE COLUMNS $("$PERL_PATH" -e '
 		UNZIP
 		PERF_
 		CURL_VERBOSE
+		I_AM_INSANE
 	));
 	my @vars = grep(/^GIT_/ && !/^GIT_($ok)/o, @env);
 	print join("\n", @vars);
@@ -1081,6 +1082,12 @@ test_lazy_prereq NOT_ROOT '
 # containing directory doesn't have read or execute permissions.
 
 test_lazy_prereq SANITY '
+
+	if test -n "$GIT_I_AM_INSANE"
+	then
+		return 1
+	fi &&
+
 	mkdir SANETESTD.1 SANETESTD.2 &&
 
 	chmod +w SANETESTD.1 SANETESTD.2 &&

Re: [regression] `make profile-install` fails in 2.10.1

[Jeff King]

On Wed, Oct 19, 2016 at 03:27:35PM -0700, Junio C Hamano wrote:

> Ouch.  Thanks for a reminder.  How about doing this for now?
> [...]
>  test_expect_success 'git add --chmod=[+-]x changes index with already added file' '
> +	rm -f foo3 xfoo3 &&
>  	echo foo >foo3 &&

Yeah, this makes sense. It does make me feel like the test should simply
be using xfoo27, or some name that is not otherwise used in the rest of
the script, but I don't mind doing the minimal thing.

-Peff

Re: [regression] `make profile-install` fails in 2.10.1

[Jeff King]

On Wed, Oct 19, 2016 at 03:30:44PM -0700, Junio C Hamano wrote:

> > Ouch.  Thanks for a reminder.  How about doing this for now?
> 
> And the hack I used to quickly test it looks like this:
> 
>     $ cd t
>     $ GIT_I_AM_INSANE=Yes sh ./t3700-add.sh
> 
> We may want a more general 
> 
>     GIT_OVERRIDE_PREREQ='!SANITY,!POSIXPERM,MINGW' make test
> 
> or something like that, though.

I don't think I've ever wanted to do that myself, but I can see how it
might be useful (e.g., claiming we don't support symlinks is another
one).

I usually just try to recreate the actual environment (e.g., run the
tests as root, run them on a loopback case-insensitive fs, etc) as that
gives a more realistic recreation.

-Peff

Re: [regression] `make profile-install` fails in 2.10.1

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> I usually just try to recreate the actual environment (e.g., run the
> tests as root, run them on a loopback case-insensitive fs, etc) as that
> gives a more realistic recreation.

True.  I just do not want to run the tests as root myself ;-)  I
wonder if fakeroot would give us good enough illusion for that--I
haven't used it for a long while.