Re: [PATCH v1 3/3] travis-ci: run Git bisect on failed tests

[Pranit Bauva <pranit.bauva@gmail.com>]

Hey Lars,

On Sun, May 22, 2016 at 4:30 PM,  <larsxschneider@gmail.com> wrote:
> From: Lars Schneider <larsxschneider@gmail.com>
>
> Junio usually pushes many commits at once to the public "pu"/"next"/
> "master" branches. If a test fails then it is not obvious what commit
> caused the failure. Therefore we run Git bisect with the merge base
> between the failing rev and its more stable branch ("next" for "pu",
> "master" for "next", and "maint" for "master") as good ref to find the
> offending commit. This is only enabled on "github.com/git/git" because
> there we can assume that all relevant branches are up to date.
>
> Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
> ---
>  .travis.yml       |  2 +-
>  ci/test-report.sh | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 66 insertions(+), 1 deletion(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index 81d2027..922807b 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -101,7 +101,7 @@ script: make --quiet test
>  after_failure:
>    - >
>      : '<-- Click here to see detailed test output!                                                        ';
> -    ./ci/test-report.sh
> +    ./ci/test-report.sh $TRAVIS_REPO_SLUG $TRAVIS_BRANCH;
>
>  notifications:
>    email: false
> diff --git a/ci/test-report.sh b/ci/test-report.sh
> index d08a999..8f7adad 100755
> --- a/ci/test-report.sh
> +++ b/ci/test-report.sh
> @@ -1,5 +1,12 @@
>  #!/bin/sh
>  #
> +# Print test results and run Git bisect on failed tests.
> +#
> +REPO_ORG_NAME=$1
> +CURRENT_BRANCH_NAME=$2
> +
> +
> +#
>  # Print test results
>  #
>  for TEST_EXIT in t/test-results/*.exit
> @@ -17,3 +24,61 @@ do
>         fi
>  done
>
> +
> +#
> +# Run Git bisect
> +#
> +run_bisect () {
> +       TEST_SCRIPT=$1
> +       BAD_REV=$2
> +       GOOD_RV=$3
> +       TMPDIR=$(mktemp -d -t "ci-report-bisect-XXXXXX" 2>/dev/null)
> +       cat > "$TMPDIR/bisect-run.sh" <<EOF

If you are doing a re-roll, then you could probably fix the style issue.
' cat >"$TMPDIR..."  '

> +
> +EOF
> +       chmod +x "$TMPDIR/bisect-run.sh"
> +       git bisect start $BAD_REV $GOOD_RV
> +       git bisect run "$TMPDIR/bisect-run.sh"
> +       if test -e ./t/$TEST_SCRIPT.sh && make --jobs=2 >/dev/null 2>&1
> +       then
> +               cd t && ./$TEST_SCRIPT.sh >/dev/null 2>&1
> +       else
> +               # If the test file does not exist or the build fails then tell
> +               # Git bisect to skip the commit.
> +               exit 125
> +       fi
> +       git bisect reset >/dev/null 2>&1
> +}
> +
> +case "$CURRENT_BRANCH_NAME" in
> +       master) STABLE_BRANCH="maint";;
> +       next)   STABLE_BRANCH="master";;
> +       pu)     STABLE_BRANCH="next";;
> +esac
> +
> +if test "$REPO_ORG_NAME" = "git/git" && test -n $STABLE_BRANCH
> +then
> +       BAD_REV=$(git rev-parse HEAD)
> +
> +       # Travis CI clones are shallow. It is possible that the last good revision
> +       # was not fetched, yet. Therefore we need to fetch all commits on the
> +       # stable branch.
> +       git config remote.origin.fetch "+refs/heads/$STABLE_BRANCH:refs/remotes/origin/$STABLE_BRANCH"
> +       git fetch --unshallow --quiet
> +       LAST_GOOD_REV=$(git merge-base $BAD_REV "remotes/origin/$STABLE_BRANCH")
> +
> +       for TEST_EXIT in t/test-results/*.exit
> +       do
> +               if test "$(cat "$TEST_EXIT")" != "0"
> +               then
> +                       TEST="${TEST_EXIT%.exit}"
> +                       TEST_SCRIPT=${TEST#t/test-results/}
> +                       echo "------------------------------------------------------------------------"
> +                       echo "  $(tput setaf 1)${TEST} Bisect$(tput sgr0)"
> +                       echo "------------------------------------------------------------------------"
> +                       run_bisect $TEST_SCRIPT $BAD_REV $LAST_GOOD_REV
> +                       echo ""
> +                       echo ""
> +               fi
> +       done
> +fi

Regards,
Pranit Bauva