From: Phillip Wood <phillip.wood123@gmail.com>
To: Johannes Schindelin via GitGitGadget <gitgitgadget@gmail.com>,
git@vger.kernel.org
Cc: Johannes Schindelin <johannes.schindelin@gmx.de>
Subject: Re: [PATCH 4/9] ci/run-build-and-tests: add some structure to the GitHub workflow output
Date: Wed, 23 Feb 2022 12:13:51 +0000 [thread overview]
Message-ID: <f73586a9-e9f1-aa1d-ac22-8102e6583e2b@gmail.com> (raw)
In-Reply-To: <9333ba781b8240f704e739b00d274f8c3d887e39.1643050574.git.gitgitgadget@gmail.com>
Hi Dscho
On 24/01/2022 18:56, Johannes Schindelin via GitGitGadget wrote:
> From: Johannes Schindelin <johannes.schindelin@gmx.de>
>
> The current output of Git's GitHub workflow can be quite confusing,
> especially for contributors new to the project.
>
> To make it more helpful, let's introduce some collapsible grouping.
> Initially, readers will see the high-level view of what actually
> happened (did the build fail, or the test suite?). To drill down, the
> respective group can be expanded.
>
> Note: sadly, workflow output currently cannot contain any nested groups
> (see https://github.com/actions/runner/issues/802 for details),
> therefore we take pains to ensure to end any previous group before
> starting a new one.
Thanks for working on this, I find it makes it much easier to get to the
information I need when a test fails. I'm not familiar with github's
grouping but I noticed something below I didn't understand.
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> ci/lib.sh | 55 ++++++++++++++++++++++++++++++++++-----
> ci/run-build-and-tests.sh | 4 +--
> ci/run-test-slice.sh | 2 +-
> 3 files changed, 51 insertions(+), 10 deletions(-)
>
> diff --git a/ci/lib.sh b/ci/lib.sh
> index 2b2c0932320..4ed8f40ab02 100755
> --- a/ci/lib.sh
> +++ b/ci/lib.sh
> @@ -1,5 +1,49 @@
> # Library of functions shared by all CI scripts
>
> +if test true != "$GITHUB_ACTIONS"
> +then
> + begin_group () { :; }
> + end_group () { :; }
> +
> + group () {
> + shift
> + "$@"
> + }
> + set -x
> +else
> + begin_group () {
> + need_to_end_group=t
> + echo "::group::$1" >&2
> + set -x
> + }
> +
> + end_group () {
> + test -n "$need_to_end_group" || return 0
> + set +x
> + need_to_end_group=
> + echo '::endgroup::' >&2
> + }
> + trap end_group EXIT
> +
> + group () {
> + set +x
> + begin_group "$1"
> + shift
> + "$@"
> + res=$?
> + end_group
> + return $res
> + }
> +
> + begin_group "CI setup"
> +fi
> +
> +# Set 'exit on error' for all CI scripts to let the caller know that
> +# something went wrong.
> +# Set tracing executed commands, primarily setting environment variables
> +# and installing dependencies.
> +set -e
The comment is moved unchanged but the set command has lost the "-x". We
now have several "set -x" commands in the functions above and one below
"end_group" lower down. Does the comment need updating as we are not
enabling the tracing of executed commands here anymore?
Best Wishes
Phillip
> skip_branch_tip_with_tag () {
> # Sometimes, a branch is pushed at the same time the tag that points
> # at the same commit as the tip of the branch is pushed, and building
> @@ -88,12 +132,6 @@ export TERM=${TERM:-dumb}
> # Clear MAKEFLAGS that may come from the outside world.
> export MAKEFLAGS=
>
> -# Set 'exit on error' for all CI scripts to let the caller know that
> -# something went wrong.
> -# Set tracing executed commands, primarily setting environment variables
> -# and installing dependencies.
> -set -ex
> -
> if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
> then
> CI_TYPE=azure-pipelines
> @@ -138,7 +176,7 @@ then
> test_name="${test_exit%.exit}"
> test_name="${test_name##*/}"
> printf "\\e[33m\\e[1m=== Failed test: ${test_name} ===\\e[m\\n"
> - cat "t/test-results/$test_name.out"
> + group "Failed test: $test_name" cat "t/test-results/$test_name.out"
>
> trash_dir="t/trash directory.$test_name"
> cp "t/test-results/$test_name.out" t/failed-test-artifacts/
> @@ -234,3 +272,6 @@ linux-leaks)
> esac
>
> MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}"
> +
> +end_group
> +set -x
> diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
> index e49f9eaa8c0..5516f45f7fe 100755
> --- a/ci/run-build-and-tests.sh
> +++ b/ci/run-build-and-tests.sh
> @@ -48,10 +48,10 @@ esac
> # Any new "test" targets should not go after this "make", but should
> # adjust $MAKE_TARGETS. Otherwise compilation-only targets above will
> # start running tests.
> -make
> +group Build make
> if test -n "$run_tests"
> then
> - make test ||
> + group "Run tests" make test ||
> handle_failed_tests
> fi
> check_unignored_build_artifacts
> diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
> index 63358c23e11..a3c67956a8d 100755
> --- a/ci/run-test-slice.sh
> +++ b/ci/run-test-slice.sh
> @@ -10,7 +10,7 @@ windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
> *) ln -s "$cache_dir/.prove" t/.prove;;
> esac
>
> -make --quiet -C t T="$(cd t &&
> +group "Run tests" make --quiet -C t T="$(cd t &&
> ./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
> tr '\n' ' ')" ||
> handle_failed_tests
next prev parent reply other threads:[~2022-02-23 12:14 UTC|newest]
Thread overview: 98+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-24 18:56 [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful Johannes Schindelin via GitGitGadget
2022-01-24 18:56 ` [PATCH 1/9] ci: fix code style Johannes Schindelin via GitGitGadget
2022-01-24 18:56 ` [PATCH 2/9] ci/run-build-and-tests: take a more high-level view Johannes Schindelin via GitGitGadget
2022-01-24 23:22 ` Eric Sunshine
2022-01-25 14:34 ` Johannes Schindelin
2022-01-24 18:56 ` [PATCH 3/9] ci: make it easier to find failed tests' logs in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-01-25 23:48 ` Ævar Arnfjörð Bjarmason
2022-01-24 18:56 ` [PATCH 4/9] ci/run-build-and-tests: add some structure to the GitHub workflow output Johannes Schindelin via GitGitGadget
2022-02-23 12:13 ` Phillip Wood [this message]
2022-02-25 13:40 ` Johannes Schindelin
2022-01-24 18:56 ` [PATCH 5/9] tests: refactor --write-junit-xml code Johannes Schindelin via GitGitGadget
2022-01-26 0:10 ` Ævar Arnfjörð Bjarmason
2022-01-24 18:56 ` [PATCH 6/9] test(junit): avoid line feeds in XML attributes Johannes Schindelin via GitGitGadget
2022-01-24 18:56 ` [PATCH 7/9] ci: optionally mark up output in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-01-24 18:56 ` [PATCH 8/9] ci: use `--github-workflow-markup` " Johannes Schindelin via GitGitGadget
2022-01-24 18:56 ` [PATCH 9/9] ci: call `finalize_test_case_output` a little later Johannes Schindelin via GitGitGadget
2022-01-26 0:25 ` [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful Ævar Arnfjörð Bjarmason
2022-01-27 16:31 ` CI "grouping" within jobs v.s. lighter split-out jobs (was: [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful) Ævar Arnfjörð Bjarmason
2022-02-19 23:46 ` [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful Johannes Schindelin
2022-02-20 2:44 ` Junio C Hamano
2022-02-20 15:25 ` Johannes Schindelin
2022-02-21 8:09 ` Ævar Arnfjörð Bjarmason
2022-02-22 10:26 ` Johannes Schindelin
2022-02-20 12:47 ` Ævar Arnfjörð Bjarmason
2022-02-22 10:30 ` Johannes Schindelin
2022-02-22 13:31 ` Ævar Arnfjörð Bjarmason
2022-02-23 12:07 ` Phillip Wood
2022-02-25 12:39 ` Ævar Arnfjörð Bjarmason
2022-02-25 14:10 ` Johannes Schindelin
2022-02-25 18:16 ` Junio C Hamano
2022-02-26 18:43 ` Junio C Hamano
2022-03-01 2:59 ` Junio C Hamano
2022-03-01 6:35 ` Junio C Hamano
2022-03-01 10:18 ` Johannes Schindelin
2022-03-01 16:52 ` Junio C Hamano
2022-03-01 10:10 ` Johannes Schindelin
2022-03-01 16:57 ` Junio C Hamano
2022-03-01 10:20 ` Johannes Schindelin
2022-03-04 7:38 ` win+VS environment has "cut" but not "paste"? Junio C Hamano
2022-03-04 9:04 ` Ævar Arnfjörð Bjarmason
2022-03-07 15:51 ` Johannes Schindelin
2022-03-07 17:05 ` Junio C Hamano
2022-03-09 13:02 ` Johannes Schindelin
2022-03-10 15:23 ` Ævar Arnfjörð Bjarmason
2022-03-07 15:48 ` Johannes Schindelin
2022-03-07 16:58 ` Junio C Hamano
2022-03-02 10:58 ` [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful Phillip Wood
2022-03-07 16:07 ` Johannes Schindelin
2022-03-07 17:11 ` Junio C Hamano
2022-03-09 11:44 ` Ævar Arnfjörð Bjarmason
2022-03-07 17:12 ` Phillip Wood
2022-03-01 10:24 ` [PATCH v2 " Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 1/9] ci: fix code style Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 2/9] ci/run-build-and-tests: take a more high-level view Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 3/9] ci: make it easier to find failed tests' logs in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 4/9] ci/run-build-and-tests: add some structure to the GitHub workflow output Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 5/9] tests: refactor --write-junit-xml code Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 6/9] test(junit): avoid line feeds in XML attributes Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 7/9] ci: optionally mark up output in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 8/9] ci: use `--github-workflow-markup` " Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 9/9] ci: call `finalize_test_case_output` a little later Johannes Schindelin via GitGitGadget
2022-03-01 19:07 ` [PATCH v2 0/9] ci: make Git's GitHub workflow output much more helpful Junio C Hamano
2022-03-02 12:22 ` Ævar Arnfjörð Bjarmason
2022-03-07 15:57 ` Johannes Schindelin
2022-03-07 16:05 ` Ævar Arnfjörð Bjarmason
2022-03-07 17:36 ` Junio C Hamano
2022-03-09 10:56 ` Ævar Arnfjörð Bjarmason
2022-03-09 13:20 ` Johannes Schindelin
2022-03-09 19:39 ` Junio C Hamano
2022-03-09 19:47 ` Ævar Arnfjörð Bjarmason
2022-03-25 0:48 ` Victoria Dye
2022-03-25 9:02 ` Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` Victoria Dye
2022-05-21 21:42 ` Johannes Schindelin
2022-05-21 23:05 ` Junio C Hamano
2022-05-22 18:48 ` Johannes Schindelin
2022-05-22 19:10 ` Junio C Hamano
2022-05-23 12:58 ` Johannes Schindelin
2022-05-22 23:27 ` Junio C Hamano
2022-05-23 18:55 ` Junio C Hamano
2022-05-23 19:21 ` Johannes Schindelin
2022-05-23 9:05 ` Ævar Arnfjörð Bjarmason
2022-05-23 18:41 ` Johannes Schindelin
2022-05-24 8:40 ` Ævar Arnfjörð Bjarmason
2022-05-21 22:18 ` [PATCH v3 00/12] " Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 01/12] ci: fix code style Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 02/12] tests: refactor --write-junit-xml code Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 03/12] test(junit): avoid line feeds in XML attributes Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 04/12] ci/run-build-and-tests: take a more high-level view Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 05/12] ci: make it easier to find failed tests' logs in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 06/12] ci/run-build-and-tests: add some structure to the GitHub workflow output Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 07/12] ci: optionally mark up output in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 08/12] ci(github): skip the logs of the successful test cases Johannes Schindelin via GitGitGadget
2022-05-24 10:47 ` Ævar Arnfjörð Bjarmason
2022-05-21 22:18 ` [PATCH v3 09/12] ci(github): avoid printing test case preamble twice Victoria Dye via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 10/12] ci: use `--github-workflow-markup` in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 11/12] ci(github): mention where the full logs can be found Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 12/12] ci: call `finalize_test_case_output` a little later Johannes Schindelin via GitGitGadget
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f73586a9-e9f1-aa1d-ac22-8102e6583e2b@gmail.com \
--to=phillip.wood123@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitgitgadget@gmail.com \
--cc=johannes.schindelin@gmx.de \
--cc=phillip.wood@dunelm.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).