git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Taylor Blau <me@ttaylorr.com>
To: Elijah Newren via GitGitGadget <gitgitgadget@gmail.com>
Cc: git@vger.kernel.org, Elijah Newren <newren@gmail.com>
Subject: Re: [PATCH 3/3] test-lib: reduce verbosity of skipped tests
Date: Tue, 13 Oct 2020 11:44:39 -0400	[thread overview]
Message-ID: <20201013154439.GB1913367@nand.local> (raw)
In-Reply-To: <bb2317972a8faf0358aaad1247fdfd2af2ea142d.1602545164.git.gitgitgadget@gmail.com>

On Mon, Oct 12, 2020 at 11:26:04PM +0000, Elijah Newren via GitGitGadget wrote:
> From: Elijah Newren <newren@gmail.com>
>
> When using the --run flag to run just two or three tests from a test
> file which contains several dozen tests, having every skipped test print
> out dozens of lines of output for the test code for that skipped test
> adds up to hundreds or thousands of lines of irrelevant output that make
> it very hard to fish out the relevant results you were looking for.
> Simplify the output for skipped tests down to just showing the one-line
> descriptions.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  t/test-lib.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> index 2aca398e1e..7602bbe9e9 100644
> --- a/t/test-lib.sh
> +++ b/t/test-lib.sh
> @@ -1059,7 +1059,7 @@ test_skip () {
>  				"      <skipped message=\"$message\" />"
>  		fi
>
> -		say_color skip >&3 "skipping test: $@"
> +		say_color skip >&3 "skipping test: $1"

I would be comfortable going further than this and removing this line
entirely. We need the "ok $test_count # skip" below since it's part of
our TAP output, but now the output looks somewhat redundant.

With this patch running a test that I'm working on with `--run=...`, I
get output that looks like:

  skipping test: writing bitmaps via command-line can duplicate .keep objects
  ok 2 # skip writing bitmaps via command-line can duplicate .keep objects (--run)

  skipping test: writing bitmaps via config can duplicate .keep objects
  ok 3 # skip writing bitmaps via config can duplicate .keep objects (--run)

Scanning over the same test description twice per skipped test makes the
output difficult (but still much easier than before) to scan. What do
you think about either of the following:

  skipping test: writing bitmaps via command-line can duplicate .keep objects
  ok 2 # skip (--run)

or:

  ok 2 # skip writing bitmaps via command-line can duplicate .keep objects (--run)

I have a slight preference towards the latter, since it keeps more of
the information in the TAP line, and it cuts the total line count of
output from skipped tests in half.

For what it's worth, I'd also be fine with the patch as-is.

Thanks,
Taylor

  reply	other threads:[~2020-10-13 15:44 UTC|newest]

Thread overview: 71+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-12 23:26 [PATCH 0/3] Make test selection easier by specifying description substrings instead of just numeric counters Elijah Newren via GitGitGadget
2020-10-12 23:26 ` [PATCH 1/3] test-lib: allow selecting tests by substring/regex with --run Elijah Newren via GitGitGadget
2020-10-13 15:39   ` Taylor Blau
2020-10-13 17:23     ` Elijah Newren
2020-10-13 17:28       ` Taylor Blau
2020-10-13 18:21         ` Elijah Newren
2020-10-12 23:26 ` [PATCH 2/3] t6006, t6012: adjust tests to use 'setup' instead of synonyms Elijah Newren via GitGitGadget
2020-10-12 23:26 ` [PATCH 3/3] test-lib: reduce verbosity of skipped tests Elijah Newren via GitGitGadget
2020-10-13 15:44   ` Taylor Blau [this message]
2020-10-13 17:56     ` Elijah Newren
2020-10-13 19:27       ` Junio C Hamano
2020-10-13 13:08 ` [PATCH 0/3] Make test selection easier by specifying description substrings instead of just numeric counters Phillip Wood
2020-10-15 11:48   ` Johannes Schindelin
2020-10-15 13:42     ` Phillip Wood
2020-10-13 19:19 ` [PATCH v2 " Elijah Newren via GitGitGadget
2020-10-13 19:19   ` [PATCH v2 1/3] test-lib: allow selecting tests by substring/regex with --run Elijah Newren via GitGitGadget
2020-10-14 17:04     ` Jeff King
2020-10-14 17:46       ` Junio C Hamano
2020-10-14 18:07         ` Jeff King
2020-10-14 19:24           ` Junio C Hamano
2020-10-14 19:41             ` Elijah Newren
2020-10-14 19:49               ` Jeff King
2020-10-15 16:09                 ` Junio C Hamano
2020-10-15 15:59               ` Junio C Hamano
2020-10-14 17:50       ` Elijah Newren
2020-10-14 17:57         ` Junio C Hamano
2020-10-14 19:12           ` Elijah Newren
2020-10-15 16:10             ` Junio C Hamano
2020-10-14 18:09         ` Jeff King
2020-10-14 19:02           ` Elijah Newren
2020-10-15 16:20       ` Junio C Hamano
2020-10-15 19:46         ` Jeff King
2020-10-15 20:08           ` Junio C Hamano
2020-10-16  0:38             ` Jeff King
2020-10-16 16:16               ` Junio C Hamano
2020-10-16 16:30                 ` Junio C Hamano
2020-10-16 20:06                 ` Jeff King
2020-10-16 20:22                   ` Junio C Hamano
2020-10-13 19:19   ` [PATCH v2 2/3] t6006, t6012: adjust tests to use 'setup' instead of synonyms Elijah Newren via GitGitGadget
2020-10-13 19:19   ` [PATCH v2 3/3] test-lib: reduce verbosity of skipped tests Elijah Newren via GitGitGadget
2020-10-14 16:53     ` Jeff King
2020-10-14 17:39       ` Elijah Newren
2020-10-14 17:55         ` Jeff King
2020-10-14 17:57           ` Elijah Newren
2020-10-13 19:49   ` [PATCH v2 0/3] Make test selection easier by specifying description substrings instead of just numeric counters Taylor Blau
2020-10-14 21:13   ` [PATCH v3 " Elijah Newren via GitGitGadget
2020-10-14 21:13     ` [PATCH v3 1/3] test-lib: allow selecting tests by substring/glob with --run Elijah Newren via GitGitGadget
2020-10-16 11:40       ` Phillip Wood
2020-10-16 17:27         ` Elijah Newren
2020-10-16 18:24           ` Phillip Wood
2020-10-16 20:02           ` Junio C Hamano
2020-10-16 20:07             ` Jeff King
2020-10-14 21:13     ` [PATCH v3 2/3] t6006, t6012: adjust tests to use 'setup' instead of synonyms Elijah Newren via GitGitGadget
2020-10-14 21:13     ` [PATCH v3 3/3] test-lib: reduce verbosity of skipped tests Elijah Newren via GitGitGadget
2020-10-15 19:48     ` [PATCH v3 0/3] Make test selection easier by specifying description substrings instead of just numeric counters Jeff King
2020-10-16 19:28     ` [PATCH v4 " Elijah Newren via GitGitGadget
2020-10-16 19:28       ` [PATCH v4 1/3] test-lib: allow selecting tests by substring/glob with --run Elijah Newren via GitGitGadget
2020-10-16 19:28       ` [PATCH v4 2/3] t6006, t6012: adjust tests to use 'setup' instead of synonyms Elijah Newren via GitGitGadget
2020-10-16 19:28       ` [PATCH v4 3/3] test-lib: reduce verbosity of skipped tests Elijah Newren via GitGitGadget
2020-10-16 20:25       ` [PATCH v4 0/3] Make test selection easier by specifying description substrings instead of just numeric counters Junio C Hamano
2020-10-16 20:56         ` Elijah Newren
2020-10-16 22:50       ` [PATCH v5 " Elijah Newren via GitGitGadget
2020-10-16 22:50         ` [PATCH v5 1/3] test-lib: allow selecting tests by substring/glob with --run Elijah Newren via GitGitGadget
2020-10-16 23:25           ` Andrei Rybak
2020-10-17 23:43             ` Elijah Newren
2020-10-16 22:50         ` [PATCH v5 2/3] t6006, t6012: adjust tests to use 'setup' instead of synonyms Elijah Newren via GitGitGadget
2020-10-16 22:50         ` [PATCH v5 3/3] test-lib: reduce verbosity of skipped tests Elijah Newren via GitGitGadget
2020-10-18  0:23         ` [PATCH v6 0/3] Make test selection easier by specifying description substrings instead of just numeric counters Elijah Newren via GitGitGadget
2020-10-18  0:23           ` [PATCH v6 1/3] test-lib: allow selecting tests by substring/glob with --run Elijah Newren via GitGitGadget
2020-10-18  0:23           ` [PATCH v6 2/3] t6006, t6012: adjust tests to use 'setup' instead of synonyms Elijah Newren via GitGitGadget
2020-10-18  0:23           ` [PATCH v6 3/3] test-lib: reduce verbosity of skipped tests Elijah Newren via GitGitGadget

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201013154439.GB1913367@nand.local \
    --to=me@ttaylorr.com \
    --cc=git@vger.kernel.org \
    --cc=gitgitgadget@gmail.com \
    --cc=newren@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).