From: Eric Sunshine <sunshine@sunshineco.com>
To: Stefan Beller <sbeller@google.com>
Cc: Git List <git@vger.kernel.org>, Elijah Newren <newren@gmail.com>,
Johannes Sixt <j6t@kdbg.org>,
Jonathan Nieder <jrnieder@gmail.com>,
Jonathan Tan <jonathantanmy@google.com>,
Junio C Hamano <gitster@pobox.com>,
Luke Diamand <luke@diamand.org>, Jeff King <peff@peff.net>
Subject: Re: [PATCH 01/25] t: use test_might_fail() instead of manipulating exit code manually
Date: Mon, 2 Jul 2018 16:58:13 -0400 [thread overview]
Message-ID: <CAPig+cSpDi8o6s2dQoDqb5_fbZ1ROY0guPxptQ=6syK6r9Shcg@mail.gmail.com> (raw)
In-Reply-To: <CAGZ79kah+mFH0YZ1L0R+f=fRVku1mesigm86oHTHKRRFc3y+0A@mail.gmail.com>
On Mon, Jul 2, 2018 at 1:44 PM Stefan Beller <sbeller@google.com> wrote:
> > diff --git a/t/t4012-diff-binary.sh b/t/t4012-diff-binary.sh
> > test_expect_success 'diff --no-index with binary creation' '
> > echo Q | q_to_nul >binary &&
> > - (: hide error code from diff, which just indicates differences
> > - git diff --binary --no-index /dev/null binary >current ||
> > - true
> > - ) &&
> > + # hide error code from diff, which just indicates differences
> > + test_might_fail git diff --binary --no-index /dev/null binary >current &&
>
> I am not sure why we need to be non-deterministic here, i.e. I would rather
> test for success or non-success error code and not just *any* error code.
>
> While I think this patch is a strict improvement for the test suite,
> I do wonder if we can tighten the exit code check here (maybe in
> a follow up series?).
I agree that such a change is out of the scope of this patch series.
But, do keep in mind that the exit code of git-diff doesn't indicate
only "success" or "error". Various exit codes represent different
conditions; for instance, 1 means differences were found, 0 means no
differences, and -1 means error. The outcome of this test is based
upon output generated by git-diff, so the test itself will fail
overall if git-diff doesn't produce expected results. Also, there is
an entire test script (t4017-diff-retval.sh) concerned with verifying
git-diff exit code, so I'm not sure we need to worry about it too much
here.
(I'm not arguing against a follow-up patch, though, just thinking
aloud about its potential value.)
next prev parent reply other threads:[~2018-07-02 20:58 UTC|newest]
Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-07-02 0:23 [PATCH 00/25] fix buggy tests, modernize tests, fix broken &&-chains Eric Sunshine
2018-07-02 0:23 ` [PATCH 01/25] t: use test_might_fail() instead of manipulating exit code manually Eric Sunshine
2018-07-02 17:44 ` Stefan Beller
2018-07-02 20:58 ` Eric Sunshine [this message]
2018-07-03 19:20 ` Junio C Hamano
2018-07-02 0:23 ` [PATCH 02/25] t: use test_write_lines() instead of series of 'echo' commands Eric Sunshine
2018-07-02 17:53 ` Stefan Beller
2018-07-02 0:23 ` [PATCH 03/25] t: use sane_unset() rather than 'unset' with broken &&-chain Eric Sunshine
2018-07-02 0:23 ` [PATCH 04/25] t: drop unnecessary terminating semicolon in subshell Eric Sunshine
2018-07-03 19:23 ` Junio C Hamano
2018-07-02 0:23 ` [PATCH 05/25] t/lib-submodule-update: fix "absorbing" test Eric Sunshine
2018-07-02 0:23 ` [PATCH 06/25] t5405: use test_must_fail() instead of checking exit code manually Eric Sunshine
2018-07-02 0:23 ` [PATCH 07/25] t5406: use write_script() instead of birthing shell script manually Eric Sunshine
2018-07-02 0:23 ` [PATCH 08/25] t5505: modernize and simplify hard-to-digest test Eric Sunshine
2018-07-02 0:23 ` [PATCH 09/25] t6036: fix broken "merge fails but has appropriate contents" tests Eric Sunshine
2018-07-02 0:23 ` [PATCH 10/25] t7201: drop pointless "exit 0" at end of subshell Eric Sunshine
2018-07-02 0:23 ` [PATCH 11/25] t7400: fix broken "submodule add/reconfigure --force" test Eric Sunshine
2018-07-16 14:43 ` Johannes Schindelin
2018-07-16 15:51 ` Johannes Schindelin
2018-07-16 18:50 ` Eric Sunshine
2018-07-16 21:37 ` Junio C Hamano
2018-07-16 23:05 ` Eric Sunshine
2018-07-17 10:09 ` Johannes Schindelin
2018-07-17 17:53 ` Junio C Hamano
2018-07-02 0:23 ` [PATCH 12/25] t7810: use test_expect_code() instead of hand-rolled comparison Eric Sunshine
2018-07-02 18:05 ` Stefan Beller
2018-07-02 21:12 ` Eric Sunshine
2018-07-02 0:23 ` [PATCH 13/25] t9001: fix broken "invoke hook" test Eric Sunshine
2018-07-02 0:23 ` [PATCH 14/25] t9814: simplify convoluted check that command correctly errors out Eric Sunshine
2018-07-02 0:23 ` [PATCH 15/25] t0000-t0999: fix broken &&-chains Eric Sunshine
2018-07-02 0:23 ` [PATCH 16/25] t1000-t1999: " Eric Sunshine
2018-07-02 0:23 ` [PATCH 17/25] t2000-t2999: " Eric Sunshine
2018-07-02 0:23 ` [PATCH 18/25] t3000-t3999: " Eric Sunshine
2018-07-02 0:23 ` [PATCH 19/25] t3030: " Eric Sunshine
2018-07-02 0:24 ` [PATCH 20/25] t4000-t4999: " Eric Sunshine
2018-07-02 0:24 ` [PATCH 21/25] t5000-t5999: " Eric Sunshine
2018-07-12 12:37 ` SZEDER Gábor
2018-07-12 17:44 ` Eric Sunshine
2018-07-12 18:19 ` Jeff King
2018-07-12 18:31 ` Junio C Hamano
2018-07-12 18:35 ` Junio C Hamano
2018-07-12 18:46 ` Eric Sunshine
2018-07-12 18:50 ` Junio C Hamano
2018-07-12 18:53 ` Eric Sunshine
2018-07-17 8:15 ` SZEDER Gábor
2018-07-02 0:24 ` [PATCH 22/25] t6000-t6999: " Eric Sunshine
2018-07-02 0:24 ` [PATCH 23/25] t7000-t7999: " Eric Sunshine
2018-07-02 0:24 ` [PATCH 24/25] t9000-t9999: " Eric Sunshine
2018-07-02 0:24 ` [PATCH 25/25] t9119: " Eric Sunshine
2018-07-02 18:27 ` [PATCH 00/25] fix buggy tests, modernize tests, " Stefan Beller
2018-07-02 21:20 ` Eric Sunshine
2018-07-03 19:40 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAPig+cSpDi8o6s2dQoDqb5_fbZ1ROY0guPxptQ=6syK6r9Shcg@mail.gmail.com' \
--to=sunshine@sunshineco.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=j6t@kdbg.org \
--cc=jonathantanmy@google.com \
--cc=jrnieder@gmail.com \
--cc=luke@diamand.org \
--cc=newren@gmail.com \
--cc=peff@peff.net \
--cc=sbeller@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).