git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Eric Sunshine <sunshine@sunshineco.com>
To: Stefan Beller <sbeller@google.com>
Cc: Git List <git@vger.kernel.org>, Elijah Newren <newren@gmail.com>,
	Johannes Sixt <j6t@kdbg.org>,
	Jonathan Nieder <jrnieder@gmail.com>,
	Jonathan Tan <jonathantanmy@google.com>,
	Junio C Hamano <gitster@pobox.com>,
	Luke Diamand <luke@diamand.org>, Jeff King <peff@peff.net>
Subject: Re: [PATCH 01/25] t: use test_might_fail() instead of manipulating exit code manually
Date: Mon, 2 Jul 2018 16:58:13 -0400	[thread overview]
Message-ID: <CAPig+cSpDi8o6s2dQoDqb5_fbZ1ROY0guPxptQ=6syK6r9Shcg@mail.gmail.com> (raw)
In-Reply-To: <CAGZ79kah+mFH0YZ1L0R+f=fRVku1mesigm86oHTHKRRFc3y+0A@mail.gmail.com>

On Mon, Jul 2, 2018 at 1:44 PM Stefan Beller <sbeller@google.com> wrote:
> > diff --git a/t/t4012-diff-binary.sh b/t/t4012-diff-binary.sh
> >  test_expect_success 'diff --no-index with binary creation' '
> >         echo Q | q_to_nul >binary &&
> > -       (: hide error code from diff, which just indicates differences
> > -        git diff --binary --no-index /dev/null binary >current ||
> > -        true
> > -       ) &&
> > +       # hide error code from diff, which just indicates differences
> > +       test_might_fail git diff --binary --no-index /dev/null binary >current &&
>
> I am not sure why we need to be non-deterministic here, i.e. I would rather
> test for success or non-success error code and not just *any* error code.
>
> While I think this patch is a strict improvement for the test suite,
> I do wonder if we can tighten the exit code check here (maybe in
> a follow up series?).

I agree that such a change is out of the scope of this patch series.

But, do keep in mind that the exit code of git-diff doesn't indicate
only "success" or "error". Various exit codes represent different
conditions; for instance, 1 means differences were found, 0 means no
differences, and -1 means error. The outcome of this test is based
upon output generated by git-diff, so the test itself will fail
overall if git-diff doesn't produce expected results. Also, there is
an entire test script (t4017-diff-retval.sh) concerned with verifying
git-diff exit code, so I'm not sure we need to worry about it too much
here.

(I'm not arguing against a follow-up patch, though, just thinking
aloud about its potential value.)

  reply	other threads:[~2018-07-02 20:58 UTC|newest]

Thread overview: 52+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-07-02  0:23 [PATCH 00/25] fix buggy tests, modernize tests, fix broken &&-chains Eric Sunshine
2018-07-02  0:23 ` [PATCH 01/25] t: use test_might_fail() instead of manipulating exit code manually Eric Sunshine
2018-07-02 17:44   ` Stefan Beller
2018-07-02 20:58     ` Eric Sunshine [this message]
2018-07-03 19:20   ` Junio C Hamano
2018-07-02  0:23 ` [PATCH 02/25] t: use test_write_lines() instead of series of 'echo' commands Eric Sunshine
2018-07-02 17:53   ` Stefan Beller
2018-07-02  0:23 ` [PATCH 03/25] t: use sane_unset() rather than 'unset' with broken &&-chain Eric Sunshine
2018-07-02  0:23 ` [PATCH 04/25] t: drop unnecessary terminating semicolon in subshell Eric Sunshine
2018-07-03 19:23   ` Junio C Hamano
2018-07-02  0:23 ` [PATCH 05/25] t/lib-submodule-update: fix "absorbing" test Eric Sunshine
2018-07-02  0:23 ` [PATCH 06/25] t5405: use test_must_fail() instead of checking exit code manually Eric Sunshine
2018-07-02  0:23 ` [PATCH 07/25] t5406: use write_script() instead of birthing shell script manually Eric Sunshine
2018-07-02  0:23 ` [PATCH 08/25] t5505: modernize and simplify hard-to-digest test Eric Sunshine
2018-07-02  0:23 ` [PATCH 09/25] t6036: fix broken "merge fails but has appropriate contents" tests Eric Sunshine
2018-07-02  0:23 ` [PATCH 10/25] t7201: drop pointless "exit 0" at end of subshell Eric Sunshine
2018-07-02  0:23 ` [PATCH 11/25] t7400: fix broken "submodule add/reconfigure --force" test Eric Sunshine
2018-07-16 14:43   ` Johannes Schindelin
2018-07-16 15:51     ` Johannes Schindelin
2018-07-16 18:50       ` Eric Sunshine
2018-07-16 21:37         ` Junio C Hamano
2018-07-16 23:05           ` Eric Sunshine
2018-07-17 10:09             ` Johannes Schindelin
2018-07-17 17:53           ` Junio C Hamano
2018-07-02  0:23 ` [PATCH 12/25] t7810: use test_expect_code() instead of hand-rolled comparison Eric Sunshine
2018-07-02 18:05   ` Stefan Beller
2018-07-02 21:12     ` Eric Sunshine
2018-07-02  0:23 ` [PATCH 13/25] t9001: fix broken "invoke hook" test Eric Sunshine
2018-07-02  0:23 ` [PATCH 14/25] t9814: simplify convoluted check that command correctly errors out Eric Sunshine
2018-07-02  0:23 ` [PATCH 15/25] t0000-t0999: fix broken &&-chains Eric Sunshine
2018-07-02  0:23 ` [PATCH 16/25] t1000-t1999: " Eric Sunshine
2018-07-02  0:23 ` [PATCH 17/25] t2000-t2999: " Eric Sunshine
2018-07-02  0:23 ` [PATCH 18/25] t3000-t3999: " Eric Sunshine
2018-07-02  0:23 ` [PATCH 19/25] t3030: " Eric Sunshine
2018-07-02  0:24 ` [PATCH 20/25] t4000-t4999: " Eric Sunshine
2018-07-02  0:24 ` [PATCH 21/25] t5000-t5999: " Eric Sunshine
2018-07-12 12:37   ` SZEDER Gábor
2018-07-12 17:44     ` Eric Sunshine
2018-07-12 18:19       ` Jeff King
2018-07-12 18:31       ` Junio C Hamano
2018-07-12 18:35         ` Junio C Hamano
2018-07-12 18:46           ` Eric Sunshine
2018-07-12 18:50             ` Junio C Hamano
2018-07-12 18:53               ` Eric Sunshine
2018-07-17  8:15           ` SZEDER Gábor
2018-07-02  0:24 ` [PATCH 22/25] t6000-t6999: " Eric Sunshine
2018-07-02  0:24 ` [PATCH 23/25] t7000-t7999: " Eric Sunshine
2018-07-02  0:24 ` [PATCH 24/25] t9000-t9999: " Eric Sunshine
2018-07-02  0:24 ` [PATCH 25/25] t9119: " Eric Sunshine
2018-07-02 18:27 ` [PATCH 00/25] fix buggy tests, modernize tests, " Stefan Beller
2018-07-02 21:20   ` Eric Sunshine
2018-07-03 19:40 ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAPig+cSpDi8o6s2dQoDqb5_fbZ1ROY0guPxptQ=6syK6r9Shcg@mail.gmail.com' \
    --to=sunshine@sunshineco.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=j6t@kdbg.org \
    --cc=jonathantanmy@google.com \
    --cc=jrnieder@gmail.com \
    --cc=luke@diamand.org \
    --cc=newren@gmail.com \
    --cc=peff@peff.net \
    --cc=sbeller@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).