git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / Atom feed
From: SZEDER Gábor <szeder.dev@gmail.com>
To: Eric Sunshine <sunshine@sunshineco.com>
Cc: Junio C Hamano <gitster@pobox.com>,
	Git mailing list <git@vger.kernel.org>,
	Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Subject: Re: [PATCH] t6018-rev-list-glob: fix 'empty stdin' test
Date: Wed, 22 Aug 2018 20:59:11 +0200
Message-ID: <CAM0VKjkVreBKQsvMZ=pEE0NN5gG0MM+XJ0MzCbw1rxi_pR+FXQ@mail.gmail.com> (raw)
In-Reply-To: <CAPig+cTQrBmgXQdwQpH7DO2AbraD3B4LiRHDx4Sf9rXg0d_eDA@mail.gmail.com>

On Wed, Aug 22, 2018 at 7:53 PM Eric Sunshine <sunshine@sunshineco.com> wrote:

> Can you say a word or two (here in the email thread) about how you're
> finding these failures (across the various test fixes you've posted
> recently)? Are you instrumenting the code in some fashion? Or, finding
> them by visual inspection?

Errors from system commands in our tests look like these:

  grep: file3: No such file or directory
  sed: -e expression #1, char 2: extra characters after command
  diff: sub1/.git: No such file or directory
  tar: rmtlseek not stopped at a record boundary
  tar: Error is not recoverable: exiting now

while errors from the shell running the test like these:

  t0020-crlf.sh: 8: eval: cannot open two: No such file
  t6018-rev-list-glob.sh: 4: eval: cannot open expect: No such file
  t7408-submodule-reference.sh: 615: test: =: unexpected operator

i.e. lines starting with various system commands' or test scripts'
names, followed by ': '.

So I've modified t/Makefile to not remove the 'test-results' directory
after a successful 'make test':

diff --git a/t/Makefile b/t/Makefile
index ea36cf7ac7..c7b1655593 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -54,10 +54,11 @@ pre-clean:
        $(RM) -r '$(TEST_RESULTS_DIRECTORY_SQ)'

 clean-except-prove-cache:
-       $(RM) -r 'trash directory'.* '$(TEST_RESULTS_DIRECTORY_SQ)'
+       $(RM) -r 'trash directory'.*
        $(RM) -r valgrind/bin

 clean: clean-except-prove-cache
+       $(RM) '$(TEST_RESULTS_DIRECTORY_SQ)'

 distclean: clean
        $(RM) .prove

And then scanned the results of a '--verbose-log -x' test run with:

  grep -E '^(awk|basename|cat|cd|chmod|cmp|cp|cut|diff|dirname|egrep|find|fgrep|grep|gunzip|gzip|ln|mkdir|mkfifo|mktemp|mv|readlink|rmdir|sed|sort|tar|touch|tr|ulimit|umask|uniq|unzip|wc|zipinfo|t[0-9][0-9][0-9][0-9]-[^:]*\.sh):
' test-results/*.out

and then, for lack of something better to do ;), I started looking at
the simpler looking errors.

I've though about how a check like this could be automated, but
haven't had any workable idea yet.  There are commands that can
legitimately print errors, e.g. when checking for a prereq which the
system doesn't have (e.g. the 'tar' errors above, I think).  And the
list of system commands in the grep pattern above is surely incomplete
and will likely change in the future...

  reply index

Thread overview: 95+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-05-02 10:54 [PATCH] checkout & worktree: introduce a core.DWIMRemote setting Ævar Arnfjörð Bjarmason
2018-05-02 15:21 ` Duy Nguyen
2018-05-02 18:00   ` Eric Sunshine
2018-05-02 18:09     ` Duy Nguyen
2018-05-02 18:25     ` Ævar Arnfjörð Bjarmason
2018-05-03 13:18       ` [PATCH v2] checkout & worktree: introduce checkout.implicitRemote Ævar Arnfjörð Bjarmason
2018-05-03 15:14         ` Duy Nguyen
2018-05-04  7:54           ` Ævar Arnfjörð Bjarmason
2018-05-04 14:58             ` Duy Nguyen
2018-05-04 18:02               ` Ævar Arnfjörð Bjarmason
2018-05-04  9:58         ` Eric Sunshine
2018-05-24 19:47           ` [PATCH v3] " Ævar Arnfjörð Bjarmason
2018-05-25  8:12             ` Junio C Hamano
2018-05-25 14:42               ` Duy Nguyen
2018-05-25 18:38                 ` Ævar Arnfjörð Bjarmason
2018-05-26 12:49                   ` Duy Nguyen
2018-05-31  7:45             ` Ævar Arnfjörð Bjarmason
2018-05-31 19:52               ` [PATCH v4 0/9] ambiguous checkout UI & checkout.defaultRemote Ævar Arnfjörð Bjarmason
2018-06-01 21:10                 ` [PATCH v5 0/8] " Ævar Arnfjörð Bjarmason
2018-06-02 11:50                   ` [PATCH v6 " Ævar Arnfjörð Bjarmason
2018-06-05 14:40                     ` [PATCH v7 " Ævar Arnfjörð Bjarmason
2018-06-05 14:40                     ` [PATCH v7 1/8] checkout tests: index should be clean after dwim checkout Ævar Arnfjörð Bjarmason
2018-06-05 15:45                       ` SZEDER Gábor
2018-07-27 17:48                         ` [PATCH] tests: make use of the test_must_be_empty function Ævar Arnfjörð Bjarmason
2018-07-27 21:50                           ` Junio C Hamano
2018-07-31  0:17                           ` SZEDER Gábor
2018-08-22 17:48                           ` [PATCH] t6018-rev-list-glob: fix 'empty stdin' test SZEDER Gábor
2018-08-22 17:53                             ` Eric Sunshine
2018-08-22 18:59                               ` SZEDER Gábor [this message]
2018-08-22 20:30                                 ` Eric Sunshine
2018-08-22 18:01                             ` Junio C Hamano
2018-08-22 18:50                             ` Junio C Hamano
2018-08-22 19:23                               ` Jeff King
2018-08-22 19:50                                 ` [PATCH] rev-list: make empty --stdin not an error Jeff King
2018-08-22 20:42                                   ` Junio C Hamano
2018-08-22 21:37                                     ` Jeff King
2018-08-22 21:50                                       ` Junio C Hamano
2018-08-22 21:55                                         ` Jeff King
2018-08-22 21:41                                     ` Junio C Hamano
2018-06-05 14:40                     ` [PATCH v7 2/8] checkout.h: wrap the arguments to unique_tracking_name() Ævar Arnfjörð Bjarmason
2018-06-05 14:40                     ` [PATCH v7 3/8] checkout.c: introduce an *_INIT macro Ævar Arnfjörð Bjarmason
2018-06-05 14:40                     ` [PATCH v7 4/8] checkout.c: change "unique" member to "num_matches" Ævar Arnfjörð Bjarmason
2018-06-05 14:40                     ` [PATCH v7 5/8] checkout: pass the "num_matches" up to callers Ævar Arnfjörð Bjarmason
2018-06-05 14:40                     ` [PATCH v7 6/8] builtin/checkout.c: use "ret" variable for return Ævar Arnfjörð Bjarmason
2018-06-05 14:40                     ` [PATCH v7 7/8] checkout: add advice for ambiguous "checkout <branch>" Ævar Arnfjörð Bjarmason
2018-06-05 14:40                     ` [PATCH v7 8/8] checkout & worktree: introduce checkout.defaultRemote Ævar Arnfjörð Bjarmason
2018-06-02 11:50                   ` [PATCH v6 1/8] checkout tests: index should be clean after dwim checkout Ævar Arnfjörð Bjarmason
2018-06-02 11:50                   ` [PATCH v6 2/8] checkout.h: wrap the arguments to unique_tracking_name() Ævar Arnfjörð Bjarmason
2018-06-02 11:50                   ` [PATCH v6 3/8] checkout.c: introduce an *_INIT macro Ævar Arnfjörð Bjarmason
2018-06-02 11:50                   ` [PATCH v6 4/8] checkout.c]: change "unique" member to "num_matches" Ævar Arnfjörð Bjarmason
2018-06-02 11:50                   ` [PATCH v6 5/8] checkout: pass the "num_matches" up to callers Ævar Arnfjörð Bjarmason
2018-06-02 11:50                   ` [PATCH v6 6/8] builtin/checkout.c: use "ret" variable for return Ævar Arnfjörð Bjarmason
2018-06-02 11:50                   ` [PATCH v6 7/8] checkout: add advice for ambiguous "checkout <branch>" Ævar Arnfjörð Bjarmason
2018-06-02 11:50                   ` [PATCH v6 8/8] checkout & worktree: introduce checkout.defaultRemote Ævar Arnfjörð Bjarmason
2018-06-03  7:58                     ` Eric Sunshine
2018-06-01 21:10                 ` [PATCH v5 1/8] checkout tests: index should be clean after dwim checkout Ævar Arnfjörð Bjarmason
2018-06-01 21:10                 ` [PATCH v5 2/8] checkout.h: wrap the arguments to unique_tracking_name() Ævar Arnfjörð Bjarmason
2018-06-01 21:10                 ` [PATCH v5 3/8] checkout.[ch]: introduce an *_INIT macro Ævar Arnfjörð Bjarmason
2018-06-01 22:40                   ` Eric Sunshine
2018-06-01 21:10                 ` [PATCH v5 4/8] checkout.[ch]: change "unique" member to "num_matches" Ævar Arnfjörð Bjarmason
2018-06-01 22:41                   ` Eric Sunshine
2018-06-01 21:10                 ` [PATCH v5 5/8] checkout: pass the "num_matches" up to callers Ævar Arnfjörð Bjarmason
2018-06-01 21:10                 ` [PATCH v5 6/8] builtin/checkout.c: use "ret" variable for return Ævar Arnfjörð Bjarmason
2018-06-01 21:10                 ` [PATCH v5 7/8] checkout: add advice for ambiguous "checkout <branch>" Ævar Arnfjörð Bjarmason
2018-06-01 22:52                   ` Eric Sunshine
2018-06-01 21:10                 ` [PATCH v5 8/8] checkout & worktree: introduce checkout.defaultRemote Ævar Arnfjörð Bjarmason
2018-05-31 19:52               ` [PATCH v4 1/9] checkout tests: index should be clean after dwim checkout Ævar Arnfjörð Bjarmason
2018-06-01  4:06                 ` Junio C Hamano
2018-06-01 19:43                   ` Ævar Arnfjörð Bjarmason
2018-05-31 19:52               ` [PATCH v4 2/9] checkout.h: wrap the arguments to unique_tracking_name() Ævar Arnfjörð Bjarmason
2018-05-31 19:52               ` [PATCH v4 3/9] checkout.[ch]: move struct declaration into *.h Ævar Arnfjörð Bjarmason
2018-05-31 21:45                 ` Thomas Gummerer
2018-06-01  2:14                   ` Junio C Hamano
2018-06-01  9:56                     ` Ævar Arnfjörð Bjarmason
2018-05-31 19:52               ` [PATCH v4 4/9] checkout.[ch]: introduce an *_INIT macro Ævar Arnfjörð Bjarmason
2018-06-01  4:16                 ` Junio C Hamano
2018-05-31 19:52               ` [PATCH v4 5/9] checkout.[ch]: change "unique" member to "num_matches" Ævar Arnfjörð Bjarmason
2018-05-31 19:52               ` [PATCH v4 6/9] checkout: pass the "num_matches" up to callers Ævar Arnfjörð Bjarmason
2018-05-31 19:52               ` [PATCH v4 7/9] builtin/checkout.c: use "ret" variable for return Ævar Arnfjörð Bjarmason
2018-05-31 19:52               ` [PATCH v4 8/9] checkout: add advice for ambiguous "checkout <branch>" Ævar Arnfjörð Bjarmason
2018-06-01  4:32                 ` Junio C Hamano
2018-06-01  5:11                   ` Junio C Hamano
2018-06-01  9:54                     ` Ævar Arnfjörð Bjarmason
2018-06-04  1:58                       ` Junio C Hamano
2018-06-01  9:50                   ` Ævar Arnfjörð Bjarmason
2018-06-01  7:53                 ` Eric Sunshine
2018-06-01 19:59                   ` Ævar Arnfjörð Bjarmason
2018-05-31 19:52               ` [PATCH v4 9/9] checkout & worktree: introduce checkout.defaultRemote Ævar Arnfjörð Bjarmason
2018-05-31 21:49                 ` Stefan Beller
2018-06-01  8:04                   ` Eric Sunshine
2018-06-01  9:47                   ` Ævar Arnfjörð Bjarmason
2018-05-31 22:22                 ` Thomas Gummerer
2018-06-01  2:17                   ` Junio C Hamano
2018-05-04  7:14       ` [PATCH] checkout & worktree: introduce a core.DWIMRemote setting Eric Sunshine
2018-05-04  7:23         ` Eric Sunshine

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAM0VKjkVreBKQsvMZ=pEE0NN5gG0MM+XJ0MzCbw1rxi_pR+FXQ@mail.gmail.com' \
    --to=szeder.dev@gmail.com \
    --cc=avarab@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=sunshine@sunshineco.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org list mirror (unofficial, one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git