From: Andreas Ericsson <ae@op5.se>
To: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Cc: Jakub Narebski <jnareb@gmail.com>,
git@vger.kernel.org, Junio C Hamano <gitster@pobox.com>
Subject: Re: [PATCH v2 0/3] Make the Git tests emit TAP format
Date: Tue, 15 Jun 2010 18:42:59 +0200 [thread overview]
Message-ID: <4C17AD93.1020402@op5.se> (raw)
In-Reply-To: <AANLkTimxE0mqmitRzlXjSAO6v7IOEg4guUnRlzJUyIF1@mail.gmail.com>
On 06/15/2010 05:17 PM, Ævar Arnfjörð Bjarmason wrote:
> On Mon, Jun 14, 2010 at 22:10, Ævar Arnfjörð Bjarmason<avarab@gmail.com> wrote:
>> On Mon, Jun 14, 2010 at 21:49, Jakub Narebski<jnareb@gmail.com> wrote:
>>> Ævar Arnfjörð Bjarmason<avarab@gmail.com> writes:
>>>
>>>> All this series does is slightly adjust the raw text output of our
>>>> tests so that it conforms. to the TAP standard, i.e. instead of this:
>>>>
>>>> $ ./t0005-signals.sh
>>>> * ok 1: sigchain works
>>>> * passed all 1 test(s)
>>>>
>>>> We get this:
>>>>
>>>> $ ./t0005-signals.sh
>>>> ok 1 - sigchain works
>>>> # passed all 1 test(s)
>>>> 1..1
>>>>
>>>> Changing the output format like this gives us the ability to run the
>>>> Git tests with any TAP tool (like prove(1)) at no extra cost. Every
>>>> other existing way of running the tests continues to work, it's just
>>>> easier for machines to read the output now.
>>
>> I'll re-submit a version of the patch with a better commit message
>> which addresses all of the below.
>>
>>> This doesn't tell us if the result of running test suite with
>>> '--verbose' and/or with '--debug' changes, and if changes how? Is it
>>> compatibile with TAP format so that TAP parsers understand it?
>>
>> It just changes in that the lines that previously said e.g. "* ok 1:
>> sigchain works" now say "ok 1 - sigchain works".
>>
>> TAP parsers still understand it, since ignoring unknown garbage is
>> part of the TAP standard.
>
> Actually it doesn't work for all the tests. The following crops up on
> prove -j 10 ./t[0-9]*.sh :: --verbose:
>
> ./t1007-hash-object.sh (Wstat: 0
> Tests: 19 Failed: 0)
> Parse errors: Tests out of sequence. Found (12) but expected (11)
> Tests out of sequence. Found (13) but expected (12)
> Tests out of sequence. Found (14) but expected (13)
> Tests out of sequence. Found (15) but expected (14)
> Tests out of sequence. Found (16) but expected (15)
Why are you using a plan at all? I absolutely detest using planned tap
series, and last I heard was that the majority of TAP-fanatics actually
agreed that using a plan to run tests was an extraordinarily lousy idea,
since it makes it harder to add tests properly.
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.
next prev parent reply other threads:[~2010-06-15 16:43 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-06-09 15:22 [PATCH v2 0/3] Make the Git tests emit TAP format Ævar Arnfjörð Bjarmason
2010-06-09 15:22 ` [PATCH v2 1/3] Make test-lib.sh emit valid " Ævar Arnfjörð Bjarmason
2010-06-14 22:01 ` Jakub Narebski
2010-06-14 22:29 ` Ævar Arnfjörð Bjarmason
2010-06-09 15:22 ` [PATCH v2 2/3] Skip tests in a way that makes sense under TAP Ævar Arnfjörð Bjarmason
2010-06-09 15:24 ` [PATCH v2 3/3] We use TAP so the Perl test can run without scaffolding Ævar Arnfjörð Bjarmason
2010-06-14 21:49 ` [PATCH v2 0/3] Make the Git tests emit TAP format Jakub Narebski
2010-06-14 22:10 ` Ævar Arnfjörð Bjarmason
2010-06-14 23:16 ` Ævar Arnfjörð Bjarmason
2010-06-15 3:08 ` Junio C Hamano
2010-06-15 3:10 ` Ævar Arnfjörð Bjarmason
2010-06-15 15:17 ` Ævar Arnfjörð Bjarmason
2010-06-15 16:42 ` Andreas Ericsson [this message]
2010-06-15 16:49 ` Ævar Arnfjörð Bjarmason
2010-06-15 22:32 ` [PATCH v3 0/5] TAP support for Git Ævar Arnfjörð Bjarmason
2010-06-15 22:32 ` [PATCH v3 1/5] Make test-lib.sh emit valid TAP format Ævar Arnfjörð Bjarmason
2010-06-15 22:32 ` [PATCH v3 2/5] Skip tests in a way that makes sense under TAP Ævar Arnfjörð Bjarmason
2010-06-15 22:32 ` [PATCH v3 3/5] We use TAP so the Perl test can run without scaffolding Ævar Arnfjörð Bjarmason
2010-06-15 22:32 ` [PATCH v3 4/5] TAP: Say "pass" rather than "ok" on an empty line Ævar Arnfjörð Bjarmason
2010-06-15 22:32 ` [PATCH v3 5/5] TAP: Make sure there's a newline before "ok" under harness Ævar Arnfjörð Bjarmason
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4C17AD93.1020402@op5.se \
--to=ae@op5.se \
--cc=avarab@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=jnareb@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).