git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / Atom feed
From: Duy Nguyen <pclouds@gmail.com>
To: Jeff King <peff@peff.net>
Cc: Christian Couder <christian.couder@gmail.com>,
	git <git@vger.kernel.org>, Stefan Beller <sbeller@google.com>,
	SZEDER Gábor <szeder.dev@gmail.com>,
	Thomas Gummerer <t.gummerer@gmail.com>,
	Оля Тележная  <olyatelezhnaya@gmail.com>,
	Matthieu Moy <Matthieu.Moy@gmail.com>
Subject: Re: GSoC 2019: Git's application submitted
Date: Wed, 6 Mar 2019 16:36:12 +0700
Message-ID: <CACsJy8BaCMsXhXSAwu+wmZCTwL5TO8wVDtGd87vRwH=AcOsThA@mail.gmail.com> (raw)
In-Reply-To: <20190306044955.GC6664@sigill.intra.peff.net>

On Wed, Mar 6, 2019 at 11:49 AM Jeff King <peff@peff.net> wrote:
>
> On Tue, Mar 05, 2019 at 07:04:59PM +0700, Duy Nguyen wrote:
>
> > On Mon, Feb 4, 2019 at 4:17 PM Christian Couder
> > <christian.couder@gmail.com> wrote:
> > >
> > > Hi everyone,
> > >
> > > There are now ideas, micro-projects and organization application pages
> > > for GSoC 2019 on https://git.github.io/
> > >
> > > It would be nice to have a few more project ideas.
> >
> > Not sure if it's too late now. Anyway this could be something fun to
> > do: support C-based tests in our test suite.
> >
> > A while back I noticed some test running very long because it was
> > trying a lot of input combination. The actual logic is not much, but
> > because of the increasing number of test cases, overhead goes off the
> > roof. The last part is probably not true, but Windows port I think is
> > hit much harder than what I experience, and I think Dscho did complain
> > about it.
> >
> > So what this project does is somehow allow people to write test cases
> > in C instead of shell. Imagine replacing t3070-wildmatch.sh with a
> > binary program t3070-wildmatch that behaves the same way. This test
> > framework needs to support the same basic feature set as test-lib.sh:
> > TAP output, test results summary, maybe -i and --valgrind... To
> > demonstrate that the test framework works, one of these long test
> > files should be rewritten in C. I'm sure there's one that is simple to
> > rewrite.
> >
> > I'm pretty sure I had some fun with this idea and made some prototype
> > but I couldn't find it. If I do, I'll post the link here.
>
> In my experience, it's nicer to have a tool written in C that can be
> driven by arbitrary input. That makes it easy to write new test cases,
> because you just have to write in some easy domain-specific format
> instead of embedding the test data in C code.
>
> And many of our tests do work like that (in fact, many of the Git
> plumbing tools function as that). E.g., test-date gives you direct
> access to the low-level routines, and we feed it a variety of dates.
>
> That doesn't help with the cost of invoking that tool over and over,
> though, once per test case. I wonder if we could have some kind of
> hybrid. I.e., where t3070 is still a shell script, but it primarily
> consists of running one big binary, like:
>
>   test-wildmatch <<-\EOF
>   case 1
>   case 2
>   ...etc
>   EOF
>
> but with one added twist: test-wildmatch would actually generate TAP
> output for each test, rather than just returning 0/1 for each success or
> failure, and being embedded in a test_expect_success.

Yeah. I mean, converting a full file is just one of the ways to go.
The exact design is up to whoever implements it.

> It seems like that would even be pretty easy to do, with the exception
> of the numbering.

I'd like some niceties from test-lib.sh though. -i should be supported
to stop at the first failure. -v would be nice if possible. Skipping
tsets also.

> It would be nice if we could intermingle this kind of
> "chunk of C tests" with normal tests, but we'd have to figure out how
> many tests it ran and increment our shell-script's counter
> appropriately.

Even if you convert a full file, you need to do that anyway, to make
sure final test run summary is correct (at least when TAP is not used)

This kind of framework, I think also helps write new unit tests. There
are cases where creating the right condition to trigger just one
function is lots of work and easy to break. Sometimes it's easier to
do everything in C when full repository is not involved.
-- 
Duy

  reply index

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-02-04  9:16 Christian Couder
     [not found] ` <CAL21Bm=K6zZ=APkiP3A_X7xVoOfx-MY2435YMp5y1ztE-xyYtg@mail.gmail.com>
2019-02-04 12:54   ` Christian Couder
2019-02-04 21:52 ` Thomas Gummerer
2019-02-05 21:17   ` Thomas Gummerer
2019-02-05 22:00     ` Christian Couder
2019-02-06 22:09       ` Thomas Gummerer
2019-02-07 19:39         ` Johannes Schindelin
2019-02-07 21:33           ` Thomas Gummerer
2019-02-11  5:41             ` Оля Тележная
2019-02-11  7:45               ` Christian Couder
2019-02-11  8:31                 ` Оля Тележная
2019-02-11 10:52                   ` Christian Couder
2019-02-13 22:36               ` Elijah Newren
2019-02-14  9:48                 ` Christian Couder
2019-02-11  8:35             ` Christian Couder
2019-02-11 22:18               ` Thomas Gummerer
2019-02-11 23:58                 ` Christian Couder
2019-02-12 20:25                   ` Thomas Gummerer
2019-02-12 20:49                     ` Christian Couder
2019-02-12 22:13                       ` Thomas Gummerer
2019-02-06 12:27     ` Johannes Schindelin
2019-03-05 12:04 ` Duy Nguyen
2019-03-05 12:23   ` Duy Nguyen
2019-03-06  4:49   ` Jeff King
2019-03-06  9:36     ` Duy Nguyen [this message]
2019-03-06 19:08       ` Jeff King
2019-03-06 14:16     ` Johannes Schindelin
2019-03-18 12:51 ` Duy Nguyen
2019-03-18 16:37   ` Christian Couder

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CACsJy8BaCMsXhXSAwu+wmZCTwL5TO8wVDtGd87vRwH=AcOsThA@mail.gmail.com' \
    --to=pclouds@gmail.com \
    --cc=Matthieu.Moy@gmail.com \
    --cc=christian.couder@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=olyatelezhnaya@gmail.com \
    --cc=peff@peff.net \
    --cc=sbeller@google.com \
    --cc=szeder.dev@gmail.com \
    --cc=t.gummerer@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org list mirror (unofficial, one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox