git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / Atom feed
From: Jeff King <peff@peff.net>
To: Duy Nguyen <pclouds@gmail.com>
Cc: Christian Couder <christian.couder@gmail.com>,
	git <git@vger.kernel.org>, Stefan Beller <sbeller@google.com>,
	SZEDER Gábor <szeder.dev@gmail.com>,
	Thomas Gummerer <t.gummerer@gmail.com>,
	Оля Тележная  <olyatelezhnaya@gmail.com>,
	Matthieu Moy <Matthieu.Moy@gmail.com>
Subject: Re: GSoC 2019: Git's application submitted
Date: Tue, 5 Mar 2019 23:49:56 -0500
Message-ID: <20190306044955.GC6664@sigill.intra.peff.net> (raw)
In-Reply-To: <CACsJy8DhDe+28z2=3v1dBqco7q2Zj12hdKKT==yWFPui8SBkiw@mail.gmail.com>

On Tue, Mar 05, 2019 at 07:04:59PM +0700, Duy Nguyen wrote:

> On Mon, Feb 4, 2019 at 4:17 PM Christian Couder
> <christian.couder@gmail.com> wrote:
> >
> > Hi everyone,
> >
> > There are now ideas, micro-projects and organization application pages
> > for GSoC 2019 on https://git.github.io/
> >
> > It would be nice to have a few more project ideas.
> 
> Not sure if it's too late now. Anyway this could be something fun to
> do: support C-based tests in our test suite.
> 
> A while back I noticed some test running very long because it was
> trying a lot of input combination. The actual logic is not much, but
> because of the increasing number of test cases, overhead goes off the
> roof. The last part is probably not true, but Windows port I think is
> hit much harder than what I experience, and I think Dscho did complain
> about it.
> 
> So what this project does is somehow allow people to write test cases
> in C instead of shell. Imagine replacing t3070-wildmatch.sh with a
> binary program t3070-wildmatch that behaves the same way. This test
> framework needs to support the same basic feature set as test-lib.sh:
> TAP output, test results summary, maybe -i and --valgrind... To
> demonstrate that the test framework works, one of these long test
> files should be rewritten in C. I'm sure there's one that is simple to
> rewrite.
> 
> I'm pretty sure I had some fun with this idea and made some prototype
> but I couldn't find it. If I do, I'll post the link here.

In my experience, it's nicer to have a tool written in C that can be
driven by arbitrary input. That makes it easy to write new test cases,
because you just have to write in some easy domain-specific format
instead of embedding the test data in C code.

And many of our tests do work like that (in fact, many of the Git
plumbing tools function as that). E.g., test-date gives you direct
access to the low-level routines, and we feed it a variety of dates.

That doesn't help with the cost of invoking that tool over and over,
though, once per test case. I wonder if we could have some kind of
hybrid. I.e., where t3070 is still a shell script, but it primarily
consists of running one big binary, like:

  test-wildmatch <<-\EOF
  case 1
  case 2
  ...etc
  EOF

but with one added twist: test-wildmatch would actually generate TAP
output for each test, rather than just returning 0/1 for each success or
failure, and being embedded in a test_expect_success.

It seems like that would even be pretty easy to do, with the exception
of the numbering. It would be nice if we could intermingle this kind of
"chunk of C tests" with normal tests, but we'd have to figure out how
many tests it ran and increment our shell-script's counter
appropriately.

-Peff

  parent reply index

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-02-04  9:16 Christian Couder
     [not found] ` <CAL21Bm=K6zZ=APkiP3A_X7xVoOfx-MY2435YMp5y1ztE-xyYtg@mail.gmail.com>
2019-02-04 12:54   ` Christian Couder
2019-02-04 21:52 ` Thomas Gummerer
2019-02-05 21:17   ` Thomas Gummerer
2019-02-05 22:00     ` Christian Couder
2019-02-06 22:09       ` Thomas Gummerer
2019-02-07 19:39         ` Johannes Schindelin
2019-02-07 21:33           ` Thomas Gummerer
2019-02-11  5:41             ` Оля Тележная
2019-02-11  7:45               ` Christian Couder
2019-02-11  8:31                 ` Оля Тележная
2019-02-11 10:52                   ` Christian Couder
2019-02-13 22:36               ` Elijah Newren
2019-02-14  9:48                 ` Christian Couder
2019-02-11  8:35             ` Christian Couder
2019-02-11 22:18               ` Thomas Gummerer
2019-02-11 23:58                 ` Christian Couder
2019-02-12 20:25                   ` Thomas Gummerer
2019-02-12 20:49                     ` Christian Couder
2019-02-12 22:13                       ` Thomas Gummerer
2019-02-06 12:27     ` Johannes Schindelin
2019-03-05 12:04 ` Duy Nguyen
2019-03-05 12:23   ` Duy Nguyen
2019-03-06  4:49   ` Jeff King [this message]
2019-03-06  9:36     ` Duy Nguyen
2019-03-06 19:08       ` Jeff King
2019-03-06 14:16     ` Johannes Schindelin
2019-03-18 12:51 ` Duy Nguyen
2019-03-18 16:37   ` Christian Couder

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190306044955.GC6664@sigill.intra.peff.net \
    --to=peff@peff.net \
    --cc=Matthieu.Moy@gmail.com \
    --cc=christian.couder@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=olyatelezhnaya@gmail.com \
    --cc=pclouds@gmail.com \
    --cc=sbeller@google.com \
    --cc=szeder.dev@gmail.com \
    --cc=t.gummerer@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org list mirror (unofficial, one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git