Re: [PATCH v4 8/7] wildmatch test: skip file creation tests on Windows proper

[Johannes Schindelin <Johannes.Schindelin@gmx.de>]

Hi Duy,

On Sun, 7 Jan 2018, Duy Nguyen wrote:

> On Sat, Jan 6, 2018 at 7:51 PM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> > Nobody likes to run tests that take too
> > long. And look at this:
> >
> >         ...
> >         ok 1511 - ipathmatch: match 'Z' '[Z-y]'
> >         ok 1512 - ipathmatch(ls): match '[Z-y]' 'Z'
> >         # still have 84 known breakage(s)
> >         # failed 52 among remaining 1428 test(s)
> >         1..1512
> >
> >         real    5m51.432s
> >         user    0m33.986s
> >         sys     2m13.162s
> >
> > Yep. It takes *over eight minutes*.
> 
> I suppose this is because the sheer number of test cases adds a lot of
> shell overhead on Windows. I wonder if it's better to rewrite this
> test in C instead. We start to do some more unit testing here and
> there and kind of abuse the sh-based test framework for this. Having a
> proper unit test framework would be good anyway since it's sometimes
> hard to create a specific scenario with high level commands.

I agree that it would make a ton of sense to use a proper, portable test
framework written in pure, portable C.

However, this ship has long sailed, hasn't it?

Of course, we do have useful stuff in t/helper/. And we have precedent for
more sensible bulk testing that does not require sh to generate or provide
lists of test data, see e.g. the basename/dirname tests. And we could
organize t/helper/ better, including a refactoring to have a single binary
rather than tons and tons of binaries that all link to libgit.a and
take a lot of space (even with LZMA compression, the current test
artifacts take about 90 megabyte!).

If I had the time I would write this up as a valuable GSoC project. Not
that Junio cares. But I do.

Ciao,
Dscho