Re: [PATCH v2 7/7] wildmatch test: create & test files on disk in addition to in-memory

["Ævar Arnfjörð Bjarmason" <avarab@gmail.com>]

On Wed, Jan 03 2018, Adam Dinwoodie jotted:

> On Monday 25 December 2017 at 12:28 am +0000, Ævar Arnfjörð Bjarmason wrote:
>> There has never been any full roundtrip testing of what git-ls-files
>> and other functions that use wildmatch() actually do, rather we've
>> been satisfied with just testing the underlying C function.
>>
>> Due to git-ls-files and friends having their own codepaths before they
>> call wildmatch() there's sometimes differences in the behavior between
>> the two, and even when we test for those (as with
>> 9e4e8a64c2 ("pathspec: die on empty strings as pathspec", 2017-06-06))
>> there was no one place where you can review how these two modes
>> differ.
>>
>> Now there is. We now attempt to create a file called $haystack and
>> match $needle against it for each pair of $needle and $haystack that
>> we were passing to test-wildmatch.
>>
>> If we can't create the file we skip the test. This ensures that we can
>> run this on all platforms and not maintain some infinitely growing
>> whitelist of e.g. platforms that don't support certain characters in
>> filenames.
>>
>> As a result of doing this we can now see the cases where these two
>> ways of testing wildmatch differ:
>>
>>  * Creating a file called 'a[]b' and running ls-files 'a[]b' will show
>>    that file, but wildmatch("a[]b", "a[]b") will not match
>>
>>  * wildmatch() won't match a file called \ against \, but ls-files
>>    will.
>>
>>  * `git --glob-pathspecs ls-files 'foo**'` will match a file
>>    'foo/bba/arr', but wildmatch won't, however pathmatch will.
>>
>>    This seems like a bug to me, the two are otherwise equivalent as
>>    these tests show.
>>
>> This also reveals the case discussed in 9e4e8a64c2 above, where '' is
>> now an error as far as ls-files is concerned, but wildmatch() itself
>> happily accepts it.
>>
>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>
> I'm seeing this test script failing on the pu branch as a result of this
> commit when building on Cygwin.  Specifically, the test fails at
> 9d45e1ca4 ("Merge branch 'bw/oidmap-autoinit' into pu", 2017-12-28), and
> bisecting points the blame at 2ee0c785a ("wildmatch test: create & test
> files on disk in addition to in-memory", 2017-12-25).
>
> I've copied the verbose error output for the first error below, and
> uploaded the full output, including verbose and trace output for the
> unexpectedly failing tests, at [0].  (With 42 failures among 1512 tests,
> there's a lot of it, so I didn't want to include it in an email.)

Does the fixup above in <878tdm8k2d.fsf@evledraar.gmail.com> work for
you, i.e. changing $10 in the script to ${10}?

I was waiting on Junio to see if he'd like to just squash that or if I
should re-send.

>     expecting success:
>                                     wildtest_file_setup &&
>                                     printf '%s' '\' >expect &&
>                                     git --glob-pathspecs ls-files -z -- '00' >actual.raw 2>actual.err &&
>                                     wildtest_stdout_stderr_cmp
>
>     ++ wildtest_file_setup
>     ++ test_when_finished '
>                     rm -rf -- * &&
>                     git reset
>             '
>     ++ test 0 = 0
>     ++ test_cleanup='{
>                     rm -rf -- * &&
>                     git reset
>
>                     } && (exit "$eval_ret"); eval_ret=$?; :'
>     ++ git add -A
>     ++ printf %s '\'
>     ++ git --glob-pathspecs ls-files -z -- 00
>     ++ wildtest_stdout_stderr_cmp
>     ++ tr -d '\0'
>     ++ test_cmp expect.err actual.err
>     ++ diff -u expect.err actual.err
>     ++ test_cmp expect actual
>     ++ diff -u expect actual
>     --- expect      2018-01-03 12:43:11.116611200 +0000
>     +++ actual      2018-01-03 12:43:11.216180400 +0000
>     @@ -1 +0,0 @@
>     -\
>     \ No newline at end of file
>     + test_eval_ret_=1
>     + want_trace
>     + test t = t
>     + test t = t
>     + set +x
>     error: last command exited with $?=1
>     ++ rm -rf -- actual actual.err actual.raw expect expect.err
>     ++ git reset
>     ++ exit 1
>     ++ eval_ret=1
>     ++ :
>     + test_eval_ret_=0
>     + want_trace
>     + test t = t
>     + test t = t
>     + set +x
>     not ok 490 - wildmatch(ls): match '00' '\'
>     #
>     #                                       wildtest_file_setup &&
>     #                                       printf '%s' '\' >expect &&
>     #                                       git --glob-pathspecs ls-files -z -- '00' >actual.raw 2>actual.err &&
>     #                                       wildtest_stdout_stderr_cmp
>     #
>
> I'm digging into the failures myself now, but wanted to report the
> problem in the name of getting more eyes on it.
>
> Adam
>
> [0]: https://gist.github.com/me-and/04443bcb00e12436f0eacce079b56d02