Re: [v2.35.0 regression] some PCRE hangs under UTF-8 locale (was: [PATCH 1/2] grep/pcre2: use PCRE2_UTF even with ASCII patterns)

["Ævar Arnfjörð Bjarmason" <avarab@gmail.com>]

On Sat, Feb 05 2022, René Scharfe wrote:

> Am 31.01.22 um 22:01 schrieb Ævar Arnfjörð Bjarmason:
>>
>> On Sun, Jan 30 2022, René Scharfe wrote:
>>
>>> Am 30.01.22 um 10:04 schrieb SZEDER Gábor:
>>>> On Sun, Jan 30, 2022 at 08:55:02AM +0100, René Scharfe wrote:
>>>>> e0c6029 (Fix inifinite loop when a single byte newline is searched in
>>>>> JIT., 2020-05-29) [1] sounds like it might have fixed it.  It's part of
>>>>> version 10.36.
>>>>
>>>> I saw this hang on two Ubuntu 20.04 based boxes, which predate that
>>>> fix you mention only by a month or two, and apparently the almost two
>>>> years since then was not enough for this fix to trickle down into
>>>> updated 20.04 pcre packages, because:
>>>>
>>>>> Do you still get the error when you disable JIT, i.e. when you use the
>>>>> pattern "(*NO_JIT)^\s" instead?
>>>>
>>>> No, with this pattern it works as expected.
>>>>
>>>> So is there a more convenient way to disable PCRE JIT in Git?  FWIW,
>>>> (non-git) 'grep -P' works with the same patterns.
>>>
>>> I don't know a better way.  We could do it automatically, though:
>>>
>>> --- >8 ---
>>> Subject: [PATCH] grep: disable JIT on PCRE2 before 10.36 to avoid endless loop
>>>
>>> Commit e0c6029 (Fix inifinite loop when a single byte newline is
>>> searched in JIT., 2020-05-29) of PCRE2 adds the following point to its
>>> ChangeLog for version 10.36:
>>>
>>>   2. Fix inifinite loop when a single byte newline is searched in JIT when
>>>   invalid utf8 mode is enabled.
>>>
>>> Avoid that bug on older versions (which are still reportedly found in
>>> the wild) by disabling the JIT when handling UTF-8.
>>>
>>> Reported-by: SZEDER Gábor <szeder.dev@gmail.com>
>>> Signed-off-by: René Scharfe <l.s.r@web.de>
>>> ---
>>> Not sure how to test it.  Killing git grep after a second or so seems a
>>> bit clumsy.  timeout(1) from GNU coreutils at least allows doing that
>>> from the shell, but it's not a standard tool.  Perhaps we need a new
>>> test helper for that purpose?
>
> https://mywiki.wooledge.org/BashFAQ/068 offers a Perl-based Shell
> function or aborting a program if it takes too long:
>
>    doalarm() { perl -e 'alarm shift; exec @ARGV' -- "$@"; }
>
> It doesn't waste time when the program finishes faster and seems to work
> fine with git grep.
>
> I can't actually test the effectiveness of the patch because PCRE2's
> JIT doesn't work on my development machine at all (Apple M1), as I just
> discovered. :-/  While we know that disabling JIT helps, we didn't
> actually determine, yet, if e0c6029 (Fix inifinite loop when a single
> byte newline is searched in JIT., 2020-05-29) really fixes the "^\s"
> bug.
>
> So I have to abandon this patch, unfortunately.  Any volunteer to pick
> it up?

We can test it in CI, and have a proposed patch from Hamza Mahfooz to do
so. See
https://lore.kernel.org/git/211220.865yrjszg4.gmgdl@evledraar.gmail.com/

There's been some minor changes to the main.yml since then, but I think
you should be able to just pick that patch up, adjust it, apply whatever
changes you want to test on top, and push it to github.