Re: possible bug in regex and dfa

[Bruno Haible <bruno@clisp.org>]

Hi Arnold,

> Dot matching newline isn't the issue here.
> 
> It's ^ matching in the middle of a string.  For my purposes, ^ should
> only match at the beginning of a *string* (as $ should only match at
> the end of a string).  I haven't rechecked POSIX, but this is how awk
> has behaved since forever.

Hmm. Regarding POSIX: I've read section 9.3.8 and 9.4.9 of [1],
the description of REG_NOTBOL, REG_NOTEOL in [2], and the description
of REG_NEWLINE in [3]. If I understand it correctly, within POSIX,
".^" should not match a newline because
  - if REG_NEWLINE is set, '^' matches after the newline but '.' does not
    match the newline,
  - if REG_NEWLINE is not set, '.' matches newline but '^' does not match
    after the newline.

However, GNU regex.h also has a flag RE_CONTEXT_INDEP_ANCHORS; I don't know
what effect it has.

> (And how I've documented things in the manual, also since forever.)

If you want the behaviour of the GNU regex to be stable over time, you
should contribute unit tests to tests/test-regex.c. So far, I see unit
tests for the flags
  REG_EXTENDED
  REG_NOSUB
  RE_SYNTAX_POSIX_BASIC
  RE_SYNTAX_GREP
  RE_SYNTAX_EGREP
  RE_SYNTAX_POSIX_EGREP
  RE_SYNTAX_EMACS
  RE_HAT_LISTS_NOT_NEWLINE
  RE_ICASE
  RE_CONTEXT_INVALID_DUP
  RE_NO_EMPTY_RANGES
but no tests at all for
  RE_SYNTAX_AWK
  RE_SYNTAX_GNU_AWK
  RE_SYNTAX_POSIX_AWK
  RE_SYNTAX_POSIX_EXTENDED
  REG_NEWLINE
  REG_NOTBOL
  REG_NOTEOL
  REG_STARTEND

Usually it takes about as many code lines to reasonably unit test some code
as the code itself has. The regex module is over 300 KB large, and its unit
test less than 20 KB large. Already from these figures you can tell that
the regex module is *NEARLY UNTESTED*.

You may say, well, some other unit tests exist in glibc, in sed, in grep,
in awk, in coreutils, etc. But it doesn't help maintenance if the unit tests
are not part of what gets tested by
  ./gnulib-tool --test --single-configure regex

Bruno

[1] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html
[2] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/regex.h.html
[3] https://pubs.opengroup.org/onlinepubs/9699919799/functions/regexec.html