[PATCH] test-lib-functions: avoid non POSIX ERE in test_dir_is_empty()

[PATCH] test-lib-functions: avoid non POSIX ERE in test_dir_is_empty()

[Carlo Marcelo Arenas Belón]

0be7d9b73d (test-lib: add test_dir_is_empty(), 2014-06-19) uses an
ERE through the egrep tool (which is not POSIX) using an ? operator
that isn't either.

replace invocation with two equivalent simpler BRE instead.

Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
---
 t/test-lib-functions.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index e28411bb75..2803c97df3 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -790,7 +790,7 @@ test_path_exists () {
 test_dir_is_empty () {
 	test "$#" -ne 1 && BUG "1 param"
 	test_path_is_dir "$1" &&
-	if test -n "$(ls -a1 "$1" | egrep -v '^\.\.?$')"
+	if test -n "$(ls -a1 "$1" | grep -v '^\.$' | grep -v '^\.\.$')"
 	then
 		echo "Directory '$1' is not empty, it contains:"
 		ls -la "$1"
-- 
2.33.0.481.g26d3bed244

Re: [PATCH] test-lib-functions: avoid non POSIX ERE in test_dir_is_empty()

[Taylor Blau]

On Wed, Aug 25, 2021 at 08:17:10PM -0700, Carlo Marcelo Arenas Belón wrote:
> 0be7d9b73d (test-lib: add test_dir_is_empty(), 2014-06-19) uses an
> ERE through the egrep tool (which is not POSIX) using an ? operator
> that isn't either.
>
> replace invocation with two equivalent simpler BRE instead.
>
> Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
> ---
>  t/test-lib-functions.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
> index e28411bb75..2803c97df3 100644
> --- a/t/test-lib-functions.sh
> +++ b/t/test-lib-functions.sh
> @@ -790,7 +790,7 @@ test_path_exists () {
>  test_dir_is_empty () {
>  	test "$#" -ne 1 && BUG "1 param"
>  	test_path_is_dir "$1" &&
> -	if test -n "$(ls -a1 "$1" | egrep -v '^\.\.?$')"
> +	if test -n "$(ls -a1 "$1" | grep -v '^\.$' | grep -v '^\.\.$')"

This replacement is correct, but I'm not sure that I necessarily find it
simpler. If we really are concerned about egrep usage, then

    if test -n "$(find "$1" | grep -v '^\.$')"

would suffice. But it looks like we are fairly OK with egrep in t (`git
grep 'egrep' -- t | wc -l` turns up 19 matches), so I'm not sure the
change is necessary in the first place.

Thanks,
Taylor

Re: [PATCH] test-lib-functions: avoid non POSIX ERE in test_dir_is_empty()

[Junio C Hamano]

Taylor Blau <me@ttaylorr.com> writes:

>> -	if test -n "$(ls -a1 "$1" | egrep -v '^\.\.?$')"
>> +	if test -n "$(ls -a1 "$1" | grep -v '^\.$' | grep -v '^\.\.$')"
>
> This replacement is correct, but I'm not sure that I necessarily find it
> simpler. If we really are concerned about egrep usage, then
>
>     if test -n "$(find "$1" | grep -v '^\.$')"
>
> would suffice. But it looks like we are fairly OK with egrep in t (`git
> grep 'egrep' -- t | wc -l` turns up 19 matches), so I'm not sure the
> change is necessary in the first place.

It is true that we have been OK with egrep.

"grep -E" is in POSIX (so is "grep -F") and that you might be able
to make an argument to use them instead of "egrep" and "fgrep", but
at least at one point in the past, 87539416 (tests: grep portability
fixes, 2008-09-30) favoured "fgrep" over "grep -F", claiming that
the former was more widely available than the latter.

The world may have changed since then, though.  IIRC, the 2008's
topic was mostly about portability to Solaris and AIX, and their
peculiarity (or their relevance) may have waned.

Re: [PATCH] test-lib-functions: avoid non POSIX ERE in test_dir_is_empty()

[Carlo Arenas]

On Wed, Aug 25, 2021 at 9:24 PM Taylor Blau <me@ttaylorr.com> wrote:
> On Wed, Aug 25, 2021 at 08:17:10PM -0700, Carlo Marcelo Arenas Belón wrote:
> > diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
> > index e28411bb75..2803c97df3 100644
> > --- a/t/test-lib-functions.sh
> > +++ b/t/test-lib-functions.sh
> > @@ -790,7 +790,7 @@ test_path_exists () {
> >  test_dir_is_empty () {
> >       test "$#" -ne 1 && BUG "1 param"
> >       test_path_is_dir "$1" &&
> > -     if test -n "$(ls -a1 "$1" | egrep -v '^\.\.?$')"
> > +     if test -n "$(ls -a1 "$1" | grep -v '^\.$' | grep -v '^\.\.$')"
>
> This replacement is correct, but I'm not sure that I necessarily find it
> simpler. If we really are concerned about egrep usage, then
>
>     if test -n "$(find "$1" | grep -v '^\.$')"

Interesting idea; if having a much simpler expression is so important
then we could do instead [^.], since all use cases will be covered
(nobody is going to create a three dotted file to workaround a test
IMHO)

> But it looks like we are fairly OK with egrep in t (`git
> grep 'egrep' -- t | wc -l` turns up 19 matches), so I'm not sure the
> change is necessary in the first place.

egrep (and also fgrep, which we intentionally support because it is
missing from some ancient AIX system[1]) will be removed in the
next[2] release of GNU grep.

Carlo

[1] 87539416fd (tests: grep portability fixes, 2008-09-30)
[2] https://git.savannah.gnu.org/cgit/grep.git/commit/?id=a9515624709865d480e3142fd959bccd1c9372d1

Re: [PATCH] test-lib-functions: avoid non POSIX ERE in test_dir_is_empty()

[Carlo Arenas]

On Wed, Aug 25, 2021 at 11:25 PM Junio C Hamano <gitster@pobox.com> wrote:
> Taylor Blau <me@ttaylorr.com> writes:
> >> -    if test -n "$(ls -a1 "$1" | egrep -v '^\.\.?$')"
> >> +    if test -n "$(ls -a1 "$1" | grep -v '^\.$' | grep -v '^\.\.$')"
> >
> > This replacement is correct, but I'm not sure that I necessarily find it
> > simpler. If we really are concerned about egrep usage, then
> >
> >     if test -n "$(find "$1" | grep -v '^\.$')"
> >
> > would suffice. But it looks like we are fairly OK with egrep in t (`git
> > grep 'egrep' -- t | wc -l` turns up 19 matches), so I'm not sure the
> > change is necessary in the first place.
>
> It is true that we have been OK with egrep.

note that will be an issue at least in a future release of GNU grep
where the deprecated {e,f}grep commands won't be available.

> "grep -E" is in POSIX (so is "grep -F") and that you might be able
> to make an argument to use them instead of "egrep" and "fgrep".

I misread the POSIX specification, and had confirmed that the ?
operator works fine in BSD and busybox grep so apologies and
will see to resubmit this change together with the others that will
be required to deal with the deprecated binaries.

> at least at one point in the past, 87539416 (tests: grep portability
> fixes, 2008-09-30) favoured "fgrep" over "grep -F", claiming that
> the former was more widely available than the latter.
>
> The world may have changed since then, though.  IIRC, the 2008's
> topic was mostly about portability to Solaris and AIX, and their
> peculiarity (or their relevance) may have waned.

was hoping to deal with fgrep using a function (just like is done with
perl), but was also hoping that a compatibility layer wouldn't be
needed.
testing on AIX (that seems to stick around) will definitely be needed.

Carlo

Re: [PATCH] test-lib-functions: avoid non POSIX ERE in test_dir_is_empty()

[Junio C Hamano]

Carlo Arenas <carenas@gmail.com> writes:

> egrep (and also fgrep, which we intentionally support because it is
> missing from some ancient AIX system[1]) will be removed in the
> next[2] release of GNU grep.

It is not a reason not to nudge us to prepare for the eventual
removal of these two commands, but we need to keep the facts
straight in our log messages.  

The way I read [2], they will only start giving a warning message
nudging the users to use "grep -[EF]", and for them to be able to
warn, I would imagine they have to stay in the release without
getting removed.

> [1] 87539416fd (tests: grep portability fixes, 2008-09-30)
> [2] https://git.savannah.gnu.org/cgit/grep.git/commit/?id=a9515624709865d480e3142fd959bccd1c9372d1

There are about 35 places in t/ we call egrep or fgrep; if we can
add a pair of replacement shell functions in t/test-lib.sh for them
to use whatever command GIT_TEST_EGREP and GIT_TEST_FGREP
environment variables specify and fall back on "grep -E/-F"
otherwise, we can prepare for the change without too much code
churning.

Something along the lines of

	# in test-lib-functions.sh
        : ${GIT_TEST_EGREP:=grep -E} ${GIT_TEST_FGREP:=grep -F}
        egrep () { $GIT_TEST_EGREP "$@" }
        fgrep () { $GIT_TEST_FGREP "$@" }

where people could do something silly like

	GIT_TEST_FGREP='command fgrep' \
	GIT_TEST_EGREP='command egrep' \
	make test

perhaps?