error, verror: Add tests

error, verror: Add tests

[Bruno Haible]

[-- Attachment #1: Type: text/plain, Size: 5208 bytes --]

In a testdir on Android 11, I am seeing these test failures:

FAIL: test-xstrtoimax.sh
========================

--- exp	2023-01-10 14:13:27.438800298 +0100
+++ out	2023-01-10 14:13:27.388800678 +0100
@@ -1,9 +1,9 @@
 1->1 ()
 -1->-1 ()
 1k->1024 ()
-invalid suffix in X argument '99999999999999999999999999999999999999999999999999999999999999999999h'
-X argument '99999999999999999999999999999999999999999999999999999999999999999999' too large
-invalid X argument 'x'
-invalid suffix in X argument '9x'
+ invalid suffix in X argument '99999999999999999999999999999999999999999999999999999999999999999999h'
+ X argument '99999999999999999999999999999999999999999999999999999999999999999999' too large
+ invalid X argument 'x'
+ invalid suffix in X argument '9x'
 010->8 ()
 MiB->1048576 ()
FAIL test-xstrtoimax.sh (exit status: 1)

FAIL: test-xstrtol.sh
=====================

--- expected	2023-01-10 14:13:28.422126166 +0100
+++ out	2023-01-10 14:13:28.378793162 +0100
@@ -1,20 +1,20 @@
 1->1 ()
 -1->-1 ()
 1k->1024 ()
-invalid suffix in X argument '99999999999999999999999999999999999999999999999999999999999999999999h'
-X argument '99999999999999999999999999999999999999999999999999999999999999999999' too large
-invalid X argument 'x'
-invalid suffix in X argument '9x'
+ invalid suffix in X argument '99999999999999999999999999999999999999999999999999999999999999999999h'
+ X argument '99999999999999999999999999999999999999999999999999999999999999999999' too large
+ invalid X argument 'x'
+ invalid suffix in X argument '9x'
 010->8 ()
 MiB->1048576 ()
-invalid suffix in X argument '1bB'
+ invalid suffix in X argument '1bB'
 1->1 ()
-invalid X argument '-1'
+ invalid X argument '-1'
 1k->1024 ()
-invalid suffix in X argument '99999999999999999999999999999999999999999999999999999999999999999999h'
-X argument '99999999999999999999999999999999999999999999999999999999999999999999' too large
-invalid X argument 'x'
-invalid suffix in X argument '9x'
+ invalid suffix in X argument '99999999999999999999999999999999999999999999999999999999999999999999h'
+ X argument '99999999999999999999999999999999999999999999999999999999999999999999' too large
+ invalid X argument 'x'
+ invalid suffix in X argument '9x'
 010->8 ()
 MiB->1048576 ()
-invalid suffix in X argument '1bB'
+ invalid suffix in X argument '1bB'
FAIL test-xstrtol.sh (exit status: 1)

FAIL: test-xstrtoll.sh
======================

--- expected	2023-01-10 14:13:29.378785569 +0100
+++ out	2023-01-10 14:13:29.328785949 +0100
@@ -1,18 +1,18 @@
 1->1 ()
 -1->-1 ()
 1k->1024 ()
-invalid suffix in X argument '99999999999999999999999999999999999999999999999999999999999999999999h'
-X argument '99999999999999999999999999999999999999999999999999999999999999999999' too large
-invalid X argument 'x'
-invalid suffix in X argument '9x'
+ invalid suffix in X argument '99999999999999999999999999999999999999999999999999999999999999999999h'
+ X argument '99999999999999999999999999999999999999999999999999999999999999999999' too large
+ invalid X argument 'x'
+ invalid suffix in X argument '9x'
 010->8 ()
 MiB->1048576 ()
 1->1 ()
-invalid X argument '-1'
+ invalid X argument '-1'
 1k->1024 ()
-invalid suffix in X argument '99999999999999999999999999999999999999999999999999999999999999999999h'
-X argument '99999999999999999999999999999999999999999999999999999999999999999999' too large
-invalid X argument 'x'
-invalid suffix in X argument '9x'
+ invalid suffix in X argument '99999999999999999999999999999999999999999999999999999999999999999999h'
+ X argument '99999999999999999999999999999999999999999999999999999999999999999999' too large
+ invalid X argument 'x'
+ invalid suffix in X argument '9x'
 010->8 ()
 MiB->1048576 ()
FAIL test-xstrtoll.sh (exit status: 1)

FAIL: test-xstrtoumax.sh
========================

--- exp	2023-01-10 14:13:30.125446567 +0100
+++ out	2023-01-10 14:13:30.082113563 +0100
@@ -1,9 +1,9 @@
 1->1 ()
-invalid X argument '-1'
+ invalid X argument '-1'
 1k->1024 ()
-invalid suffix in X argument '99999999999999999999999999999999999999999999999999999999999999999999h'
-X argument '99999999999999999999999999999999999999999999999999999999999999999999' too large
-invalid X argument 'x'
-invalid suffix in X argument '9x'
+ invalid suffix in X argument '99999999999999999999999999999999999999999999999999999999999999999999h'
+ X argument '99999999999999999999999999999999999999999999999999999999999999999999' too large
+ invalid X argument 'x'
+ invalid suffix in X argument '9x'
 010->8 ()
 MiB->1048576 ()
FAIL test-xstrtoumax.sh (exit status: 1)


Looking at the code of xstrtol-error, this seems to indicate that the error()
functions inserts an additional space where it shouldn't.

So, as a first step, let me add a unit test for the 'error' module. And, while
at it, also one for the 'verror' module.


2023-01-12  Bruno Haible  <bruno@clisp.org>

	verror: Add tests.
	* tests/test-verror.sh: New file, based on tests/test-error.sh.
	* tests/test-verror.c: New file, based on tests/test-error.c.
	* modules/verror-tests: New file.

2023-01-12  Bruno Haible  <bruno@clisp.org>

	error: Add tests.
	* tests/test-error.sh: New file.
	* tests/test-error.c: New file.
	* modules/error-tests: New file.


[-- Attachment #2: 0001-error-Add-tests.patch --]
[-- Type: text/x-patch, Size: 5622 bytes --]

From 4545825d5a650954c4885460b719e0978f8a214d Mon Sep 17 00:00:00 2001
From: Bruno Haible <bruno@clisp.org>
Date: Thu, 12 Jan 2023 11:48:27 +0100
Subject: [PATCH 1/2] error: Add tests.

* tests/test-error.sh: New file.
* tests/test-error.c: New file.
* modules/error-tests: New file.
---
 ChangeLog           |  7 ++++
 modules/error-tests | 12 +++++++
 tests/test-error.c  | 84 +++++++++++++++++++++++++++++++++++++++++++++
 tests/test-error.sh | 36 +++++++++++++++++++
 4 files changed, 139 insertions(+)
 create mode 100644 modules/error-tests
 create mode 100644 tests/test-error.c
 create mode 100755 tests/test-error.sh

diff --git a/ChangeLog b/ChangeLog
index da069889d2..841d6e818e 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,10 @@
+2023-01-12  Bruno Haible  <bruno@clisp.org>
+
+	error: Add tests.
+	* tests/test-error.sh: New file.
+	* tests/test-error.c: New file.
+	* modules/error-tests: New file.
+
 2023-01-11  Bruno Haible  <bruno@clisp.org>
 
 	xstrtol-error: Improve comments.
diff --git a/modules/error-tests b/modules/error-tests
new file mode 100644
index 0000000000..c0cef0f9b6
--- /dev/null
+++ b/modules/error-tests
@@ -0,0 +1,12 @@
+Files:
+tests/test-error.sh
+tests/test-error.c
+tests/macros.h
+
+Depends-on:
+
+configure.ac:
+
+Makefile.am:
+TESTS += test-error.sh
+check_PROGRAMS += test-error
diff --git a/tests/test-error.c b/tests/test-error.c
new file mode 100644
index 0000000000..67600d34f9
--- /dev/null
+++ b/tests/test-error.c
@@ -0,0 +1,84 @@
+/* Test of error.h functions.
+   Copyright (C) 2023 Free Software Foundation, Inc.
+
+   This program is free software: you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation, either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <https://www.gnu.org/licenses/>.  */
+
+/* Written by Bruno Haible <bruno@clisp.org>, 2023.  */
+
+#include <config.h>
+
+#include "error.h"
+
+#include <errno.h>
+
+#include "macros.h"
+
+/* Custom function to not show the program name in error messages.  */
+static void
+print_no_progname (void)
+{
+}
+
+int
+main (int argc, char *argv[])
+{
+  /* Test error() function with zero STATUS and zero ERRNUM.  */
+  error (0, 0, "bummer");
+  /* With format string arguments.  */
+  errno = EINVAL; /* should be ignored */
+  error (0, 0, "Zonk %d%d%d is too large", 1, 2, 3);
+  /* With non-ASCII characters.  */
+  error (0, 0, "Pokémon started");
+  /* Verify error_message_count.  */
+  ASSERT (error_message_count == 3);
+
+  /* Test error_at_line() function with zero STATUS and zero ERRNUM.  */
+  error_at_line (0, 0, "d1/foo.c", 10, "invalid blub");
+  error_at_line (0, 0, "d1/foo.c", 10, "invalid blarn");
+  /* Verify error_message_count.  */
+  ASSERT (error_message_count == 5);
+
+  /* Test error_one_per_line.  */
+  error_one_per_line = 1;
+  error_at_line (0, 0, "d1/foo.c", 10, "unsupported glink");
+  /* Another line number.  */
+  error_at_line (0, 0, "d1/foo.c", 13, "invalid brump");
+  /* Another file name.  */
+  error_at_line (0, 0, "d2/foo.c", 13, "unsupported flinge");
+  /* Same file name and same line number => message not shown.  */
+  error_at_line (0, 0, "d2/foo.c", 13, "invalid bark");
+  /* Verify error_message_count.  */
+  ASSERT (error_message_count == 8);
+  error_one_per_line = 0;
+
+  /* Test error_print_progname.  */
+  error_print_progname = print_no_progname;
+  error (0, 0, "hammer");
+  error (0, 0, "boing %d%d%d is too large", 1, 2, 3);
+  error_at_line (0, 0, "d2/bar.c", 11, "bark too loud");
+  /* Verify error_message_count.  */
+  ASSERT (error_message_count == 11);
+  error_print_progname = NULL;
+
+  /* Test error() function with nonzero ERRNUM.  */
+  errno = EINVAL; /* should be ignored */
+  error (0, EACCES, "can't steal");
+  /* Verify error_message_count.  */
+  ASSERT (error_message_count == 12);
+
+  /* Test error() function with nonzero STATUS.  */
+  error (4, 0, "fatal error");
+
+  return 0;
+}
diff --git a/tests/test-error.sh b/tests/test-error.sh
new file mode 100755
index 0000000000..d173e183f4
--- /dev/null
+++ b/tests/test-error.sh
@@ -0,0 +1,36 @@
+#!/bin/sh
+# Test of the 'error' module.
+
+. "${srcdir=.}/init.sh"; path_prepend_ .
+
+${CHECKER} test-error${EXEEXT} > out 2> err
+# Verify the exit code.
+case $? in
+  4) ;;
+  *) Exit 1;;
+esac
+
+# Normalize the stderr output on Windows platforms.
+tr -d '\015' < err | sed 's,.*test-error[.ex]*:,test-error:,' > err2 || Exit 1
+
+# Verify the stderr output.
+compare - err2 <<\EOF || Exit 1
+test-error: bummer
+test-error: Zonk 123 is too large
+test-error: Pokémon started
+test-error:d1/foo.c:10: invalid blub
+test-error:d1/foo.c:10: invalid blarn
+test-error:d1/foo.c:10: unsupported glink
+test-error:d1/foo.c:13: invalid brump
+test-error:d2/foo.c:13: unsupported flinge
+hammer
+boing 123 is too large
+d2/bar.c:11: bark too loud
+test-error: can't steal: Permission denied
+test-error: fatal error
+EOF
+
+# Verify the stdout output.
+test -s out && Exit 1
+
+Exit 0
-- 
2.34.1


[-- Attachment #3: 0002-verror-Add-tests.patch --]
[-- Type: text/x-patch, Size: 6544 bytes --]

From c621f9452f0a4a16a52295ee867cae2b0461b949 Mon Sep 17 00:00:00 2001
From: Bruno Haible <bruno@clisp.org>
Date: Thu, 12 Jan 2023 12:34:43 +0100
Subject: [PATCH 2/2] verror: Add tests.

* tests/test-verror.sh: New file, based on tests/test-error.sh.
* tests/test-verror.c: New file, based on tests/test-error.c.
* modules/verror-tests: New file.
---
 ChangeLog            |   7 +++
 modules/verror-tests |  12 ++++
 tests/test-verror.c  | 127 +++++++++++++++++++++++++++++++++++++++++++
 tests/test-verror.sh |  36 ++++++++++++
 4 files changed, 182 insertions(+)
 create mode 100644 modules/verror-tests
 create mode 100644 tests/test-verror.c
 create mode 100755 tests/test-verror.sh

diff --git a/ChangeLog b/ChangeLog
index 841d6e818e..07f525af08 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,10 @@
+2023-01-12  Bruno Haible  <bruno@clisp.org>
+
+	verror: Add tests.
+	* tests/test-verror.sh: New file, based on tests/test-error.sh.
+	* tests/test-verror.c: New file, based on tests/test-error.c.
+	* modules/verror-tests: New file.
+
 2023-01-12  Bruno Haible  <bruno@clisp.org>
 
 	error: Add tests.
diff --git a/modules/verror-tests b/modules/verror-tests
new file mode 100644
index 0000000000..e02309658a
--- /dev/null
+++ b/modules/verror-tests
@@ -0,0 +1,12 @@
+Files:
+tests/test-verror.sh
+tests/test-verror.c
+tests/macros.h
+
+Depends-on:
+
+configure.ac:
+
+Makefile.am:
+TESTS += test-verror.sh
+check_PROGRAMS += test-verror
diff --git a/tests/test-verror.c b/tests/test-verror.c
new file mode 100644
index 0000000000..c5a62cddec
--- /dev/null
+++ b/tests/test-verror.c
@@ -0,0 +1,127 @@
+/* Test of verror.h functions.
+   Copyright (C) 2023 Free Software Foundation, Inc.
+
+   This program is free software: you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation, either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <https://www.gnu.org/licenses/>.  */
+
+/* Written by Bruno Haible <bruno@clisp.org>, 2023.  */
+
+#include <config.h>
+
+#include "verror.h"
+
+#include <errno.h>
+#include <stdarg.h>
+
+#include "error.h"
+#include "macros.h"
+
+/* Custom function to not show the program name in error messages.  */
+static void
+print_no_progname (void)
+{
+}
+
+static void
+test_zero (const char *format, ...)
+{
+  va_list args;
+
+  va_start (args, format);
+  verror (0, 0, format, args);
+  va_end (args);
+}
+
+static void
+test_zero_at_line (const char *filename, unsigned int lineno,
+                   const char *format, ...)
+{
+  va_list args;
+
+  va_start (args, format);
+  verror_at_line (0, 0, filename, lineno, format, args);
+  va_end (args);
+}
+
+static void
+test_errnum (const char *format, ...)
+{
+  va_list args;
+
+  va_start (args, format);
+  verror (0, EACCES, format, args);
+  va_end (args);
+}
+
+static void
+test_fatal (const char *format, ...)
+{
+  va_list args;
+
+  va_start (args, format);
+  verror (4, 0, format, args);
+  va_end (args);
+}
+
+int
+main (int argc, char *argv[])
+{
+  /* Test verror() function with zero STATUS and zero ERRNUM.  */
+  test_zero ("bummer");
+  /* With format string arguments.  */
+  errno = EINVAL; /* should be ignored */
+  test_zero ("Zonk %d%d%d is too large", 1, 2, 3);
+  /* With non-ASCII characters.  */
+  test_zero ("Pokémon started");
+  /* Verify error_message_count.  */
+  ASSERT (error_message_count == 3);
+
+  /* Test verror_at_line() function with zero STATUS and zero ERRNUM.  */
+  test_zero_at_line ("d1/foo.c", 10, "invalid blub");
+  test_zero_at_line ("d1/foo.c", 10, "invalid blarn");
+  /* Verify error_message_count.  */
+  ASSERT (error_message_count == 5);
+
+  /* Test error_one_per_line.  */
+  error_one_per_line = 1;
+  test_zero_at_line ("d1/foo.c", 10, "unsupported glink");
+  /* Another line number.  */
+  test_zero_at_line ("d1/foo.c", 13, "invalid brump");
+  /* Another file name.  */
+  test_zero_at_line ("d2/foo.c", 13, "unsupported flinge");
+  /* Same file name and same line number => message not shown.  */
+  test_zero_at_line ("d2/foo.c", 13, "invalid bark");
+  /* Verify error_message_count.  */
+  ASSERT (error_message_count == 8);
+  error_one_per_line = 0;
+
+  /* Test error_print_progname.  */
+  error_print_progname = print_no_progname;
+  test_zero ("hammer");
+  test_zero ("boing %d%d%d is too large", 1, 2, 3);
+  test_zero_at_line ("d2/bar.c", 11, "bark too loud");
+  /* Verify error_message_count.  */
+  ASSERT (error_message_count == 11);
+  error_print_progname = NULL;
+
+  /* Test verror() function with nonzero ERRNUM.  */
+  errno = EINVAL; /* should be ignored */
+  test_errnum ("can't steal");
+  /* Verify error_message_count.  */
+  ASSERT (error_message_count == 12);
+
+  /* Test verror() function with nonzero STATUS.  */
+  test_fatal ("fatal error");
+
+  return 0;
+}
diff --git a/tests/test-verror.sh b/tests/test-verror.sh
new file mode 100755
index 0000000000..3652437e94
--- /dev/null
+++ b/tests/test-verror.sh
@@ -0,0 +1,36 @@
+#!/bin/sh
+# Test of the 'verror' module.
+
+. "${srcdir=.}/init.sh"; path_prepend_ .
+
+${CHECKER} test-verror${EXEEXT} > out 2> err
+# Verify the exit code.
+case $? in
+  4) ;;
+  *) Exit 1;;
+esac
+
+# Normalize the stderr output on Windows platforms.
+tr -d '\015' < err | sed 's,.*test-verror[.ex]*:,test-verror:,' > err2 || Exit 1
+
+# Verify the stderr output.
+compare - err2 <<\EOF || Exit 1
+test-verror: bummer
+test-verror: Zonk 123 is too large
+test-verror: Pokémon started
+test-verror:d1/foo.c:10: invalid blub
+test-verror:d1/foo.c:10: invalid blarn
+test-verror:d1/foo.c:10: unsupported glink
+test-verror:d1/foo.c:13: invalid brump
+test-verror:d2/foo.c:13: unsupported flinge
+hammer
+boing 123 is too large
+d2/bar.c:11: bark too loud
+test-verror: can't steal: Permission denied
+test-verror: fatal error
+EOF
+
+# Verify the stdout output.
+test -s out && Exit 1
+
+Exit 0
-- 
2.34.1

Re: error, verror: Add tests

[Bruno Haible]

> 2023-01-12  Bruno Haible  <bruno@clisp.org>
> 
> 	verror: Add tests.
> 	* tests/test-verror.sh: New file, based on tests/test-error.sh.
> 	* tests/test-verror.c: New file, based on tests/test-error.c.
> 	* modules/verror-tests: New file.
> 
> 2023-01-12  Bruno Haible  <bruno@clisp.org>
> 
> 	error: Add tests.
> 	* tests/test-error.sh: New file.
> 	* tests/test-error.c: New file.
> 	* modules/error-tests: New file.

Note that the tests do *not* test the situation of error_at_line with a NULL
file name argument, since this is not described as supported in the glibc
documentation.


2023-01-12  Bruno Haible  <bruno@clisp.org>

	error, verror tests: Add clarification.
	* tests/test-error.c (main): Clarify why we don't test error_at_line
	with a NULL filename.
	* tests/test-verror.c (main): Clarify why we don't test verror_at_line
	with a NULL filename.

diff --git a/tests/test-error.c b/tests/test-error.c
index 67600d34f9..697e783eee 100644
--- a/tests/test-error.c
+++ b/tests/test-error.c
@@ -66,6 +66,10 @@ main (int argc, char *argv[])
   error_print_progname = print_no_progname;
   error (0, 0, "hammer");
   error (0, 0, "boing %d%d%d is too large", 1, 2, 3);
+  #if 0
+  /* The documentation does not describe the output if the file name is NULL. */
+  error_at_line (0, 0, NULL, 42, "drummer too loud");
+  #endif
   error_at_line (0, 0, "d2/bar.c", 11, "bark too loud");
   /* Verify error_message_count.  */
   ASSERT (error_message_count == 11);
diff --git a/tests/test-verror.c b/tests/test-verror.c
index c5a62cddec..7c808d7908 100644
--- a/tests/test-verror.c
+++ b/tests/test-verror.c
@@ -109,6 +109,10 @@ main (int argc, char *argv[])
   error_print_progname = print_no_progname;
   test_zero ("hammer");
   test_zero ("boing %d%d%d is too large", 1, 2, 3);
+  #if 0
+  /* The documentation does not describe the output if the file name is NULL. */
+  test_zero_at_line (NULL, 42, "drummer too loud");
+  #endif
   test_zero_at_line ("d2/bar.c", 11, "bark too loud");
   /* Verify error_message_count.  */
   ASSERT (error_message_count == 11);

Re: error, verror: Add tests

[Bruno Haible]

Building an m4 snapshot on FreeBSD, with GNU gettext installed, I see a
link error of 'test-error', regarding the symbol 'libintl_gettext'.
This patch should fix it.


2023-01-14  Bruno Haible  <bruno@clisp.org>

	error, verror tests: Fix link error when the package uses libintl.
	* modules/error-tests (Makefile.am): Link test-error with $(LIBINTL).
	* modules/verror-tests (Makefile.am): Link test-verror with $(LIBINTL).

diff --git a/modules/error-tests b/modules/error-tests
index c0cef0f9b6..58369089e3 100644
--- a/modules/error-tests
+++ b/modules/error-tests
@@ -10,3 +10,4 @@ configure.ac:
 Makefile.am:
 TESTS += test-error.sh
 check_PROGRAMS += test-error
+test_error_LDADD = $(LDADD) $(LIBINTL)
diff --git a/modules/verror-tests b/modules/verror-tests
index e02309658a..bd76d768c1 100644
--- a/modules/verror-tests
+++ b/modules/verror-tests
@@ -10,3 +10,4 @@ configure.ac:
 Makefile.am:
 TESTS += test-verror.sh
 check_PROGRAMS += test-verror
+test_verror_LDADD = $(LDADD) $(LIBINTL)