add script for running tests under valgrind

add script for running tests under valgrind

[Bruno Haible]

This patch does two things:
  * It adds a script for running tests under valgrind, that I developed for
    GNU libunistring and want to use elsewhere as well, to gnulib.
  * It revamps the documentation section "Running self-tests under valgrind"
    to mention alternative approaches as well.


2019-02-04  Bruno Haible  <bruno@clisp.org>

	Add script for running tests under valgrind.
	* build-aux/run-test: New file, from GNU libunistring.
	* doc/valgrind-tests.texi: Rewritten to mention alternative approaches
	as well.

diff --git a/build-aux/run-test b/build-aux/run-test
new file mode 100755
index 0000000..87432c7
--- /dev/null
+++ b/build-aux/run-test
@@ -0,0 +1,148 @@
+#!/bin/sh
+#
+# Copyright (C) 2009-2019 Free Software Foundation, Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+#
+
+# This program is a test driver that supports running a test under valgrind.
+# Usage: run-test CHECKER PROGRAM [ARGUMENT...]
+
+progname=$0
+
+# func_usage
+# outputs to stdout the --help usage message.
+func_usage ()
+{
+  echo "\
+Usage: run-test [OPTION...] CHECKER PROGRAM [ARGUMENT...]
+
+Runs PROGRAM under the control of CHECKER.
+
+CHECKER may be empty or a valgrind command with some options, such as
+'valgrind --tool=memcheck --num-callers=20 --leak-check=yes --leak-resolution=high --show-reachable=yes'.
+
+When CHECKER is not empty, it is recommended that the package has been
+configured with
+  --disable-shared     so that tests are real executables and not libtool
+                       wrapper scripts, and
+  CFLAGS=\"-g\"          so that valgrind shows line numbers.
+
+Report bugs to Bruno Haible."
+}
+
+# func_version
+# outputs to stdout the --version message.
+func_version ()
+{
+  echo "\
+run-test (GNU gnulib)
+Copyright (C) 2009-2019 Free Software Foundation, Inc.
+License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>
+This is free software: you are free to change and redistribute it.
+There is NO WARRANTY, to the extent permitted by law.
+
+Written by" "Bruno Haible"
+}
+
+# func_fatal_error message
+# outputs to stderr a fatal error message, and terminates the program.
+# Input:
+# - progname                 name of this program
+func_fatal_error ()
+{
+  echo "$progname: *** $1" 1>&2
+  echo "$progname: *** Stop." 1>&2
+  func_exit 1
+}
+
+# Command-line option processing.
+# Removes the OPTIONS from the arguments. Sets the variables:
+# - checker        wrapper program for executables
+{
+  while test $# -gt 0; do
+    case "$1" in
+      --help | --hel | --he | --h )
+        func_usage
+        exit $? ;;
+      --version | --versio | --versi | --vers | --ver | --ve | --v )
+        func_version
+        exit $? ;;
+      -- )
+        # Stop option processing
+        shift
+        break ;;
+      -* )
+        echo "run-test: unknown option $1" 1>&2
+        echo "Try 'run-test --help' for more information." 1>&2
+        exit 1 ;;
+      * )
+        break ;;
+    esac
+  done
+
+  if test $# -lt 2; then
+    echo "run-test: too few arguments" 1>&2
+    echo "Try 'run-test --help' for more information." 1>&2
+    exit 1
+  fi
+
+  checker="$1"
+  shift
+}
+
+if test -z "$checker"; then
+  # No checker. Run the test directly.
+  case "$1" in
+    *.sh)
+     # Support environments where sh exists but not /bin/sh.
+     exec sh "$@"
+     ;;
+    *)
+     exec "$@"
+     ;;
+  esac
+else
+  # Using valgrind. We want to apply valgrind only to executables, not to
+  # shell scripts, because
+  # 1. we don't want to look for memory leaks in bash,
+  # 2. on a bi-arch system, we would get an error message such as
+  #    "valgrind: wrong executable class (eg. 32-bit instead of 64-bit)".
+  case "$1" in
+    *.sh)
+      # A shell script. Ignore the checker.
+      # Support environments where sh exists but not /bin/sh.
+      exec sh "$@"
+      ;;
+    *)
+      # The 'file' command is not portable enough. So, look
+      # at the first two bytes of the file. Are they '#!'?
+      if { if od -A x < /dev/null >/dev/null 2>/dev/null; then
+             # Use POSIX od.
+             firstbytes=`od -A n -t o1 -N 2 < "$1" | tr -d ' '`
+           else
+             # Use BSD hexdump.
+             firstbytes=`dd if="$1" bs=1 count=2 2>/dev/null | hexdump -e '1/1 "%03o"'`
+           fi
+           test "$firstbytes" = "043041"
+         }; then
+        # A shell script. Ignore the checker.
+        exec "$@"
+      else
+        # An executable. Use the checker.
+        exec $checker "$@"
+      fi
+      ;;
+  esac
+fi
diff --git a/doc/valgrind-tests.texi b/doc/valgrind-tests.texi
index ec37202..fef910e 100644
--- a/doc/valgrind-tests.texi
+++ b/doc/valgrind-tests.texi
@@ -1,10 +1,25 @@
 @node Running self-tests under valgrind
 @section Running self-tests under valgrind
 
+@cindex valgrind
+
 For projects written in C or similar languages, running the self-tests
-under Valgrind can reveal hard to find memory issues.  The
-@code{valgrind-tests} module searches for Valgrind and declares the
-@code{VALGRIND} automake variable for use with automake's
+under Valgrind can reveal hard to find memory issues.  Gnulib supports
+two ways to make use of Valgrind: one that enables use of Valgrind at
+configure time, when @code{configure} found it to be present; and one
+at the discretion of the developer.
+
+@menu
+* Using valgrind automatically::
+* Using valgrind manually::
+* Valgrind and shell scripts::
+@end menu
+
+@node Using valgrind automatically
+@subsection Using valgrind without developer intervention
+
+The @code{valgrind-tests} module searches for Valgrind at configure time
+and declares the @code{VALGRIND} automake variable for use with automake's
 @code{TESTS_ENVIRONMENT}.
 
 After importing the @code{valgrind-tests} module to your project, you
@@ -15,10 +30,44 @@ self-tests:
 TESTS_ENVIRONMENT = $(VALGRIND)
 @end smallexample
 
-This will run all self-checks under valgrind.  This can be wasteful if
-you have many shell scripts or other non-binaries.  Using the Automake
-parallel-tests feature, this can be avoided by using the following
-instead:
+This will run all self-checks under valgrind.
+
+@node Using valgrind manually
+@subsection Using valgrind at the developer's discretion
+
+In this approach, you define a @code{Makefile.am} variable @samp{VALGRIND}
+(or, more abstractly, @samp{CHECKER}), that is usually set to empty.
+When you have configured and built the package and you decide that you want
+to run the tests with valgrind, you do so by modifying the definition of
+@samp{VALGRIND} in the Makefile.
+
+@node Valgrind and shell scripts
+@subsection How to use Valgrind with shell scripts
+
+It is not desirable to apply valgrind to shell scripts or other non-binaries,
+because
+@itemize @bullet
+@item
+It is wasteful, and you usually don't want to look for memory leaks in bash.
+@item
+On a bi-arch system, you may get an error message such as
+"valgrind: wrong executable class (eg. 32-bit instead of 64-bit)".
+@end itemize
+
+There are two ways to avoid this:
+
+@itemize @bullet
+@item
+You can make use of the @code{build-aux/run-test} script from Gnulib.
+Add these lines to your @code{Makefile.am}:
+
+@smallexample
+# This must be the last thing that gets added to TESTS_ENVIRONMENT.
+TESTS_ENVIRONMENT += $(SHELL) $(top_srcdir)/build-aux/run-test '$(VALGRIND)'
+@end smallexample
+
+@item
+Using the Automake parallel-tests feature, you can use the following instead:
 
 @smallexample
 AUTOMAKE_OPTIONS = parallel-tests
@@ -27,8 +76,12 @@ LOG_COMPILER = $(VALGRIND)
 @end smallexample
 
 Then valgrind will only be used for the non-.sh and non-.pl tests.
-However, this means that binaries invoked through scripts will not be
-invoked under valgrind, which could be solved by adding the following:
+@end itemize
+
+However, with this measure in place, binaries invoked through scripts will
+not be invoked under valgrind.  This can be solved by defining environment
+variables in the @code{TESTS_ENVIRONMENT} variable that are then used by the
+shell scripts.  For example, add the following:
 
 @smallexample
 TESTS_ENVIRONMENT = VALGRIND='$(VALGRIND)'