From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS22989 209.51.188.0/24 X-Spam-Status: No, score=-4.0 required=3.0 tests=AWL,BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 240EC1F5AE for ; Fri, 14 May 2021 12:51:54 +0000 (UTC) Received: from localhost ([::1]:33660 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lhXIG-0005Ai-T3 for normalperson@yhbt.net; Fri, 14 May 2021 08:51:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58602) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lhWvM-0003Vg-5v for bug-gnulib@gnu.org; Fri, 14 May 2021 08:28:12 -0400 Received: from uggla.sjd.se ([2001:9b1:8633::107]:33582) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lhWvA-000344-5X for bug-gnulib@gnu.org; Fri, 14 May 2021 08:28:11 -0400 DKIM-Signature: v=1; a=ed25519-sha256; q=dns/txt; c=relaxed/relaxed; d=josefsson.org; s=ed2101; h=Content-Type:MIME-Version:Message-ID:Date: Subject:To:From:Sender:Reply-To:Cc:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=YCUNLNo7kaNRA7y97mG+4oIvbpB8Y9m7vLuUEeMuLXE=; t=1620995277; x=1622204877; b=GdEJcFgXl4uV84qtK6xlCcQSWma3CZj2RjP9LO5eAC68D1YXkWuCxNcfjCE3vLqIcQTFRGFrxf 8ZDj/5ZhdMDA==; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=josefsson.org; s=rsa2101; h=Content-Type:MIME-Version:Message-ID:Date: Subject:To:From:Sender:Reply-To:Cc:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=YCUNLNo7kaNRA7y97mG+4oIvbpB8Y9m7vLuUEeMuLXE=; t=1620995277; x=1622204877; b=AZ+sDkgrybv2qjpDHJcFvh8saCCt0jIZFq5jSAlPxDZtwLRH3ZKpOfn1ZZxMJdWLQau4WEK9kC rsEkqZNHbjdBdTx91MsAuhtMN2T04R6X66Hn7SzgW1cSec5qxMQXo0Hm1w69v+MAk0QDaa5enVLR/ lBOQhXDmaBTnJtqCtTOzaQDycqsIoj+9BqOQjgQcFFGsUORePdbGbXiiZsWiPAXE7sqwNBq+DrWUt nYfTshWuETJb64ibH9s5JygYAuCD28ysqKGehofdzmxeQTlVrORkLO29Gj1zeAkliQGPQezhEpHx+ iFRMmgQI00Jo/8UetbuRvvdHNUDSGGR8UhKdUIf1w+3aBMsiB16D/Tc3M2lD3C+rMM3aewQiNohw9 tZFhhQYhH7zv60kHZe5aqResZ12ubK24b8uk5zRDzU2IfFpJRo5DveO0rQaTDV6EOuLdDKWrtb ; Received: from [2001:9b1:41ac:ff00:d573:7565:e01a:ce5b] (port=60772 helo=latte) by uggla.sjd.se with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lhWv5-0002l0-8N for bug-gnulib@gnu.org; Fri, 14 May 2021 12:27:55 +0000 X-Hashcash: 1:22:210514:bug-gnulib@gnu.org::lHNaMcmLpgURVUBU:FshC To: bug-gnulib@gnu.org Subject: [PATCH] valgrind-tests: Better option handling. OpenPGP: id=B1D2BD1375BECB784CF4F8C4D73CF638C53C06BE; url=https://josefsson.org/key-20190320.txt Date: Fri, 14 May 2021 14:27:54 +0200 Message-ID: <87lf8hxzxx.fsf@latte.josefsson.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Received-SPF: pass client-ip=2001:9b1:8633::107; envelope-from=simon@josefsson.org; helo=uggla.sjd.se X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: bug-gnulib@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Gnulib discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnulib-bounces+normalperson=yhbt.net@gnu.org Sender: "bug-gnulib" Reply-to: Simon Josefsson From: Simon Josefsson via Gnulib discussion list --==-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain Hi. I have pushed the attached patch. It should be completely backwards compatible, or there is a bug, so please test this. It adds two new variables to allow both developers and users to influence the valgrind options used. You may want to re-read the manual, I updated it to assume people use the parallel automake test harness today. Developers may want to use 'LOG_COMPILER=$(LOG_VALGRIND)' instead of 'LOG_COMPILER=$(VALGRIND)' in your Makefile.am, to make it easier for end-users to modify the options passed to valgrind. The old style still works fine, and in the same way as before. The reason for the change is to enable end-users to be able to change VALGRINDFLAGS during a 'make check' phase if they want. /Simon --=-=-= Content-Type: text/x-diff; charset=iso-8859-1 Content-Disposition: inline; filename=0001-valgrind-tests-Better-option-handling.patch Content-Transfer-Encoding: quoted-printable From=20e0a1ea5e1d20918f6850a2ab8cb86c691cd8b46c Mon Sep 17 00:00:00 2001 From: Simon Josefsson Date: Fri, 14 May 2021 14:17:20 +0200 Subject: [PATCH] valgrind-tests: Better option handling. * m4/valgrind-tests.m4: Support new variables VALGRINDFLAGS and DEFAULT_VALGRINDFLAGS. * doc/valgrind-tests.texi (Running self-tests under valgrind): Improve. =2D-- ChangeLog | 8 +++ doc/valgrind-tests.texi | 105 +++++++++++++++++++++++++++++++++++----- m4/valgrind-tests.m4 | 67 +++++++++++++++++-------- 3 files changed, 148 insertions(+), 32 deletions(-) diff --git a/ChangeLog b/ChangeLog index 4716f78a8..daeb44ab4 100644 =2D-- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,11 @@ +2021-05-14 Simon Josefsson + + valgrind-tests: Better option handling. + * m4/valgrind-tests.m4: Support new variables VALGRINDFLAGS and + DEFAULT_VALGRINDFLAGS. + * doc/valgrind-tests.texi (Running self-tests under valgrind): + Improve. + 2021-05-14 Markus M=FCtzel (tiny change) =20 windows-spawn: Don't assume that UNICODE is not defined. diff --git a/doc/valgrind-tests.texi b/doc/valgrind-tests.texi index fef910edf..9f3b30968 100644 =2D-- a/doc/valgrind-tests.texi +++ b/doc/valgrind-tests.texi @@ -11,6 +11,7 @@ at the discretion of the developer. =20 @menu * Using valgrind automatically:: +* Valgrind options:: * Using valgrind manually:: * Valgrind and shell scripts:: @end menu @@ -19,19 +20,93 @@ at the discretion of the developer. @subsection Using valgrind without developer intervention =20 The @code{valgrind-tests} module searches for Valgrind at configure time =2Dand declares the @code{VALGRIND} automake variable for use with automake= 's =2D@code{TESTS_ENVIRONMENT}. +and declares the @code{LOG_VALGRIND} automake variable for use with +automake's @code{LOG_COMPILER}. =20 After importing the @code{valgrind-tests} module to your project, you use it by adding the following to the @code{Makefile.am} that runs the self-tests: =20 @smallexample =2DTESTS_ENVIRONMENT =3D $(VALGRIND) +LOG_COMPILER =3D $(LOG_VALGRIND) @end smallexample =20 This will run all self-checks under valgrind. =20 +Replace @code{LOG_COMPILER} with @code{TESTS_ENVIRONMENT} if you are +using the old serial test harness. + +If you desire a project-wide decision that valgrind is not enabled by +default, but still allow users to enable it with +@code{--enable-valgrind-tests} you may put the following in configure.ac +before gl_INIT. + +@smallexample +gl_VALGRIND_TESTS_DEFAULT_NO +@end smallexample + +@node Valgrind options +@subsection Valgrind options + +The @code{VALGRIND} variable holds the name of the valgrind binary and +some options passed to valgrind. You may provide additional options +that are passed to valgrind using the @samp{VALGRINDFLAGS} variable, for +example: + +@smallexample +./configure VALGRINDFLAGS=3D"--suppressions=3D/your/local/valgrind/suppres= sions/file.txt" +@end smallexample + +Alternatively during build phase: + +@smallexample +make check VALGRINDFLAGS=3D"--suppressions=3D/your/local/valgrind/suppress= ions/file.txt" +@end smallexample + +This is useful if you have a valgrind suppression files that are needed +to avoid triggering errors for known errors, typically in system +libraries. + +The @code{VALGRIND} variable include options that are useful when +valgrind is run non-interactively through the test harness. The default +parameters are @code{-q} to silence the output, +@code{--error-exitcode=3D1} to cause valgrind errors to be treated as +fatal errors, and @code{--leak-check=3Dfull} to check for memory leaks. + +These options can be controlled through the @code{DEFAULT_VALGRINDFLAGS} +variable. For example, when configuring the package: + +@smallexample +./configure DEFAULT_VALGRINDFLAGS=3D"--quiet" +@end smallexample + +Alternatively, during the build phase: + +@smallexample +make check DEFAULT_VALGRINDFLAGS=3D"--quiet" +@end smallexample + +That would have the effect of removing @code{--error-exitcode=3D1} and +@code{--leak-check=3Dfull} from the default options, thus causing any +valgrind errors to be silently ignored, instead of causing fatal test +failures. + +As a developer you may use the variables in @code{configure.ac} before +calling @code{gl_INIT}, like this if your program has deeply-nested call +chains: + +@smallexample +gl_EARLY +... +VALGRINDFLAGS=3D"$VALGRINDFLAGS --num-callers=3D42" +... +gl_INIT +@end smallexample + +Note that any user-supplied @code{VALGRINDFLAGS} value is preserved, +which is usually what you want. + + @node Using valgrind manually @subsection Using valgrind at the developer's discretion =20 @@ -58,24 +133,28 @@ There are two ways to avoid this: =20 @itemize @bullet @item =2DYou can make use of the @code{build-aux/run-test} script from Gnulib. =2DAdd these lines to your @code{Makefile.am}: +Using the Automake parallel-tests feature, you can use the following inste= ad: =20 @smallexample =2D# This must be the last thing that gets added to TESTS_ENVIRONMENT. =2DTESTS_ENVIRONMENT +=3D $(SHELL) $(top_srcdir)/build-aux/run-test '$(VALG= RIND)' +TEST_EXTENSIONS =3D .pl .sh +LOG_COMPILER =3D $(LOG_VALGRIND) @end smallexample =20 +Then valgrind will only be used for the non-.sh and non-.pl tests. + +For old automake, you will need @code{AUTOMAKE_OPTIONS =3D parallel-tests} +to enable the parallel test harness. + @item =2DUsing the Automake parallel-tests feature, you can use the following ins= tead: +You can make use of the @code{build-aux/run-test} script from Gnulib. +Add these lines to your @code{Makefile.am}: =20 @smallexample =2DAUTOMAKE_OPTIONS =3D parallel-tests =2DTEST_EXTENSIONS =3D .pl .sh =2DLOG_COMPILER =3D $(VALGRIND) +LOG_COMPILER +=3D $(SHELL) $(top_srcdir)/build-aux/run-test '$(LOG_VALGRIN= D)' @end smallexample =20 =2DThen valgrind will only be used for the non-.sh and non-.pl tests. +Replace @code{LOG_COMPILER} with @code{TESTS_ENVIRONMENT} if you use the +old serial test harness. @end itemize =20 However, with this measure in place, binaries invoked through scripts will @@ -84,7 +163,7 @@ variables in the @code{TESTS_ENVIRONMENT} variable that = are then used by the shell scripts. For example, add the following: =20 @smallexample =2DTESTS_ENVIRONMENT =3D VALGRIND=3D'$(VALGRIND)' +TESTS_ENVIRONMENT =3D VALGRIND=3D'$(LOG_VALGRIND)' @end smallexample =20 And then modify the shell scripts to invoke the binary prefixed with diff --git a/m4/valgrind-tests.m4 b/m4/valgrind-tests.m4 index 50d90e145..9fc9d5b37 100644 =2D-- a/m4/valgrind-tests.m4 +++ b/m4/valgrind-tests.m4 @@ -1,4 +1,4 @@ =2D# valgrind-tests.m4 serial 6 +# valgrind-tests.m4 serial 7 dnl Copyright (C) 2008-2021 Free Software Foundation, Inc. dnl This file is free software; the Free Software Foundation dnl gives unlimited permission to copy and/or distribute it, @@ -8,18 +8,60 @@ dnl From Simon Josefsson =20 # gl_VALGRIND_TESTS() # ------------------- =2D# Check if valgrind is available, and set VALGRIND to it if available. =2DAC_DEFUN([gl_VALGRIND_TESTS], +# Check if valgrind is available. + +# Sets VALGRIND to command line (including options) to invoke valgrind +# with, may be used directly in autoconf, makefiles or shell scripts. + +# Sets LOG_VALGRIND, suitable for use with LOG_COMPILER, that in a +# makefile will expand to command line to invoke self-tests with, +# i.e., LOG_VALGRIND =3D $(VALGRIND) $(DEFAULT_VALGRINDFLAGS) +# $(VALGRINDFLAGS). + +# Whether to look for valgrind and set the variables can be influenced +# by calling gl_VALGRIND_TESTS_DEFAULT_NO in configure.ac. +# Regardless, the user can change the choice through the options +# --enable-valgrind-tests or --disable-valgrind-tests. + +# You may modify the VALGRIND, DEFAULT_VALGRINDFLAGS and VALGRINDFLAGS +# variables before calling this function to override defaults. Either +# as developer from configure.ac or user on the ./configure command +# line. + +AC_DEFUN([gl_VALGRIND_TESTS_DEFAULT_NO], +[ + gl_valgrind_tests_default=3Dno +]) + +AC_DEFUN_ONCE([gl_VALGRIND_TESTS], [ AC_ARG_ENABLE([valgrind-tests], AS_HELP_STRING([--disable-valgrind-tests], [don't try to run self tests under valgrind]), =2D [opt_valgrind_tests=3D$enableval], [opt_valgrind_tests=3Dyes]) + [opt_valgrind_tests=3D$enableval], [opt_valgrind_tests=3D${gl_valgrind= _tests_default:-yes}]) =20 # Run self-tests under valgrind? if test "$opt_valgrind_tests" =3D "yes" && test "$cross_compiling" =3D n= o; then AC_CHECK_PROGS([VALGRIND], [valgrind]) =20 + AC_SUBST([DEFAULT_VALGRINDFLAGS]) + if test -z "$DEFAULT_VALGRINDFLAGS"; then + DEFAULT_VALGRINDFLAGS=3D"-q --error-exitcode=3D1 --leak-check=3Dfull" + fi + AC_ARG_VAR([VALGRINDFLAGS], [Additional flags for Valgrind]) + + if test -n "$VALGRIND"; then + AC_CACHE_CHECK([for valgrind options for tests], + [gl_cv_opt_valgrind_tests], + [AS_IF([$VALGRIND $DEFAULT_VALGRINDFLAGS $VALGRINDFLAGS true], + [gl_cv_opt_valgrind_tests=3D"$DEFAULT_VALGRINDFLAGS $VALGRI= NDFLAGS"], + [gl_cv_opt_valgrind_tests=3Dno]) + ]) + if test "$gl_cv_opt_valgrind_tests" !=3D no; then + VALGRIND=3D"$VALGRIND $gl_cv_opt_valgrind_tests" + fi + fi + if test -n "$VALGRIND"; then dnl On Ubuntu 16.04, /usr/bin/valgrind works only on 64-bit executab= les dnl but fails on 32-bit executables (with exit code 1) and on x86_64= -x32 @@ -28,7 +70,7 @@ AC_DEFUN([gl_VALGRIND_TESTS], [gl_cv_prog_valgrind_works], [AC_RUN_IFELSE( [AC_LANG_SOURCE([[int main () { return 0; }]])], =2D [$VALGRIND ./conftest$ac_exeext 2>/dev/null + [$VALGRIND $gl_cv_opt_valgrind_tests ./conftest$ac_exeext 2>/de= v/null if test $? =3D 0; then gl_cv_prog_valgrind_works=3Dyes else @@ -42,19 +84,6 @@ AC_DEFUN([gl_VALGRIND_TESTS], fi fi =20 =2D if test -n "$VALGRIND"; then =2D AC_CACHE_CHECK([for valgrind options for tests], =2D [gl_cv_opt_valgrind_tests], =2D [gl_valgrind_opts=3D'-q --error-exitcode=3D1 --leak-check=3Dfull' =2D if $VALGRIND $gl_valgrind_opts ls > /dev/null 2>&1; then =2D gl_cv_opt_valgrind_tests=3D"$gl_valgrind_opts" =2D else =2D gl_cv_opt_valgrind_tests=3Dno =2D fi =2D ]) =2D if test "$gl_cv_opt_valgrind_tests" !=3D no; then =2D VALGRIND=3D"$VALGRIND $gl_cv_opt_valgrind_tests" =2D fi =2D fi + AC_SUBST([LOG_VALGRIND], ["\$(VALGRIND) \$(DEFAULT_VALGRINDFLAGS) \$(VA= LGRINDFLAGS)"]) fi ]) =2D-=20 2.20.1 --=-=-=-- --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYIAB0WIQSjzJyHC50xCrrUzy9RcisI/kdFogUCYJ5sygAKCRBRcisI/kdF ojkRAP44pxFyU8mabIDNg2X5UfP/xS+jgBZ/kPwjXS26aUjgrAD9FcoDFOaCDOFf OP0cVkKYvOJLIud8og32ayviT+zwZgA= =DtrA -----END PGP SIGNATURE----- --==-=-=--