bug-gnulib@gnu.org mirror (unofficial)
 help / color / mirror / Atom feed
From: Bruno Haible <bruno@clisp.org>
To: Simon Josefsson <simon@josefsson.org>
Cc: noloader@gmail.com, Bernhard Voelker <mail@bernhard-voelker.de>,
	bug-gnulib@gnu.org
Subject: Re: warnings in unit tests
Date: Thu, 10 Jun 2021 21:51:05 +0200	[thread overview]
Message-ID: <3563143.CciQfH6Ort@omega> (raw)
In-Reply-To: <87k0n2nnn7.fsf@latte.josefsson.org>

Simon Josefsson wrote:
> Most of my projects have multiple gnulib instances in
> them, which gnulib self-tests does not support.

Yes, this situation is not supported.

> I try to have gnulib tests enabled, but sometimes I disable them because
> having them enabled leads to problems that are too time-consuming to
> debug and fix.  ...
> 
> I've seen over the years a number of cases where old releases of my
> packages fail to 'make check' correctly only because of a gnulib
> self-test that was 1) simply buggy, or 2) the gnulib replacement code
> had bugs in them on some platform, or 3) the self-test tested a
> corner-case that newer platforms (for valid reasons) chose to behave
> differently for.

Failing tests can happen, yes. That's true for all test suites.

But it shouldn't be you as a package maintainer who debugs these tests.
(Only the tests for the modules that you contributed; i.e. wearing your
Gnulib contributor hat :-).

> I think there is room for improvements in this field

From what you are telling, the first improvement should be to clarify
where to report failing tests. Done as follows:


2021-06-10  Bruno Haible  <bruno@clisp.org>

	Clarify where to report test failures from Gnulib tests.
	* gnulib-tool (func_emit_tests_Makefile_am): Emit a dependency of
	'check-am' on 'check-notice' that prints a notice.

diff --git a/gnulib-tool b/gnulib-tool
index d41d58c..4f97e63 100755
--- a/gnulib-tool
+++ b/gnulib-tool
@@ -4275,6 +4275,14 @@ func_emit_tests_Makefile_am ()
   echo "	@echo '## You can ignore compiler warnings in this directory.  ##'"
   echo "	@echo '## ---------------------------------------------------- ##'"
   echo
+  # Arrange to print a message before executing the tests in this directory.
+  echo "check-am: check-notice"
+  echo "check-notice:"
+  echo "	@echo '## ---------------------------------------------------- ##'"
+  echo "	@echo '## ------------------- Gnulib tests ------------------- ##'"
+  echo "	@echo '## Please report test failures to <bug-gnulib@gnu.org>. ##'"
+  echo "	@echo '## ---------------------------------------------------- ##'"
+  echo
   echo "# Clean up after Solaris cc."
   echo "clean-local:"
   echo "	rm -rf SunWS_cache"



  reply	other threads:[~2021-06-10 19:51 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-16 17:01 new module 'sigsegv' Bruno Haible
2021-06-06 23:27 ` Dmitry V. Levin
2021-06-07  0:49   ` Bruno Haible
2021-06-07 10:29     ` Dmitry V. Levin
2021-06-08  1:45       ` Jim Meyering
2021-06-08  2:40         ` warnings in unit tests Bruno Haible
2021-06-08  5:55           ` Jim Meyering
2021-06-08  8:56             ` Bruno Haible
2021-06-09  0:41               ` Dmitry V. Levin
2021-06-10 20:05                 ` Bruno Haible
     [not found]             ` <CAH8yC8kHTq5J9onJj+2jwy_DwzXrwujqFs9TEBxGh5k_KCu=kg@mail.gmail.com>
2021-06-08 10:57               ` Bruno Haible
2021-06-08 16:42                 ` Paul Eggert
2021-06-09 13:35                   ` Dmitry V. Levin
2021-06-09 19:38                   ` Bruno Haible
2021-06-10 19:39                   ` Bruno Haible
2021-06-09  7:23                 ` Bernhard Voelker
2021-06-09 14:17                   ` Bruno Haible
2021-06-10  8:13                     ` Simon Josefsson via Gnulib discussion list
2021-06-10 19:51                       ` Bruno Haible [this message]
2021-06-10 21:49                         ` Simon Josefsson via Gnulib discussion list
2021-06-11 12:21                         ` Eric Blake
2021-06-11 13:57                           ` Bruno Haible
2021-06-19 12:02 ` new module 'sigsegv' Bruno Haible
2021-06-21 18:22   ` [PATCH] sigsegv, sigsegv-tests: Assign my contributions to the FSF Eric Blake

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://lists.gnu.org/mailman/listinfo/bug-gnulib

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3563143.CciQfH6Ort@omega \
    --to=bruno@clisp.org \
    --cc=bug-gnulib@gnu.org \
    --cc=mail@bernhard-voelker.de \
    --cc=noloader@gmail.com \
    --cc=simon@josefsson.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).