From: Bruno Haible <bruno@clisp.org>
To: Simon Josefsson <simon@josefsson.org>
Cc: noloader@gmail.com, Bernhard Voelker <mail@bernhard-voelker.de>,
bug-gnulib@gnu.org
Subject: Re: warnings in unit tests
Date: Thu, 10 Jun 2021 21:51:05 +0200 [thread overview]
Message-ID: <3563143.CciQfH6Ort@omega> (raw)
In-Reply-To: <87k0n2nnn7.fsf@latte.josefsson.org>
Simon Josefsson wrote:
> Most of my projects have multiple gnulib instances in
> them, which gnulib self-tests does not support.
Yes, this situation is not supported.
> I try to have gnulib tests enabled, but sometimes I disable them because
> having them enabled leads to problems that are too time-consuming to
> debug and fix. ...
>
> I've seen over the years a number of cases where old releases of my
> packages fail to 'make check' correctly only because of a gnulib
> self-test that was 1) simply buggy, or 2) the gnulib replacement code
> had bugs in them on some platform, or 3) the self-test tested a
> corner-case that newer platforms (for valid reasons) chose to behave
> differently for.
Failing tests can happen, yes. That's true for all test suites.
But it shouldn't be you as a package maintainer who debugs these tests.
(Only the tests for the modules that you contributed; i.e. wearing your
Gnulib contributor hat :-).
> I think there is room for improvements in this field
From what you are telling, the first improvement should be to clarify
where to report failing tests. Done as follows:
2021-06-10 Bruno Haible <bruno@clisp.org>
Clarify where to report test failures from Gnulib tests.
* gnulib-tool (func_emit_tests_Makefile_am): Emit a dependency of
'check-am' on 'check-notice' that prints a notice.
diff --git a/gnulib-tool b/gnulib-tool
index d41d58c..4f97e63 100755
--- a/gnulib-tool
+++ b/gnulib-tool
@@ -4275,6 +4275,14 @@ func_emit_tests_Makefile_am ()
echo " @echo '## You can ignore compiler warnings in this directory. ##'"
echo " @echo '## ---------------------------------------------------- ##'"
echo
+ # Arrange to print a message before executing the tests in this directory.
+ echo "check-am: check-notice"
+ echo "check-notice:"
+ echo " @echo '## ---------------------------------------------------- ##'"
+ echo " @echo '## ------------------- Gnulib tests ------------------- ##'"
+ echo " @echo '## Please report test failures to <bug-gnulib@gnu.org>. ##'"
+ echo " @echo '## ---------------------------------------------------- ##'"
+ echo
echo "# Clean up after Solaris cc."
echo "clean-local:"
echo " rm -rf SunWS_cache"
next prev parent reply other threads:[~2021-06-10 19:51 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-05-16 17:01 new module 'sigsegv' Bruno Haible
2021-06-06 23:27 ` Dmitry V. Levin
2021-06-07 0:49 ` Bruno Haible
2021-06-07 10:29 ` Dmitry V. Levin
2021-06-08 1:45 ` Jim Meyering
2021-06-08 2:40 ` warnings in unit tests Bruno Haible
2021-06-08 5:55 ` Jim Meyering
2021-06-08 8:56 ` Bruno Haible
2021-06-09 0:41 ` Dmitry V. Levin
2021-06-10 20:05 ` Bruno Haible
[not found] ` <CAH8yC8kHTq5J9onJj+2jwy_DwzXrwujqFs9TEBxGh5k_KCu=kg@mail.gmail.com>
2021-06-08 10:57 ` Bruno Haible
2021-06-08 16:42 ` Paul Eggert
2021-06-09 13:35 ` Dmitry V. Levin
2021-06-09 19:38 ` Bruno Haible
2021-06-10 19:39 ` Bruno Haible
2021-06-09 7:23 ` Bernhard Voelker
2021-06-09 14:17 ` Bruno Haible
2021-06-10 8:13 ` Simon Josefsson via Gnulib discussion list
2021-06-10 19:51 ` Bruno Haible [this message]
2021-06-10 21:49 ` Simon Josefsson via Gnulib discussion list
2021-06-11 12:21 ` Eric Blake
2021-06-11 13:57 ` Bruno Haible
2021-06-19 12:02 ` new module 'sigsegv' Bruno Haible
2021-06-21 18:22 ` [PATCH] sigsegv, sigsegv-tests: Assign my contributions to the FSF Eric Blake
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://lists.gnu.org/mailman/listinfo/bug-gnulib
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3563143.CciQfH6Ort@omega \
--to=bruno@clisp.org \
--cc=bug-gnulib@gnu.org \
--cc=mail@bernhard-voelker.de \
--cc=noloader@gmail.com \
--cc=simon@josefsson.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).