From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS22989 209.51.188.0/24 X-Spam-Status: No, score=-3.9 required=3.0 tests=AWL,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 247581F5AE for ; Tue, 8 Jun 2021 05:56:18 +0000 (UTC) Received: from localhost ([::1]:56890 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lqUin-0000CG-0M for normalperson@yhbt.net; Tue, 08 Jun 2021 01:56:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34730) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lqUij-0000C6-Jx for bug-gnulib@gnu.org; Tue, 08 Jun 2021 01:56:13 -0400 Received: from mail-yb1-f174.google.com ([209.85.219.174]:46629) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lqUif-0002T1-BD for bug-gnulib@gnu.org; Tue, 08 Jun 2021 01:56:13 -0400 Received: by mail-yb1-f174.google.com with SMTP id y2so28425859ybq.13 for ; Mon, 07 Jun 2021 22:56:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=93lGe40ptrKhbO5rTCH6pJa1tDdogPjj7OLL5rBWa8E=; b=gRVIWfQWwmsyBFezjcjSVZJ+LOGFQRFc6glicxcQPki41/UApxntNOV+7CpcHPFpsp fHngtFlIreR1i6Eos3MBpNecrzvT2ADA3aE0ZZp9eCZxFVWy0sTLJsB7hglvb9JdlV7+ wSWQ1nYVMhpReE4PwPe9b80HaJ3Ds8ZQAY6015+cVjocMSrhYdi90wxI1ROmybzPTtr+ Rm96fjZYXF+Eob3zjp3QsWuzEkvwVcwVUEJ2JfSAra0u6gqRJM0oka7K5LhF8wdShYwX qo8sheaNqPbnIxkaFP3a1HPsDL3gPUriBFQoUGAOjexBcvtndcpNjmmuCSTFIlHu4FnK XNcQ== X-Gm-Message-State: AOAM530UUX0sRN6/iWNN6Y2GukQfSRzmy3/B/uNyjhZ/VS+DK1g4cyKB dbpzBf3s34q9cUy9jcKk14P+0z0RE1gltkvHd9Q= X-Google-Smtp-Source: ABdhPJzLVWCySPsvidLvu5PRzXGI8UOyDiIi+nQwaCL3GqpnPO/F3EGum7YYhrG0Q6irFueFvWg2oTHBm32uhBPZtYk= X-Received: by 2002:a05:6902:1342:: with SMTP id g2mr30174936ybu.474.1623131768365; Mon, 07 Jun 2021 22:56:08 -0700 (PDT) MIME-Version: 1.0 References: <15897619.AQZsorJ5YP@omega> <20210607102932.GA24455@altlinux.org> <2113844.jgEgOJX5uS@omega> In-Reply-To: <2113844.jgEgOJX5uS@omega> From: Jim Meyering Date: Mon, 7 Jun 2021 22:55:56 -0700 Message-ID: Subject: Re: warnings in unit tests To: Bruno Haible Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=209.85.219.174; envelope-from=meyering@gmail.com; helo=mail-yb1-f174.google.com X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, FREEMAIL_FORGED_FROMDOMAIN=0.248, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: bug-gnulib@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Gnulib discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "bug-gnulib@gnu.org List" Errors-To: bug-gnulib-bounces+normalperson=yhbt.net@gnu.org Sender: "bug-gnulib" On Mon, Jun 7, 2021 at 7:41 PM Bruno Haible wrote: > Hi Jim, > > > > > > @@ -183,6 +183,9 @@ main () > > > > > *(volatile int *) (page + 0x678) =3D 42; > > > > > break; > > > > > case 3: > > > > > +#if 6 < __GNUC__ > > > > > +# pragma GCC diagnostic ignored "-Wnull-dereference" > > > > > +#endif > > > > > *(volatile int *) 0 =3D 42; > > > > > break; > > > > > case 4: > > > > > > > > We shouldn't spend time eliminating warnings from test code. > > > > > > > > The goal is to have a good coverage of the lib/* code with unit tes= ts. > > > > That means, we need to > > > > - make it easy to write unit tests, > > > > - not make it time-consuming to maintain them. > > > > > > > > Eliminating warnings from lib/* code is useful, to avoid bugs in th= e > > > > programs. But eliminating warnings from tests/* code goes against t= he > > > > goal of increasing test coverage. > > > > Hi Bruno, > > Isn't this code exceptional enough to merit three lines of warning supp= ression? > > No. Good unit tests do exceptional things, like calling close(-1), passin= g > NULL pointers to various functions, and other things that e.g. Coverity > warns about. > > > I would like to continue to use -Werror with most warning options even > > in test code, > > as long as the cost is low. This feels like a very low one-time cost. > > No, the cost is not low. Dmitry's patch also changed the linkage of 11 > functions. He did so to silence warnings (-Wmissing-prototypes) which are > 1) not part of '-Wall', > 2) just pointless for tests [my opinion as maintainer of most of these > tests]. > > I don't want the maintainers of packages that use Gnulib to push costs > onto Gnulib, when it is just for their personal preference. > > The tests in Gnulib are there to evaluate the reliability of Gnulib's > modules on a particular platform. The more test coverage we have, the > better. > > Warnings in the unit tests are not problems. Test *failures* are problems= . > > gnulib-tool has options --with-tests and --tests-base, that allow a > package to ship the Gnulib tests and thus help Gnulib (by having more > people run the unit tests and report possible test failures). > > If a package maintainer insists on having warning-free builds, I kindly > ask them to not bundle the Gnulib tests any more =E2=80=94 because requir= ing > warning-free tests from Gnulib is more of an impediment than of a help. > > For those package maintainers who are OK to accept warnings in the > Gnulib tests directory, on the other hand, I repeat my offer to > automatically add -Wno-error in the tests/Makefile.am of that directory. I can live without -Wmissing-prototypes in gnulib tests, but I still remember times where using that option exposed a real bug. My point about the cost/benefit was regarding that 3-line addition for a single, deliberate NULL-deref. That one really does not deserve to quash -Wnull-dereference for all tests.