gl_CACHE_VAL_SILENT won't work correctly with upcoming autoconf 2.70

gl_CACHE_VAL_SILENT won't work correctly with upcoming autoconf 2.70

[Zack Weinberg]

I'm doing groundwork for the long-overdue release of autoconf 2.70,
and part of that is checking to make sure that various changes in
development trunk since 2.69 haven't broken third-party macros.  I
noticed that gnulib-common.m4 contains a macro, gl_CACHE_VAL_SILENT,
that attempts to make AC_CACHE_VAL not print " (cached)" by changing
the value of the shell variable $as_echo_n.  This has had no effect
with development autoconf's AC_CACHE_VAL since 2013 (!)  and since,
um, today, it will also trigger -Wobsolete warnings.

Abstractly, the best thing to do about this would be to remove the
macro and change all of its uses to be AC_CACHE_CHECK, with proper
messaging, instead.  This would only not be possible if there was ever
a gl_CACHE_VAL_SILENT invocation *nested inside* an AC_CACHE_CHECK
invocation -- which shouldn't be done anyway, since the entire thing
would be skipped if the outer test is cached.  However, reviewing the
various uses, they are complicated enough that I don't think I am the
best person to write that patch.  I may still attempt it ...
eventually ... but I have a whole bunch of other work to do to make
the release possible, and this particular problem is mostly cosmetic.

zw

Re: gl_CACHE_VAL_SILENT won't work correctly with upcoming autoconf 2.70

[Bruno Haible]

[CCing autoconf@gnu.org]

Hi Zack,

> Abstractly, the best thing to do about this would be to remove the
> macro and change all of its uses to be AC_CACHE_CHECK, with proper
> messaging, instead.

The point is that we don't want messaging for some tests. Most people
find the 'configure' output already long enough.

The underlying problem in Autoconf is the following: There is a macro
AC_CACHE_CHECK that does messaging, and a macro AC_CACHE_VAL whose
main purpose is to make a cache lookup, not messaging, but it
nevertheless outputs '(cached)' strings occasionally.

Please fix this correctly
  - either by having AC_CACHE_VAL set some shell variable that
    AC_CACHE_CHECK will inspect after executing AC_CACHE_VAL,
  - or by introducing a macro that is like AC_CACHE_VAL but produces
    no spurious output.
Then AC_CACHE_VAL or the other macro is properly silent, and then no
gl_CACHE_VAL_SILENT will be needed any more.

Bruno

Re: gl_CACHE_VAL_SILENT won't work correctly with upcoming autoconf 2.70

[Nick Bowler]

Hi Bruno,

On 13/03/2020, Bruno Haible <bruno@clisp.org> wrote:
> The underlying problem in Autoconf is the following: There is a macro
> AC_CACHE_CHECK that does messaging, and a macro AC_CACHE_VAL whose
> main purpose is to make a cache lookup, not messaging, but it
> nevertheless outputs '(cached)' strings occasionally.
>
> Please fix this correctly
[...]
>   - or by introducing a macro that is like AC_CACHE_VAL but produces
>     no spurious output.

Such a macro already exists in Autoconf.  If you do not need the
messaging from AC_CACHE_VAL you can simply use AS_VAR_SET_IF.

For example:

  AS_VAR_SET_IF([my_cv_no_printing], [my_cv_no_printing=hello])

You could also just write a normal shell if statement.

Cheers,
  Nick

Re: gl_CACHE_VAL_SILENT won't work correctly with upcoming autoconf 2.70

[Paul Eggert]

On 3/13/20 3:04 PM, Nick Bowler wrote:
>    AS_VAR_SET_IF([my_cv_no_printing], [my_cv_no_printing=hello])

I assume you meant

   AS_VAR_SET_IF([my_cv_no_printing], [], [my_cv_no_printing=hello])

Re: gl_CACHE_VAL_SILENT won't work correctly with upcoming autoconf 2.70

[Bruno Haible]

Nick Bowler and Paul Eggert wrote:
> >    AS_VAR_SET_IF([my_cv_no_printing], [my_cv_no_printing=hello])
> 
>    AS_VAR_SET_IF([my_cv_no_printing], [], [my_cv_no_printing=hello])

But then I lose
  - the various AC_DIAGNOSE checks that are in AC_CACHE_VAL,
  - possible future improvements / changes to the caching mechanism.
    (I find it a dangerous bet to assume that the caching will only
    be based on a shell variable lookup and not something else.)

I would prefer to define gl_CACHE_VAL_SILENT as an invocation of
AC_CACHE_VAL that merely neutralizes the _AS_ECHO_N call. Such as
(untested):

AC_DEFUN([gl_CACHE_VAL_SILENT],
[
  m4_pushdef([_AS_ECHO_N], [])
  AC_CACHE_VAL([$1], [$2])
  m4_popdef([_AS_ECHO_N])
])

Bruno

Re: gl_CACHE_VAL_SILENT won't work correctly with upcoming autoconf 2.70

[Zack Weinberg]

On Fri, Mar 13, 2020 at 5:54 PM Bruno Haible <bruno@clisp.org> wrote:
>
> [CCing autoconf@gnu.org]
>
> Hi Zack,
>
> > Abstractly, the best thing to do about this would be to remove the
> > macro and change all of its uses to be AC_CACHE_CHECK, with proper
> > messaging, instead.
>
> The point is that we don't want messaging for some tests. Most people
> find the 'configure' output already long enough.

 I disagree with this on principle.  If you're doing any test that's
complicated enough to be worth caching, then you should use
AC_CACHE_CHECK and provide messaging for it.  Yeah, most of the time
that text is just going to scroll by unread, but when something goes
wrong, the poor schmuck who has to debug it wants to know that it was
the test for `ceil` (for instance) that hung the build, not the
unrelated thing that happens to have been tested right before that.

> The underlying problem in Autoconf is the following: There is a macro
> AC_CACHE_CHECK that does messaging, and a macro AC_CACHE_VAL whose
> main purpose is to make a cache lookup, not messaging, but it
> nevertheless outputs '(cached)' strings occasionally.

So based on the above my attitude is that AC_CACHE_VAL (and
AC_MSG_CHECKING and AC_MSG_RESULT) exist only for backward
compatibility's sake and what you're supposed to be using is
AC_CACHE_CHECK, and I'm not inclined to spend any of my own hacking
time on "a macro that is like AC_CACHE_VAL but produces no spurious
output".

On the other hand, if you write a patch and Paul or Eric likes it, I
won't stand in its way, either.

zw

Re: gl_CACHE_VAL_SILENT won't work correctly with upcoming autoconf 2.70

[Nick Bowler]

On 2020-03-13, Paul Eggert <eggert@cs.ucla.edu> wrote:
> On 3/13/20 3:04 PM, Nick Bowler wrote:
>>    AS_VAR_SET_IF([my_cv_no_printing], [my_cv_no_printing=hello])
>
> I assume you meant
>
>    AS_VAR_SET_IF([my_cv_no_printing], [], [my_cv_no_printing=hello])

Yes, thank you for the correction.

Cheers,
  Nick

Re: debugging autoconf tests (was: gl_CACHE_VAL_SILENT)

[Bruno Haible]

Hi Zack,

> Yeah, most of the time
> that text is just going to scroll by unread, but when something goes
> wrong, the poor schmuck who has to debug it wants to know that it was
> the test for `ceil` (for instance) that hung the build, not the
> unrelated thing that happens to have been tested right before that.

Hanging autoconf tests are rare. And even in that case, looking at
the configure output does not help much, because it won't tell you
whether the compiler is taking hours or whether the test program hangs.
In practice, when something hangs, "ps aux" or "pstree -p" (when available)
is a better starting point for debugging.

In the much more frequent case that an autoconf test produces an unexpected
result, the most efficient debugging technique is:
  1. look at the S[...] and D[...] lines in config.status. This will tell
     you whether the problem is on the configure.ac + *.m4 side or on the
     Makefile.{am,in} side.
  2. If it is on the configure.ac + *.m4 side, rerun the configuration with
     option '-C'. Then look at the config.cache. This will tell you whether
     the problem is inside or outside AC_CACHE_VAL/AC_CACHE_CHECK calls.
  3. If it is inside AC_CACHE_VAL/AC_CACHE_CHECK calls, then look at the
     config.log. This will show you the test programs, the compiler options
     with which they are compiled, and the test programs' exit code.

In this case as well, the configure output is hardly useful.

The only real uses of the configure output, for me who is maintaining hundreds
of autoconf macros in gnulib, are:
  - Compare the configuration results between different platforms (e.g.
    linux-mipseb and linux-mipsel).
  - Look for spurious error messages such as
      test: =: unary operator expected
    that indicate broken shell code.

The other use of this output is, of course, to let the non-expert user
keep patient.

Due to all this, executing a test silently is not as problematic as
you think.

Bruno

Re: debugging autoconf tests (was: gl_CACHE_VAL_SILENT)

[Zack Weinberg]

On Fri, Mar 13, 2020 at 10:49 PM Bruno Haible <bruno@clisp.org> wrote:
>> but when something goes
> > wrong, the poor schmuck who has to debug it wants to know that it was
> > the test for `ceil` (for instance) that hung the build, not the
> > unrelated thing that happens to have been tested right before that.
>
> Hanging autoconf tests are rare. And even in that case, looking at
> the configure output does not help much, because it won't tell you
> whether the compiler is taking hours or whether the test program hangs.
> In practice, when something hangs, "ps aux" or "pstree -p" (when available)
> is a better starting point for debugging.

Aha, you're coming at this from the perspective of debugging with
interactive access to the system where the problem happens.

All of the worst autoconf-related bugs I've had to deal with in the
past decade or so, showed up in CI environments that I did _not_ have
interactive access to.  I only got the transcript of the build.  I had
to specifically add code to dump the contents of config.log, and then
track things down further by adding extra AC_MSG_* within the problem
macro.  You can probably imagine how frustrating this can be,
especially when the cycle time is a half-hour or more.  I haven't had
to deal with any of these problems with a project that made heavy use
of gnulib, but I would have been quite annoyed to find that I was
barking up the wrong tree because configure didn't tell me that it was
doing some check because it didn't seem important enough.

Also, overriding $as_echo(_n), and your suggestion to override
AS_ECHO_N with pushdef, are more problematic than you may realize
because they affect what gets written to config.log, not just what
gets written to configure's stderr.  (Because configure writes to
config.log with a construct like AS_ECHO([...]) >&AS_MESSAGE_LOG_FD.)

May I suggest that what you really want is a --quiet switch for
configure, a la V=0 with automake-generated makefiles?  If you had
that, would you be OK with changing all of the gnulib .m4 files that
currently use gl_AC_CACHE_VAL_SILENT to use AC_CACHE_CHECK instead?

zw