Re: How can Autoconf help with the transition to stricter compilation defaults?

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-10 09:16, Zack Weinberg wrote:
> Changes to handle C23 built-in ‘bool’ better are under development but
> the design has not yet been finalized.

[I'm cc'ing this to bug-gnulib too.]

To my mind this is the biggest outstanding issue in Autoconf as far as 
C23 goes, as the upgrade path for Autoconf's existing bool support is 
not entirely clear. As Florian mentioned, distros can't assume Autoconf 
upgrades when building other packages; that being said, we should get 
Autoconf's bool support fixed sooner rather than later as bool hassles 
will remain until Autoconf is fixed and these fixes are propagated to 
Autoconf's users.

Here's the main Autoconf issue issue with bool. Traditionally, Autoconf 
supported K&R C, C89, C99, etc. At some point (I'm not sure when), 
Autoconf started requiring C89 or later. Is it now OK for Autoconf to 
require C99 or later, as far as bool is concerned? If so, that'll 
considerably simplify the ongoing maintenance hassle for bool.

Requiring C99-or-later bool is the option that Gnulib has taken. Its 
'stdbool' module and its gl_C_BOOL macro assumes C99 or later, and as 
far as I know no Gnulib-using package is using Gnulib's 'stdbool-c99' 
module which we kept around in case somebody still needed bool to work 
atop a C89 system. (We considered supporting C23 bool atop C89 but it 
was too painful.)

If we follow Gnulib's lead, Autoconf will generate a config.h that does 
"#include <stdbool.h>" on pre-C23 systems, and this config.h will not 
not work on pre-C99 systems. This of course could in theory break some 
programs, just as compiling them with C23 can break them. But I don't 
see any better option at this point. And besides, what package outside 
of a museum still requires C89 and don't work with C99?

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-10 10:19, Aaron Ballman wrote:
> In terms of the Clang side of things, I don't think we've formed any
> sort of official stance on how to handle that yet. It's UB (you can
> declare the C standard library interface without UB but calling any
> function with a mismatched signature is UB)

The Autoconf-generated code is never executed, so this is not a runtime 
issue; it's merely an issue of undefined behavior during translation. A 
problem could occur with a picky compiler or linker that rejects modules 
with mismatched function type declarations. Does Clang do that, or 
require or use such a linker? If not, there's no practical problem here. 
If so, it'd be helpful if Clang continued to support its traditional 
behavior that doesn't reject Autoconf's test cases, and for this to be 
the default.

Autoconf arose because one cannot ask something like "Can I call the 
renameat2 function with its usual signature?" in standard C code and one 
must escape into something like the shell to handle such questions. C23 
and GCC and Clang have added a few features to answer such questions, 
such as __has_include. But these features don't go nearly far enough. 
For example, __has_include tells me only whether the include file 
<foo.h> exists; it won't tell me whether I can successfully include 
<foo.h>, or whether <foo.h> will declare the function 'bar', or whether 
'bar' will have a signature compatible with my code's calls to 'bar'.

If I could request a single thing from the C23/GCC/Clang side, I'd ask 
for better facilities to be able to ask such questions within C code, 
without using the shell. Then a good chunk of Autoconf could dry up and 
blow away.

I realize that I'm asking for a lot. For example, a traditional 
implementation cannot answer the renameat2 question without consulting 
the linker. That being said, this ability is essential for modular 
programming at the low level, and if compilers don't provide this 
ability Autoconf will simply have to do the best it can, regardless of 
whether it generates source code that relies on undefined behavior.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 4071 bytes --]



> On 10 Nov 2022, at 17:16, Zack Weinberg <zack@owlfolio.org> wrote:
> 
> I’m the closest thing Autoconf has to a lead maintainer at present.
> 
> It’s come to my attention (via https://lwn.net/Articles/913505/ and
> https://fedoraproject.org/wiki/Changes/PortingToModernC) that GCC and
> Clang both plan to disable several “legacy” C language features by
> default in a near-future release (GCC 14, Clang 16) (see the Fedora
> wiki link for a list).  I understand that this change potentially
> breaks a lot of old dusty code, and in particular that
> Autoconf-generated configure scripts use constructs that may *silently
> give the wrong answer to a probe* when a stricter compiler is in use.

Thank you for asking. The fixes in git get us there, I think, as far
as you can, with the exception of the stuff you (and I) mention below.

> 
> [...]
> 
> The biggest remaining (potential) problem, that I’m aware of, is that
> AC_CHECK_FUNC unconditionally declares the function we’re probing for
> as ‘char NAME (void)’, and asks the compiler to call it with no
> arguments, regardless of what its prototype actually is.  It is not
> clear to me whether this will still work with the planned changes to
> the compilers.  Both GCC 12 and Clang 14 have on-by-default warnings
> triggered by ‘extern char memcpy(void);’ (or any other standard
> library function whose prototype is coded into the compiler) and this
> already causes problems for people who run configure scripts with
> CC='cc -Werror'.  Unfortunately this is very hard to fix — we would
> have to build a comprehensive list of library functions into Autoconf,
> mapping each to either its documented prototype or to a header where
> it ought to be declared; in the latter case we would also have to make
> e.g. AC_CHECK_FUNCS([getaddrinfo]) imply AC_CHECK_HEADERS([sys/types.h
> sys/socket.h netdb.h]) which might mess up configure scripts that
> aren’t expecting headers to be probed at that point.
> 
> How important do you think it is for this to be fixed?
> 
> Are there any other changes you would like to see in a near-future
> Autoconf 2.72 in order to make this transition easier?

This might be a WONTFIX but let me mention it just for
the record:
1. AC_CHECK_FUNCS is, as you've noticed, very noisy.

I would support having a hardcoded list for certain CHOSTs
as Rich suggests to reduce noise, but I don't think we can
do this accurately very quickly.

I think for Gentoo's efforts, I might just build up a set
of cache variables for glibc/musl on each arch to
reduce the noise in logs. But that's time consuming
and brittle still, so I'm not sure.

(Note that Gentoo and Fedora are taking two complementary
but different approaches here:
we're diffing config.logs and other compiler
output, in addition to build logs, while Florian for Fedora
Is building a list of functions which *we know* are available
In a specific environment and patching gcc to spit out
logs when something in said list is missing. This mitigates
noise for things like functions in libbsd, which I'm finding
a bit of a pain.)

I'll see what others say.

2. I often have to set the following cache variables to
reduce noise in logs:
* ac_cv_c_undeclared_builtin_options="none needed"
* ac_cv_header_sys_types_h_makedev=no
* gl_cv_compiler_check_decl_option="-Werror=implicit-function-declaration" (obviously this is gnulib)
* gl_cv_minmax_in_limits_h=no

I don't know if we can do anything to make these tests smarter
or just leave it as-is. It's fine if we can't, as exporting the cache
vars is not a bad workaround for us when doing testing.

> 
> zw
> 
> p.s. GCC and Clang folks: As long as you’re changing the defaults out
> from under people, can you please also remove the last few predefined
> user-namespace macros (-Dlinux, -Dunix, -Darm, etc) from all the
> -std=gnuXX modes?

I support this as well. This kind of thing has led to endless
bugs in userland, see https://reviews.llvm.org/D137511.


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 358 bytes --]

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Aaron Ballman]

On Thu, Nov 10, 2022 at 4:05 PM Paul Eggert <eggert@cs.ucla.edu> wrote:
>
> On 2022-11-10 10:19, Aaron Ballman wrote:
> > In terms of the Clang side of things, I don't think we've formed any
> > sort of official stance on how to handle that yet. It's UB (you can
> > declare the C standard library interface without UB but calling any
> > function with a mismatched signature is UB)
>
> The Autoconf-generated code is never executed, so this is not a runtime
> issue; it's merely an issue of undefined behavior during translation.

FWIW, the only thing the (Clang) compiler is aware of is translation.
So from the frontend perspective, we can't tell the difference between
"trust me this is safe because it never gets executed" and "this is a
CVE". We believe the runtime behavior is sufficiently dangerous to
warrant a conservative view that any call to a function will be a call
that gets executed at runtime, hence a definitive signature mismatch
is something we feel comfortable diagnosing (in some form) by default.

> A
> problem could occur with a picky compiler or linker that rejects modules
> with mismatched function type declarations. Does Clang do that, or
> require or use such a linker? If not, there's no practical problem here.
> If so, it'd be helpful if Clang continued to support its traditional
> behavior that doesn't reject Autoconf's test cases, and for this to be
> the default.

Clang doesn't require such a linker (we work with various system linkers).

> Autoconf arose because one cannot ask something like "Can I call the
> renameat2 function with its usual signature?" in standard C code and one
> must escape into something like the shell to handle such questions. C23
> and GCC and Clang have added a few features to answer such questions,
> such as __has_include. But these features don't go nearly far enough.
> For example, __has_include tells me only whether the include file
> <foo.h> exists; it won't tell me whether I can successfully include
> <foo.h>, or whether <foo.h> will declare the function 'bar', or whether
> 'bar' will have a signature compatible with my code's calls to 'bar'.
>
> If I could request a single thing from the C23/GCC/Clang side, I'd ask
> for better facilities to be able to ask such questions within C code,
> without using the shell. Then a good chunk of Autoconf could dry up and
> blow away.
>
> I realize that I'm asking for a lot. For example, a traditional
> implementation cannot answer the renameat2 question without consulting
> the linker. That being said, this ability is essential for modular
> programming at the low level, and if compilers don't provide this
> ability Autoconf will simply have to do the best it can, regardless of
> whether it generates source code that relies on undefined behavior.

This would be challenging for an implementation like Clang where we
work with an arbitrary C runtime library (which may be dynamically
loaded on the target machine) and an arbitrary linker.

~Aaron

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Zack Weinberg]

Demi Marie Obenour <demiobenour@gmail.com> writes:
> On 11/10/22 15:19, Paul Eggert wrote:
>> Here's the main Autoconf issue issue with bool. Traditionally, Autoconf 
>> supported K&R C, C89, C99, etc. At some point (I'm not sure when), 
>> Autoconf started requiring C89 or later. Is it now OK for Autoconf to 
>> require C99 or later, as far as bool is concerned? If so, that'll 
>> considerably simplify the ongoing maintenance hassle for bool.
>>
>> Requiring C99-or-later bool is the option that Gnulib has taken. Its 
>> 'stdbool' module and its gl_C_BOOL macro assumes C99 or later, and as 
>> far as I know no Gnulib-using package is using Gnulib's 'stdbool-c99' 
>> module which we kept around in case somebody still needed bool to work 
>> atop a C89 system. (We considered supporting C23 bool atop C89 but it 
>> was too painful.)
>
> I am fine with this option.

On the whole I’m also fine with this, but I think we need to consider,
separately, two kinds of Autoconf-using packages and three kinds of
compilers.

I think it’s definitely fine if Autoconf-using packages that request
support for ‘bool’, using either AC_C_BOOL or gl_C_BOOL, start requiring
a C99 compiler as of 2.72 (but see below).  I suspect there are existing
packages (Kermit comes to mind) that intend to maintain compatibility
with C89 compilers *forever*, and will choose *not* to use AC_C_BOOL,
and will be annoyed if AC_PROG_CC by itself starts rejecting C89 compilers.

(We may eventually decide we don’t support C89 compilers *at all*
anymore but that should be an independent discussion and not driven by
‘bool’.)

Then, on the compiler side of things, there’s compilers that have
complete support for ‘bool’ as it was specified in C99 (i.e. both the
‘_Bool’ keyword is recognized and a useful ‘stdbool.h’); there’s
compilers that have ‘_Bool’ but *don’t* have a useful ‘stdbool.h’; and
there’s compilers that don’t have any “true Boolean type” (as I put it
in the manual) at all.  In earlier discussions, IIRC, we determined that
compilers in all three of these categories do exist.

I suggest that what we mean by “Packages that use AC_C_BOOL require a
C99 compiler” is precisely “When @code{AC_C_BOOL} is used,
@command{configure} will error out if the C compiler does not recognize
either @samp{bool var;} or @samp{_Bool var;} as a valid complete
translation unit.”  In other words, the third category of compilers, the
ones that don’t have any “true Boolean type” (whose name we know), are
rejected.  However, compilers in the second category (_Bool available,
stdbool.h either unavailable or does not work) will be accepted and
config.h will perform the #defines that stdbool.h ought to have
performed.

> I just checked and both GCC 12.2 and clang 14 support <stdbool.h> in C89
> mode.  I do get a -Wc99-extensions warning from clang but that can easily
> be suppressed with -Wno-c99-extensions.

Good to know, thanks.

zw

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-11 07:11, Aaron Ballman wrote:
> We believe the runtime behavior is sufficiently dangerous to
> warrant a conservative view that any call to a function will be a call
> that gets executed at runtime, hence a definitive signature mismatch
> is something we feel comfortable diagnosing (in some form) by default.

As long as these diagnostics by default do not cause the compiler to 
exit with nonzero status, we should be OK with Autoconf-generated 
'configure' scripts. Although there will be problems with people who run 
"./configure CFLAGS='-Werror'", that sort of usage has always been 
problematic and unsupported by Autoconf, so we can simply continue to 
tell people "don't do that".

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-11 07:11, Aaron Ballman wrote:
> Clang doesn't require such a linker (we work with various system linkers).

As long as the system linkers continue to work as they have 
traditionally worked, we're fine.

> the frontend perspective, we can't tell the difference between
> "trust me this is safe because it never gets executed" and "this is a
> CVE"

If some system linker ever attempts to reject links with mismatched 
signatures, Autoconf-generated code will need to have a way to shut that 
off. I hope Clang maintainers can be cajoled into supporting that, if 
the time comes. Perhaps there can be a #pragma, or a compile-time 
option, to do that.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Aaron Ballman]

On Sat, Nov 12, 2022 at 7:43 PM Paul Eggert <eggert@cs.ucla.edu> wrote:
>
> On 2022-11-11 07:11, Aaron Ballman wrote:
> > We believe the runtime behavior is sufficiently dangerous to
> > warrant a conservative view that any call to a function will be a call
> > that gets executed at runtime, hence a definitive signature mismatch
> > is something we feel comfortable diagnosing (in some form) by default.
>
> As long as these diagnostics by default do not cause the compiler to
> exit with nonzero status, we should be OK with Autoconf-generated
> 'configure' scripts. Although there will be problems with people who run
> "./configure CFLAGS='-Werror'", that sort of usage has always been
> problematic and unsupported by Autoconf, so we can simply continue to
> tell people "don't do that".

That's good to know, but is a problem more generally -- we are
strengthening more diagnostics to be warnings that are treated as an
error by default. This gives our users the best experience in terms of
diagnostic behavior -- they're clearly alerted to serious issues in
their code (either issues of conformance, like with use of implicit
int or implicit function decls in C99 or later, or issues of security
like statically known instances of UB), but they still have the chance
to downgrade the diagnostic back into a warning (good as a temporary
solution to start migrating code) or disable the diagnostic entirely
(good if you plan to never update your compiler version but otherwise
not recommended). Some of these diagnostics are expected to change to
be error-only diagnostics in the future, so this strengthening helps
to set user expectations as well.

That's why it's generally a problem when autoconf relies on invalid
language constructs -- it creates a tension between the autoconf uses
and improving the C ecosystem.  The autoconf uses aren't always
unreasonable, but are very much a special case scenario compared to
general C development. I suspect that as the security posture of the C
language and its implementations improves in response to recent
concerns around suitability of the language
(https://media.defense.gov/2022/Nov/10/2003112742/-1/-1/0/CSI_SOFTWARE_MEMORY_SAFETY.PDF),
this tension will come up more frequently.

~Aaron

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-14 04:41, Aaron Ballman wrote:
> it's generally a problem when autoconf relies on invalid
> language constructs

Autoconf *must* rely on invalid language constructs, if only to test 
whether the language constructs work. And Clang therefore must be 
careful about how it diagnoses invalid constructs. Clang shouldn't 
willy-nilly change the way it reports invalid constructs, as that can 
break Autoconf.

> issues of security
> like statically known instances of UB

It's fine to report those; I'm not saying don't report them. All I'm 
saying is that Clang should be careful about *how* it reports them.

At the very least if there are going to be changes in this area, the 
Clang developers should notify Autoconf (and presumably other) 
downstream users of the changes, and provide a supported way to get the 
old behavior for reporting, and give downstream time to adapt.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Florian Weimer]

* Paul Eggert:

> On 2022-11-14 04:41, Aaron Ballman wrote:
>> it's generally a problem when autoconf relies on invalid
>> language constructs
>
> Autoconf *must* rely on invalid language constructs, if only to test
> whether the language constructs work. And Clang therefore must be 
> careful about how it diagnoses invalid constructs. Clang shouldn't
> willy-nilly change the way it reports invalid constructs, as that can 
> break Autoconf.

This is only true for the status quo.  We could finally band together
and define an interface between autoconf and the toolchain that avoids
feature probing through source code fragments for common cases.  It
might make configure scripts to run quite a bit faster, too.

That being said, system compilers need to be careful when turning
warnings into errors by default, but that doesn't mean we should never
do such changes, particularly when we know based on interactions with
programmers that the warnings are not sufficient for avoiding wasted
time.

Thanks,
Florian

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Aaron Ballman]

On Mon, Nov 14, 2022 at 1:14 PM Paul Eggert <eggert@cs.ucla.edu> wrote:
>
> On 2022-11-14 04:41, Aaron Ballman wrote:
> > it's generally a problem when autoconf relies on invalid
> > language constructs
>
> Autoconf *must* rely on invalid language constructs, if only to test
> whether the language constructs work. And Clang therefore must be
> careful about how it diagnoses invalid constructs. Clang shouldn't
> willy-nilly change the way it reports invalid constructs, as that can
> break Autoconf.
>
> > issues of security
> > like statically known instances of UB
>
> It's fine to report those; I'm not saying don't report them. All I'm
> saying is that Clang should be careful about *how* it reports them.
>
> At the very least if there are going to be changes in this area, the
> Clang developers should notify Autoconf (and presumably other)
> downstream users of the changes, and provide a supported way to get the
> old behavior for reporting, and give downstream time to adapt.

Definitely agreed about the communication aspects! I mentioned this upthread:

FWIW, we're working on improving communication
about potentially disruptive changes to Clang, so you might want to
consider either subscribing to the clang-vendors code review group at
https://reviews.llvm.org/project/members/113/ (if you want to be
involved in code review before things land) or the Announcements
discourse channel at https://discourse.llvm.org/c/announce/ (if you
want to be notified after something lands but before Clang ships).

One other thing we've done recently is starting to call out
potentially disruptive changes in the release notes as well:
https://clang.llvm.org/docs/ReleaseNotes.html#potentially-breaking-changes
-- but this is more for notifying folks after a release goes out, so
one of the other approaches is more proactive if the goal is to alert
Clang developers to serious deployment problems before we ship.

~Aaron

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 1035 bytes --]



> On 13 Nov 2022, at 00:43, Paul Eggert <eggert@cs.ucla.edu> wrote:
> 
> On 2022-11-11 07:11, Aaron Ballman wrote:
>> We believe the runtime behavior is sufficiently dangerous to
>> warrant a conservative view that any call to a function will be a call
>> that gets executed at runtime, hence a definitive signature mismatch
>> is something we feel comfortable diagnosing (in some form) by default.
> 
> As long as these diagnostics by default do not cause the compiler to exit with nonzero status, we should be OK with Autoconf-generated 'configure' scripts. Although there will be problems with people who run "./configure CFLAGS='-Werror'", that sort of usage has always been problematic and unsupported by Autoconf, so we can simply continue to tell people "don't do that".
> 

Is there somewhere in the autoconf docs we actually say this?

I've seen a few instances of folks adding it themselves very
early in their configure scripts (which is a pain for distros
anyway) which then ends up affecting the rest.

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 358 bytes --]

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Zack Weinberg]

On Tue, Nov 15, 2022, at 12:03 AM, Sam James wrote:
>> On 13 Nov 2022, at 00:43, Paul Eggert <eggert@cs.ucla.edu> wrote:
>> 
>> Although there will be problems with people who run "./configure CFLAGS='-Werror'", that sort of usage has always been problematic and unsupported by Autoconf, so we can simply continue to tell people "don't do that".
>> 
>
> Is there somewhere in the autoconf docs we actually say this?
>
> I've seen a few instances of folks adding it themselves very
> early in their configure scripts (which is a pain for distros
> anyway) which then ends up affecting the rest.

It's mentioned in the NEWS entry for 2.70: https://git.savannah.gnu.org/cgit/autoconf.git/tree/NEWS#n170

It should be discussed in the actual manual as well, but I've been reluctant to add anything about warnings to the manual as long as Autoconf proper doesn't have any support for controlling warnings.

Note that there have been bug reports for cases where running builds with more warnings than configure uses (and I think also -Werror), means that configure checks spuriously succeed (i.e. configure reports that something is available, but then you get compiler errors on the code that tries to use it).  I can't remember any concrete examples right now, though.

zw

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 1364 bytes --]



> On 15 Nov 2022, at 13:30, Zack Weinberg <zack@owlfolio.org> wrote:
> 
> On Tue, Nov 15, 2022, at 12:03 AM, Sam James wrote:
>>> On 13 Nov 2022, at 00:43, Paul Eggert <eggert@cs.ucla.edu> wrote:
>>> 
>>> Although there will be problems with people who run "./configure CFLAGS='-Werror'", that sort of usage has always been problematic and unsupported by Autoconf, so we can simply continue to tell people "don't do that".
>>> 
>> 
>> Is there somewhere in the autoconf docs we actually say this?
>> 
>> I've seen a few instances of folks adding it themselves very
>> early in their configure scripts (which is a pain for distros
>> anyway) which then ends up affecting the rest.
> 
> It's mentioned in the NEWS entry for 2.70: https://git.savannah.gnu.org/cgit/autoconf.git/tree/NEWS#n170

Thanks, sorry, I didn't think to check NEWS. This is good enough for me to link folks to, anyway, for the time being.

> Note that there have been bug reports for cases where running builds with more warnings than configure uses (and I think also -Werror), means that configure checks spuriously succeed (i.e. configure reports that something is available, but then you get compiler errors on the code that tries to use it).  I can't remember any concrete examples right now, though.

I can totally imagine that, don't stress about the examples.


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 358 bytes --]

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Jonathan Wakely]

On Mon, 14 Nov 2022 at 18:15, Paul Eggert <eggert@cs.ucla.edu> wrote:
>
> On 2022-11-14 04:41, Aaron Ballman wrote:
> > it's generally a problem when autoconf relies on invalid
> > language constructs
>
> Autoconf *must* rely on invalid language constructs, if only to test
> whether the language constructs work. And Clang therefore must be
> careful about how it diagnoses invalid constructs. Clang shouldn't
> willy-nilly change the way it reports invalid constructs, as that can
> break Autoconf.

There's a difference between checking whether an invalid construct
works, where that construct is the subject of the test, and
incidentally relying on invalid language constructs as part of testing
for other things.



> > issues of security
> > like statically known instances of UB
>
> It's fine to report those; I'm not saying don't report them. All I'm
> saying is that Clang should be careful about *how* it reports them.

Could you clarify what you mean, with a concrete example? Surely as
long as errors are reported on stderr and the compiler exits with
non-zero status, that's an acceptable way to report errors? What kind
of changes to error reporting are you saying to be careful with?

If Clang starts to diagnose a given provable-UB case (or any other
construct) as an error instead of a warning, then it seems entirely
correct for autoconf to report that the case does not work. That's the
desired behaviour, isn't it?

What we don't want is for autoconf to start reporting that *other*
things don't work, as a result of autoconf relying on UB or ill-formed
code when trying to check other things like the presence of function
'foo'. And that's why autoconf should avoid using invalid/undefined
code when possible.

>
> At the very least if there are going to be changes in this area, the
> Clang developers should notify Autoconf (and presumably other)
> downstream users of the changes, and provide a supported way to get the
> old behavior for reporting, and give downstream time to adapt.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-15 06:50, Jonathan Wakely wrote:
> Could you clarify what you mean, with a concrete example? Surely as
> long as errors are reported on stderr and the compiler exits with
> non-zero status, that's an acceptable way to report errors?

Not if the "error" is harmless as far as Autoconf is concerned, which is 
what led to this thread. The concrete example here is that Autoconf 
needs to check whether a function can be linked to (as opposed to 
checking the function's signature). Clang shouldn't get in the way.

In lots of places the C standard says behavior is undefined, even though 
the behavior is fine on the current platform for the intended use. It's 
not just the example we're talking about; adding zero to a null pointer 
is another such example.

In such cases it's OK for Clang to warn, but having Clang exit with 
nonzero status is overkill and counterproductive.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Jonathan Wakely]

On Tue, 15 Nov 2022 at 19:08, Paul Eggert <eggert@cs.ucla.edu> wrote:
>
> On 2022-11-15 06:50, Jonathan Wakely wrote:
> > Could you clarify what you mean, with a concrete example? Surely as
> > long as errors are reported on stderr and the compiler exits with
> > non-zero status, that's an acceptable way to report errors?
>
> Not if the "error" is harmless as far as Autoconf is concerned, which is
> what led to this thread. The concrete example here is that Autoconf
> needs to check whether a function can be linked to (as opposed to
> checking the function's signature). Clang shouldn't get in the way.

Another perspective is that autoconf shouldn't get in the way of
making the C and C++ toolchain more secure by default.

>
> In lots of places the C standard says behavior is undefined, even though
> the behavior is fine on the current platform for the intended use. It's
> not just the example we're talking about; adding zero to a null pointer
> is another such example.
>
> In such cases it's OK for Clang to warn, but having Clang exit with
> nonzero status is overkill and counterproductive.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-15 11:27, Jonathan Wakely wrote:
> Another perspective is that autoconf shouldn't get in the way of
> making the C and C++ toolchain more secure by default.

Can you cite any examples of a real-world security flaw what would be 
found by Clang erroring out because 'char foo(void);' is the wrong 
prototype? Is it plausible that any such security flaw exists?

On the contrary, it's more likely that Clang's erroring out here would 
*introduce* a security flaw, because it would cause 'configure' to 
incorrectly infer that an important security-relevant function is 
missing and that a flawed substitute needs to be used.

Let's focus on real problems rather than worrying about imaginary ones.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Aaron Ballman]

On Tue, Nov 15, 2022 at 2:08 PM Paul Eggert <eggert@cs.ucla.edu> wrote:
>
> On 2022-11-15 06:50, Jonathan Wakely wrote:
> > Could you clarify what you mean, with a concrete example? Surely as
> > long as errors are reported on stderr and the compiler exits with
> > non-zero status, that's an acceptable way to report errors?
>
> Not if the "error" is harmless as far as Autoconf is concerned, which is
> what led to this thread. The concrete example here is that Autoconf
> needs to check whether a function can be linked to (as opposed to
> checking the function's signature). Clang shouldn't get in the way.

What is harmless to autoconf is a critical flaw in another context.

> In lots of places the C standard says behavior is undefined, even though
> the behavior is fine on the current platform for the intended use. It's
> not just the example we're talking about; adding zero to a null pointer
> is another such example.
>
> In such cases it's OK for Clang to warn, but having Clang exit with
> nonzero status is overkill and counterproductive.

I don't know that this is particularly persuasive -- it effectively
boils down to another variant of "I want to rely on a specific
behavior for something that is UB". I don't think Clang can promise
that we're not going to turn more statically-known UB into errors.

~Aaron

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Aaron Ballman]

On Tue, Nov 15, 2022 at 3:27 PM Paul Eggert <eggert@cs.ucla.edu> wrote:
>
> On 2022-11-15 11:27, Jonathan Wakely wrote:
> > Another perspective is that autoconf shouldn't get in the way of
> > making the C and C++ toolchain more secure by default.
>
> Can you cite any examples of a real-world security flaw what would be
> found by Clang erroring out because 'char foo(void);' is the wrong
> prototype? Is it plausible that any such security flaw exists?

CVE-2006-1174 is a possibly reasonable example. That one is
specifically about the K&R C open() interface, but the reason the CVE
happened was because a required argument was not passed which is
exactly the kind of problem you'd get from a prototype mismatch.

I think autoconf's usage pattern is well outside of common C coding
practices. Most folks who call a function expect the call to plausibly
happen at runtime (rather than do so just to see if the linker will
complain or not), and I don't know of another context in which anyone
expects calling a function with an incorrect signature will lead to
good outcomes.

> On the contrary, it's more likely that Clang's erroring out here would
> *introduce* a security flaw, because it would cause 'configure' to
> incorrectly infer that an important security-relevant function is
> missing and that a flawed substitute needs to be used.
>
> Let's focus on real problems rather than worrying about imaginary ones.

If the symbol exists and `configure` says it does not, that's the bug
and it's not with the host compiler. You can run into that same bug
with use of `-Werror`, as others have pointed out. So strengthening
warnings doesn't introduce any NEW problems into autoconf, it
exacerbates existing ones.

~Aaron

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

>> Can you cite any examples of a real-world security flaw what would be
>> found by Clang erroring out because 'char foo(void);' is the wrong
>> prototype? Is it plausible that any such security flaw exists?

> CVE-2006-1174 is a possibly reasonable example.

CVE-2006-1174 is not an example, because no prototype 'char foo(void);' 
was involved.[1]

Here's a more concrete proposal that should allay any realistic concerns 
about CVEs. If Clang decides to change its behavior in this area, so 
that Clang starts to exit with nonzero status if the user declares a 
function with a prototype incompatible with the C library, it would be a 
service to real users if Clang merely issues a warning and does *not* 
error out if the declaration is 'char foo();' (current Autoconf) or 
'char foo(void);' (future Autoconf).

This may be a hack, but it's a *good* hack. It's likely to fix 
real-world bugs that would be caused if Clang becomes overly pedantic by 
default here. And the probability of introducing real-world bugs is 
essentially zero.


> You can run into that same bug
> with use of `-Werror`

That's OK, as the Autoconf documentation already says "don't do that". 
And if Clang exits with nonzero status in the above situation when 
-Werror is specified, that's fine too.


[1] CVE-2006-1174 is not even an example of prototype mismatch, as the 
bad C code did not misdeclare the function in question: it used a POSIX 
standard include file for the function prototype, which is the standard 
way to do things.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Ben Boeckel]

On Tue, Nov 15, 2022 at 15:09:19 -0800, Paul Eggert wrote:
> This may be a hack, but it's a *good* hack. It's likely to fix 
> real-world bugs that would be caused if Clang becomes overly pedantic by 
> default here. And the probability of introducing real-world bugs is 
> essentially zero.

FWIW, CMake uses the same signature for detecting whether a function
exists or not:

    https://gitlab.kitware.com/cmake/cmake/-/blob/master/Modules/CheckFunctionExists.c

It's also been like this (without the `void`) for 20 years; the `void`
argument was added 6 years ago:

    https://gitlab.kitware.com/cmake/cmake/-/commits/master/Modules/CheckFunctionExists.c

--Ben

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Bob Friesenhahn]

On Tue, 15 Nov 2022, Sam James wrote:

>
>
>> On 13 Nov 2022, at 00:43, Paul Eggert <eggert@cs.ucla.edu> wrote:
>>
>> On 2022-11-11 07:11, Aaron Ballman wrote:
>>> We believe the runtime behavior is sufficiently dangerous to
>>> warrant a conservative view that any call to a function will be a call
>>> that gets executed at runtime, hence a definitive signature mismatch
>>> is something we feel comfortable diagnosing (in some form) by default.
>>
>> As long as these diagnostics by default do not cause the compiler to exit with nonzero status, we should be OK with Autoconf-generated 'configure' scripts. Although there will be problems with people who run "./configure CFLAGS='-Werror'", that sort of usage has always been problematic and unsupported by Autoconf, so we can simply continue to tell people "don't do that".
>>
>
> Is there somewhere in the autoconf docs we actually say this?
>
> I've seen a few instances of folks adding it themselves very
> early in their configure scripts (which is a pain for distros
> anyway) which then ends up affecting the rest.

Autoconf can help with this issue due to GCC and some other compilers 
providing extensions (usually a pragma) to control warnings while 
compiling the C code.  So configure can run without -Werror, but 
Autoconf could help by providing an easy way for enabling -Werror 
while compiling the application.

Of course the above does not require Autoconf since application 
developers can figure it out by themselves using preprocessor logic 
and knowledge of compiler-specific behavior.

If Autoconf is able to help, then the convoluted code can be in just 
one place (in Autoconf).

Bob
-- 
Bob Friesenhahn
bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
Public Key,     http://www.simplesystems.org/users/bfriesen/public-key.txt

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Michael Matz]

Hi,

On Tue, 15 Nov 2022, Paul Eggert wrote:

> On 2022-11-15 11:27, Jonathan Wakely wrote:
> > Another perspective is that autoconf shouldn't get in the way of
> > making the C and C++ toolchain more secure by default.
> 
> Can you cite any examples of a real-world security flaw what would be 
> found by Clang erroring out because 'char foo(void);' is the wrong 
> prototype? Is it plausible that any such security flaw exists?
> 
> On the contrary, it's more likely that Clang's erroring out here would 
> *introduce* a security flaw, because it would cause 'configure' to 
> incorrectly infer that an important security-relevant function is 
> missing and that a flawed substitute needs to be used.
> 
> Let's focus on real problems rather than worrying about imaginary ones.

I sympathize, and I would think a compiler emitting an error (not a 
warning) in the situation at hand (in absence of -Werror) is overly 
pedantic.  But, could autoconf perhaps avoid the problem?  AFAICS the 
ac_fn_c_check_func really does only a link test to check for symbol 
existence, and the perceived problem is that the call statement in main() 
invokes UB.  So, let's avoid the call then while retaining the access to 
the symbol?  Like:

-----
char foobar(void);
int main(void) {
  return &foobar != 0;
}
-----

No call involved: no reason for compiler to complain.  The prototype decl 
itself will still be "wrong", but compilers complaining about that (in 
absence of a pre-existing different prototype, which is avoided by 
autoconf) seem unlikely.

Obviously this program will also say "foobar exists" if it's a data 
symbol, but that's the same with the variant using the call on most 
platforms (after all it's not run).

The idea is so obvious that I'm probably missing something, why autoconf 
can't use that idiom instead.  But perhaps the (historic?) reasons why it 
couldn't be used are gone now?


Ciao,
Michael.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Alexander Monakov]

On Wed, 16 Nov 2022, Michael Matz via Gcc wrote:

> I sympathize, and I would think a compiler emitting an error (not a 
> warning) in the situation at hand (in absence of -Werror) is overly 
> pedantic.  But, could autoconf perhaps avoid the problem?  AFAICS the 
> ac_fn_c_check_func really does only a link test to check for symbol 
> existence, and the perceived problem is that the call statement in main() 
> invokes UB.  So, let's avoid the call then while retaining the access to 
> the symbol?  Like:
> 
> -----
> char foobar(void);
> int main(void) {
>   return &foobar != 0;
> }
> -----
> 
> No call involved: no reason for compiler to complain.  The prototype decl 
> itself will still be "wrong", but compilers complaining about that (in 
> absence of a pre-existing different prototype, which is avoided by 
> autoconf) seem unlikely.
> 
> Obviously this program will also say "foobar exists" if it's a data 
> symbol, but that's the same with the variant using the call on most 
> platforms (after all it's not run).
> 
> The idea is so obvious that I'm probably missing something, why autoconf 
> can't use that idiom instead.  But perhaps the (historic?) reasons why it 
> couldn't be used are gone now?

Ironically, modern GCC and LLVM optimize '&foobar != 0' to '1' even at -O0,
and thus no symbol reference remains in the resulting assembly.

Alexander

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Michael Matz]

Hey,

On Wed, 16 Nov 2022, Alexander Monakov wrote:

> > The idea is so obvious that I'm probably missing something, why autoconf 
> > can't use that idiom instead.  But perhaps the (historic?) reasons why it 
> > couldn't be used are gone now?
> 
> Ironically, modern GCC and LLVM optimize '&foobar != 0' to '1' even at -O0,
> and thus no symbol reference remains in the resulting assembly.

Err, right, *head-->table*.
Playing with volatile should help:

char foobar(void);
char (* volatile ptr)(void);
int main(void) {
    ptr = foobar;
    return ptr != 0;
}


Ciao,
Michael.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Richard Biener]

On Wed, Nov 16, 2022 at 4:02 PM Michael Matz via Gcc <gcc@gcc.gnu.org> wrote:
>
> Hey,
>
> On Wed, 16 Nov 2022, Alexander Monakov wrote:
>
> > > The idea is so obvious that I'm probably missing something, why autoconf
> > > can't use that idiom instead.  But perhaps the (historic?) reasons why it
> > > couldn't be used are gone now?
> >
> > Ironically, modern GCC and LLVM optimize '&foobar != 0' to '1' even at -O0,
> > and thus no symbol reference remains in the resulting assembly.
>
> Err, right, *head-->table*.
> Playing with volatile should help:
>
> char foobar(void);
> char (* volatile ptr)(void);
> int main(void) {
>     ptr = foobar;
>     return ptr != 0;
> }

using printf for foobar this works even with GCC 2.95.2 and with trunk
and -Wall diagnoses

t.c:1:6: warning: conflicting types for built-in function 'printf';
expected 'int(const char *, ...)' [-Wbuiltin-declaration-mismatch]
    1 | char printf(void);
      |      ^~~~~~
t.c:1:1: note: 'printf' is declared in header '<stdio.h>'
  +++ |+#include <stdio.h>
    1 | char printf(void);

so without -Werror this should be fine.

Richard.

>
>
> Ciao,
> Michael.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 1578 bytes --]



> On 16 Nov 2022, at 15:27, Richard Biener <richard.guenther@gmail.com> wrote:
> 
> On Wed, Nov 16, 2022 at 4:02 PM Michael Matz via Gcc <gcc@gcc.gnu.org> wrote:
>> 
>> Hey,
>> 
>> On Wed, 16 Nov 2022, Alexander Monakov wrote:
>> 
>>>> The idea is so obvious that I'm probably missing something, why autoconf
>>>> can't use that idiom instead.  But perhaps the (historic?) reasons why it
>>>> couldn't be used are gone now?
>>> 
>>> Ironically, modern GCC and LLVM optimize '&foobar != 0' to '1' even at -O0,
>>> and thus no symbol reference remains in the resulting assembly.
>> 
>> Err, right, *head-->table*.
>> Playing with volatile should help:
>> 
>> char foobar(void);
>> char (* volatile ptr)(void);
>> int main(void) {
>>    ptr = foobar;
>>    return ptr != 0;
>> }
> 
> using printf for foobar this works even with GCC 2.95.2 and with trunk
> and -Wall diagnoses
> 
> t.c:1:6: warning: conflicting types for built-in function 'printf';
> expected 'int(const char *, ...)' [-Wbuiltin-declaration-mismatch]
>    1 | char printf(void);
>      |      ^~~~~~
> t.c:1:1: note: 'printf' is declared in header '<stdio.h>'
>  +++ |+#include <stdio.h>
>    1 | char printf(void);
> 
> so without -Werror this should be fine.
> 
Unrelated but I was a bit tempted to ask for throwing in -Wbuiltin-declaration-mismatch
to default -Werror while Clang 16 was at it, but I suppose we don't want the world to
burn too much, and it's got a very obvious usecase (this one) whereas implicit
func decls are too hard to justify.

> Richard.

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 358 bytes --]

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Michael Matz]

Hello,

On Wed, 16 Nov 2022, Sam James wrote:

> Unrelated but I was a bit tempted to ask for throwing in 
> -Wbuiltin-declaration-mismatch to default -Werror while Clang 16 was at 
> it, but I suppose we don't want the world to burn too much,

:-)  It's IMHO a bug in the standard that it misses "if any of its 
associated headers are included" in the item for reservation of external 
linkage identifiers; it has that for all other items about reserved 
identifiers in the Library clause.  If that restriction were added you 
couldn't justify erroring on the example at hand (because it doesn't 
include e.g. <stdio.h> and then printf wouldn't be reserved).  A warning 
is of course always okay and reasonable.  As is, you could justify 
erroring out, but I too think that would be overzealous.


Ciao,
Michael.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Jonathan Wakely]

On Wed, 16 Nov 2022 at 15:59, Michael Matz <matz@suse.de> wrote:
>
> Hello,
>
> On Wed, 16 Nov 2022, Sam James wrote:
>
> > Unrelated but I was a bit tempted to ask for throwing in
> > -Wbuiltin-declaration-mismatch to default -Werror while Clang 16 was at
> > it, but I suppose we don't want the world to burn too much,
>
> :-)  It's IMHO a bug in the standard that it misses "if any of its
> associated headers are included" in the item for reservation of external
> linkage identifiers; it has that for all other items about reserved
> identifiers in the Library clause.  If that restriction were added you
> couldn't justify erroring on the example at hand (because it doesn't
> include e.g. <stdio.h> and then printf wouldn't be reserved).  A warning
> is of course always okay and reasonable.  As is, you could justify
> erroring out, but I too think that would be overzealous.


I think that's very intentional and not a defect in the standard.

If one TU was allowed to define:

void printf() { }

and have that compiled into the program, then that would cause
unexpected behaviour for every other TU which includes <stdio.h> and
calls printf. They would get the non-standard rogue printf.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Michael Matz]

Hello,

On Wed, 16 Nov 2022, Jonathan Wakely wrote:

> > > Unrelated but I was a bit tempted to ask for throwing in
> > > -Wbuiltin-declaration-mismatch to default -Werror while Clang 16 was at
> > > it, but I suppose we don't want the world to burn too much,
> >
> > :-)  It's IMHO a bug in the standard that it misses "if any of its
> > associated headers are included" in the item for reservation of external
> > linkage identifiers; it has that for all other items about reserved
> > identifiers in the Library clause.  If that restriction were added you
> > couldn't justify erroring on the example at hand (because it doesn't
> > include e.g. <stdio.h> and then printf wouldn't be reserved).  A warning
> > is of course always okay and reasonable.  As is, you could justify
> > erroring out, but I too think that would be overzealous.
> 
> 
> I think that's very intentional and not a defect in the standard.
> 
> If one TU was allowed to define:
> 
> void printf() { }
> 
> and have that compiled into the program, then that would cause
> unexpected behaviour for every other TU which includes <stdio.h> and
> calls printf. They would get the non-standard rogue printf.

True.  But suppose the restriction would be added.  I could argue that 
then your problem program (in some other TU) _does_ include the header, 
hence the identifier would have been reserved and so the above definition 
would have been wrong.  I.e. I think adding the restriction wouldn't allow 
the problematic situation either.

I'm aware that the argument would then invoke all the usual problems of 
what constitutes a full program, and if that includes the library even 
when not including headers and so on.  And in any case currently the 
standard does say they're reserved so it's idle speculation anyway :)


Ciao,
Michael.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Jonathan Wakely]

On Wed, 16 Nov 2022 at 16:34, Michael Matz <matz@suse.de> wrote:
>
> Hello,
>
> On Wed, 16 Nov 2022, Jonathan Wakely wrote:
>
> > > > Unrelated but I was a bit tempted to ask for throwing in
> > > > -Wbuiltin-declaration-mismatch to default -Werror while Clang 16 was at
> > > > it, but I suppose we don't want the world to burn too much,
> > >
> > > :-)  It's IMHO a bug in the standard that it misses "if any of its
> > > associated headers are included" in the item for reservation of external
> > > linkage identifiers; it has that for all other items about reserved
> > > identifiers in the Library clause.  If that restriction were added you
> > > couldn't justify erroring on the example at hand (because it doesn't
> > > include e.g. <stdio.h> and then printf wouldn't be reserved).  A warning
> > > is of course always okay and reasonable.  As is, you could justify
> > > erroring out, but I too think that would be overzealous.
> >
> >
> > I think that's very intentional and not a defect in the standard.
> >
> > If one TU was allowed to define:
> >
> > void printf() { }
> >
> > and have that compiled into the program, then that would cause
> > unexpected behaviour for every other TU which includes <stdio.h> and
> > calls printf. They would get the non-standard rogue printf.
>
> True.  But suppose the restriction would be added.  I could argue that
> then your problem program (in some other TU) _does_ include the header,
> hence the identifier would have been reserved and so the above definition
> would have been wrong.  I.e. I think adding the restriction wouldn't allow
> the problematic situation either.
>
> I'm aware that the argument would then invoke all the usual problems of
> what constitutes a full program, and if that includes the library even
> when not including headers and so on.  And in any case currently the
> standard does say they're reserved so it's idle speculation anyway :)

Since we're idly speculating anyway ...

I'd always assumed the "if any of its associated headers is included"
meant in the current TU, but it doesn't actually say that. Which does
suggest that I can't use the identifier "assert" anywhere in a program
if any TU in the program includes <assert.h>. Huh.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-16 06:26, Michael Matz wrote:
> char foobar(void);
> int main(void) {
>    return &foobar != 0;
> }

That still has undefined behavior according to draft C23, which says 
behavior is undefined when foobar is (say) 'memset_explicit' because the 
declaration 'char memset_explicit(void);' disagrees with draft C23's 
declaration of 'memset_explicit'. It doesn't matter whether there's a 
call to 'memset_explicit'. See draft C23 §6.2.7, which says "All 
declarations that refer to the same object or function shall have 
compatible type; otherwise, the behavior is undefined."

If Clang's threatened pickiness were of some real use elsewhere, it 
might be justifiable for default Clang to break Autoconf. But so far we 
haven't seen real-world uses that would justify this pickiness for 
Autoconf's use of 'char memset_explicit(void);'.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Jeffrey Walton]

On Wed, Nov 16, 2022 at 1:18 PM Paul Eggert <eggert@cs.ucla.edu> wrote:
> ...
> If Clang's threatened pickiness were of some real use elsewhere, it
> might be justifiable for default Clang to break Autoconf. But so far we
> haven't seen real-world uses that would justify this pickiness for
> Autoconf's use of 'char memset_explicit(void);'.

This line of arguments is not persuasive. It is full of logical fallacies.

Maybe the Autoconf folks should argue something along the lines of,
"this is the _only_ way to do it because <...>."

Jeff

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Zack Weinberg]

On Wed, Nov 16, 2022, at 1:17 PM, Paul Eggert wrote:
...
> If Clang's threatened pickiness were of some real use elsewhere, it 
> might be justifiable for default Clang to break Autoconf. But so far we 
> haven't seen real-world uses that would justify this pickiness for 
> Autoconf's use of 'char memset_explicit(void);'.

I don't have time this week to really get into this argument but I want to point out that I'm generally in agreement with Rich Felker's argument (in https://ewontfix.com/13/) that AC_CHECK_FUNC *should not* just probe for linkability of a symbol, because:

 - Not including the appropriate headers means that the probe bypasses compile-time symbol renaming and therefore probes for *the wrong symbol*, potentially causing both false detection and false non-detection (to tie it to another subthread, notice that one of the things -D_TIME_BITS=64 triggers (from glibc's headers) is enable dozens of __REDIRECT annotations in time.h)

 - Only probing the symbol, and not its type signature, means for instance that if the program expects GNU strerror_r but the system provides POSIX strerror_r, or vice versa, Autoconf's check will succeed but the program will fail to compile (in more subtle cases it might be miscompiled instead)

(N.B. I *don't* agree with the assertion at the bottom of that page that "taking explicit action to prevent linking against [symbols intended to be exposed for binary compatibility only], involves hacks that are even worse and more fragile than what configure is doing" -- yes, it sucks that the toolchain support for ELF symbol versioning is still mostly absent, 20 years after the concept was introduced, but `asm(".symver __strtod_l_compat, strtod_l@SOME_CONCRETE_VERSION_TAG")` is straightforward (if cryptic) and robust on all the platforms where it works at all.)

zw

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Michael Matz]

Hello,

On Wed, 16 Nov 2022, Paul Eggert wrote:

> On 2022-11-16 06:26, Michael Matz wrote:
> > char foobar(void);
> > int main(void) {
> >    return &foobar != 0;
> > }
> 
> That still has undefined behavior according to draft C23,

This is correct (and also holds for the actually working variant later, 
with a volatile variable).  If your argument is then that as both 
solutions for the link-test problem are relying on undefined behaviour 
they are equivalent and hence no change is needed, you have a point, but I 
disagree.  In practice one (with the call) will cause more problems than 
the other (with address taking).

> If Clang's threatened pickiness were of some real use elsewhere, it 
> might be justifiable for default Clang to break Autoconf. But so far we 
> haven't seen real-world uses that would justify this pickiness for 
> Autoconf's use of 'char memset_explicit(void);'.

Note that both, GCC and clang, already warn (not error out!) about the 
mismatching decl, even without any headers.  So we are in the pickiness 
era already.

I.e. a C file containing just a single line "char printf(void);" will be 
warned about, by default.  There is about nothing that autoconf could do 
to rectify this, except containing a long list of prototypes for 
well-known functions, with the associated maintenance hassle.  But 
autoconf _can_ do something about how the decls are used in the 
link-tests.


Ciao,
Michael.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Jason Merrill]

[-- Attachment #1: Type: text/plain, Size: 940 bytes --]

On Sat, Nov 12, 2022 at 7:44 PM Paul Eggert <eggert@cs.ucla.edu> wrote:

> On 2022-11-11 07:11, Aaron Ballman wrote:
> > Clang doesn't require such a linker (we work with various system
> linkers).
>
> As long as the system linkers continue to work as they have
> traditionally worked, we're fine.
>
> > the frontend perspective, we can't tell the difference between
> > "trust me this is safe because it never gets executed" and "this is a
> > CVE"
>
> If some system linker ever attempts to reject links with mismatched
> signatures, Autoconf-generated code will need to have a way to shut that
> off. I hope Clang maintainers can be cajoled into supporting that, if
> the time comes. Perhaps there can be a #pragma, or a compile-time
> option, to do that.
>

There has been discussion of the problems with compile-time options
elsewhere in the thread, but the #pragma idea sounds promising, as older
compilers can just ignore it.

Jason

[-- Attachment #2: Type: text/html, Size: 1402 bytes --]

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-16 10:40, Jeffrey Walton wrote:
> This line of arguments is not persuasive. It is full of logical fallacies.

... none of which you stated.

No matter how we solve the problem, it will be a hack that exploits 
"logical fallacies" (whatever that means). However, a reaction "You 
violated the C standard! You deserve to be punished!" is not the best 
one for the overall software ecosystem. Lots of programs violate the C 
standard every day, and Clang supports them anyway.

Yesterday I dealt with this Autoconf bug report:

https://lists.gnu.org/r/autoconf/2022-11/msg00092.html

which basically said, "Here's some longstanding buggy code that uses 
Autoconf. This buggy code happened to work in the previous stable 
Autoconf version, but it stopped working in the bleeding-edge version."

Did I respond, "That's buggy code and it deserves to be punished?" No, I 
responded that it's buggy code that needs to be fixed (and gave a fix), 
but fixing this sort of thing is a hassle for distributors and so I also 
installed a minor hack to bleeding-edge Autoconf that lets the buggy 
code work again, at least for now. 
<https://lists.gnu.org/r/autoconf/2022-11/msg00118.html>

It would help if Clang developers could cooperate to address this 
potential problem with stricter compilation defaults. It's a real 
problem. And it shouldn't require much work on the Clang side to address it.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-16 10:59, Zack Weinberg wrote:
> I'm generally in agreement with Rich Felker's argument (inhttps://ewontfix.com/13/) that AC_CHECK_FUNC*should not*  just probe for linkability of a symbol

So am I. I'm not saying Autoconf should never change here, only that the 
change would not be trivial. It would require changing many configure.ac 
scripts scattered over many software projects, because Autoconf cannot 
be expected to know every signature of every function in every library.

Any such transition could not be done in a week, or a month, or even a 
year. I would guess it would take a decade at least. In the meantime if 
Clang becomes pickier by default it would be helpful if there were a 
well-defined way to shut off Clang's pickiness. If the Clang developers 
provide such a way we can use it; if not, Autoconf will just have to do 
what it always does, and figure a way out anyway (hey! it could drop 
into assembler...).

Things would be simpler if Clang became pickier by default only for 
declarations that are not "char foo();" or "char foo(void);". Then, 
existing 'configure' scripts would still work.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Bruno Haible]

Paul Eggert wrote:
> > AC_CHECK_FUNC *should not*  just probe for linkability of a symbol
> 
> ... Autoconf cannot 
> be expected to know every signature of every function in every library.

Clang will surely not acquire knowledge about "every library", right,
only about the C library according to relevant standards (ISO C, POSIX)?

> ... In the meantime if 
> Clang becomes pickier by default it would be helpful if there were a 
> well-defined way to shut off Clang's pickiness.

To me it seems the problem is not Clang's pickiness, but rather the
assumptions that it makes about the target environment. There is a
documented way to ask Clang and GCC "assume a free-standing implementation,
not an environment that has the ISO C / POSIX / LSB / ... functions":
  1) -ffreestanding         [1][2]
  2) -fno-builtin

Can we assume that these options will continue to work?

If so, all Autoconf needs to do is to pass these options to the compiler
in AC_CHECK_FUNC checks. Then we don't need to add knowledge about each
of the hundreds of standard libc functions into Autoconf.

Bruno

[1] https://clang.llvm.org/docs/ClangCommandLineReference.html
[2] https://gcc.gnu.org/onlinedocs/gcc-12.2.0/gcc/C-Dialect-Options.html

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 11/17/22 13:35, Bruno Haible wrote:

> Clang will surely not acquire knowledge about "every library", right,
> only about the C library according to relevant standards (ISO C, POSIX)?

I don't know the Clang developers' plans. But if I wanted Clang to be 
picky then yes, I'd have it know about every library the program links 
to. It's some work but doable via techniques like -flto. Let's hope this 
isn't enabled by default.


> There is a
> documented way to ask Clang and GCC "assume a free-standing implementation,
> not an environment that has the ISO C / POSIX / LSB / ... functions":
>    1) -ffreestanding         [1][2]
>    2) -fno-builtin

Thanks, good suggestion. If this suffices to pacify Clang, we can use it 
in Autoconf: that is, we cause ./configure to link a simple program with 
-fno-builtin and if that works, then ./configure uses -fno-builtin when 
doing "does this function exist?" link-time tests.

This hack wouldn't suffice if Clang starts doing link-time type checking 
by default. However, link-time type checking doesn't seem to be on the 
Clang developers' to-do list, so we should be OK at least for now.