Why does gnulib use makefile rules rather than configure?

Why does gnulib use makefile rules rather than configure?

[Paul Smith]

I'm looking at allowing GNU make to use more gnulib facilities, but
I've run into a serious problem.

It seems that a lot of gnulib modules rely on makefile rules added to
Makefile.in to construct files, rather than using traditional configure
replacement .in file conversions.

This is a real issue for me because I've always provided a shell
script, build.sh, which can be used to bootstrap an instance of make if
the user doesn't already have one.

The build.sh script relies on the user first running configure to
detect all the normal system-specific behaviors and generate the
standard configure output files such as config.h etc., but then instead
of running "make" (which they don't have), they run ./build.sh.

However, when using gnulib this no longer works because many gnulib
modules seem to delegate various configuration operations to the
generated makefile, rather than using configure to do it.

As a simple example, consider alloca-opt.  gnulib provides alloca.in.h
then adds a Makefile.am rule to convert it to alloca.h, that uses sed
to replace one value:

  sed -e 's|@''HAVE_ALLOCA_H''@|$(HAVE_ALLOCA_H)|g'

The prevalence of this type of behavior in gnulib means that I either
have to give up on using gnulib with GNU make, or else give up on
providing a bootstrapping script.


I don't see why these replacements couldn't instead be done via
configure and its built-in replacement facilities.  Why can't we add
these headers as AC_CONFIG_FILES() and allow them to be generated by
the configure script, instead of requiring makefile rules to do it?

Re: Why does gnulib use makefile rules rather than configure?

[Paul Eggert]

On 9/8/19 2:06 PM, Paul Smith wrote:
> Why can't we add
> these headers as AC_CONFIG_FILES() and allow them to be generated by
> the configure script, instead of requiring makefile rules to do it?

Makefile rules can do things that an Autoconf substitition can't, or at least 
can't do easily. For example, the arpa_inet module (the 2nd one I looked at in 
response to your email) doesn't merely substitute; it also copies three files' 
contents into lib/arpa_inet.h.

Typically 'make' beats 'configure' to do this sort of thing, since 'make' is 
more powerful - e.g., it can run in parallel.

Admittedly GNU Make is a special case, since you want to build it without having 
'make'. And if it's just a few Gnulib modules and they're not doing anything 
fancy, perhaps we can modify the modules to use 'configure' substitutions 
instead of 'make' rules. On the other hand perhaps it's time to stop worrying 
about building GNU make without a 'make'. We don't worry that GCC can't be built 
on a system without a C compiler, so why worry about building GNU Make on a 
system without 'make'?

Re: Why does gnulib use makefile rules rather than configure?

[Paul Smith]

On Sun, 2019-09-08 at 16:40 -0700, Paul Eggert wrote:
> Admittedly GNU Make is a special case, since you want to build it
> without having 'make'. And if it's just a few Gnulib modules and
> they're not doing anything fancy, perhaps we can modify the modules
> to use 'configure' substitutions instead of 'make' rules.

I enhanced GNU make's build.sh to be fancy enough to manage some basic
substitutions, which are good enough for the current suite of gnulib
modules.  We'll see what happens in the future!

I recognize there are some situations where make snippets are really
the best way, but it seems like they're being used even in places where
configure.ac would be just as simple.  I think it would be a good
"statement of policy" for gnulib that if there are equivalent ways of
handling it, configuration.ac should be preferred over Makefile.  I
understand I'm probably the only person who cares.

> On the other hand perhaps it's time to stop worrying about building
> GNU make without a 'make'. We don't worry that GCC can't be built 
> on a system without a C compiler, so why worry about building GNU
> Make on a system without 'make'?

Well, IMO the two situations are not comparable: no one would expect a
C compiler to be written in (for example) assembly (and anyway what
would your assembler be written in... maybe sh?!?! :)).

I take your point but I daresay that if GCC had managed to not require
a C compiler to build itself, somehow, they would surely be unhappy
about ditching this very useful capability in order to satisfy gnulib
module builds :).

Similarly I'm loathe to ditch this "make-less build" capability for GNU
make unless/until there's no alternative.

Re: Why does gnulib use makefile rules rather than configure?

[Bruno Haible]

Paul Smith wrote:
> I recognize there are some situations where make snippets are really
> the best way, but it seems like they're being used even in places where
> configure.ac would be just as simple.  I think it would be a good
> "statement of policy" for gnulib that if there are equivalent ways of
> handling it, configuration.ac should be preferred over Makefile.

There are several reasons why gnulib uses Makefile.am snippets:

  * [As Paul mentioned] sed substitutions in a Makefile rule are more
    powerful than those supported by AC_CONFIG_FILES.

  * There are cases in which we want to have a file generated in some
    conditions only. (Example: modules/limits-h.) In such a case, there
    is the need to remove the file when the developer switches from
    a configuration which needs a generated limits.h to one that doesn't.
    There is no place in configure.ac where this removal could be done.
    Developers not always use
      make -k distclean; ./configure; make
    Sometimes they also do
      ./configure; make clean; make
    and in such situations a mix between a generation rule in configure.ac
    and a removal rule in the Makefile does not work well.

  * The file hierarchy of the package is not something Gnulib can dictate;
    it has to obey the file hierarchy given by the package's developers.
    Through a long and painful experience (Jim may remember the AC_LIBOBJ
    controversy...) we arrived at the conclusion that
      - putting file/directory names into .m4 files must be avoided,
      - putting file/directory names into the configure.ac part of a
        module description can be done but needs careful coding,
      - putting file/directory names into a Makefile.am snippet is a
        no-brainer.

  * Some packages build specific subdirectories only in specific
    conditions. For example, GNU gettext builds its libasprintf/ directory
    only when a C++ compiler is present. When you have a configure.ac rule
    that generates a file in such a directory, it basically bypasses the
    decision of the top-level Makefile whether to build the particular
    subdirectory.
    Again, this presents the risk of left-over files after "make distclean",
    and similar trouble.

Therefore I disagree with your proposed "statement of policy".

Bruno

Re: Why does gnulib use makefile rules rather than configure?

[Bruno Haible]

Hi Paul,

> This is a real issue for me because I've always provided a shell
> script, build.sh, which can be used to bootstrap an instance of make if
> the user doesn't already have one.

The information about bootstrapping [1][2] gives a different picture
of bootstrapping. For many packages, nowadays, bootstrapping involves
multiple steps:
  - To bootstrap GCC 9.x, one starts with TinyCC, which is used to build
    GCC 2.95.x, which builds GCC 4.7.4 (including C++ support!), with
    which one can then build GCC 9.x. [3]
  - To bootstrap SBCL, one starts with GNU clisp, which can be used to
    build SBCL. [2]

That is, you don't need a shell script that builds GNU make. People can also
have a shell script that builds another 'make' implementation (a BSD 'make',
or GNU make 4.2.1), and with this 'make' binary you can then build
GNU make, including all possible Automake conditionals, gnulib generated
files and so on.

Yes, it's more effort for the Guix people to build something in 2 steps
rather than in 1 step. But it's surmountable.

Bruno

[1] https://guix.gnu.org/manual/en/html_node/Bootstrapping.html
[2] https://bootstrappable.org/best-practises.html
[3] https://bootstrappable.org/projects/mes.html