[ruby-core:109097] [Ruby master Misc#18691] An option to run `make rbconfig.rb` in a different directory

["jaruga (Jun Aruga)" <noreply@ruby-lang.org>]

Issue #18691 has been updated by jaruga (Jun Aruga).


nobu (Nobuyoshi Nakada) wrote in #note-12:
> jaruga (Jun Aruga) wrote in #note-10:
> > I tested. And I see that the `make install` also needs the `ARCH_FLAG` to build some native extension gems.
> > 
> > ```
> > $ make install ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
> > ```
> 
> That is because extension libraries in the bundled gems are deferred building until `install` right now.
> It is a workaround for gem installation complexity and I recognize it as a bug.

I see. The ruby/ruby source `753da6deca34eb7d5d61a26cf66b014ad77ad51d` I tested includes the commit <https://github.com/ruby/ruby/commit/5c1b76a3a55afeb07116bbd3492303c6b6cd890d> to fix the issue <https://bugs.ruby-lang.org/issues/18373>. However it seems that the issue is still not fixed. It's great if someone will fix it.

> > ```
> >    * build_only_flags:    --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 \
> >                           -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1
> > ```
> 
> BTW, why these two flags are need both, and `--specs` and `-specs` differ?

Oh I didn't notice that. The gcc manual <https://gcc.gnu.org/onlinedocs/gcc/Option-Summary.html> says the `-specs` is correct. However `--specs` also works? I am not sure.

I executed the reproducer below with only the `-specs` options again on my forked repository's branch: <https://github.com/junaruga/ruby/commits/wip/configure-arch-flag>. And the `-specs` flags are not used in the ruby/debug's build. Here is the log.

```
$ ./autogen.sh

$ CFLAGS='-O2' build_only_flags="-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1" ./configure --prefix=$(pwd)/dest/ruby --enable-shared --enable-mkmf-verbose | tee configure.log
...
   * build_only_flags:    -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 \
                          -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1
...
$ grep build_only_flags config.status
ac_cs_config='--prefix=/home/jaruga/git/ruby/ruby/dest/ruby --enable-shared --enable-mkmf-verbose '\''build_only_flags=-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1'\'' CFLAGS=-O2'
  set X /bin/sh './configure'  '--prefix=/home/jaruga/git/ruby/ruby/dest/ruby' '--enable-shared' '--enable-mkmf-verbose' 'build_only_flags=-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1' 'CFLAGS=-O2' $ac_configure_extra_args --no-create --no-recursion
S["configure_args"]=" '--prefix=/home/jaruga/git/ruby/ruby/dest/ruby' '--enable-shared' '--enable-mkmf-verbose' 'build_only_flags=-specs=/usr/lib/rpm/redhat/redhat-harde"\
S["build_only_flags"]="-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1"

$ grep ^ARCH_FLAG Makefile
ARCH_FLAG =  -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1

$ make V=1 2>&1 | tee make.log

$ grep main.c make.log
gcc  -O2 -fPIC -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -DRUBY_DEVEL=1 -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -I. -I.ext/include/x86_64-linux -I./include -I. -I./enc/unicode/14.0.0    -o main.o -c ./main.c
 91% [863/939]  main.c

$ grep redhat rbconfig.rb
  CONFIG["configure_args"] = " '--prefix=/home/jaruga/git/ruby/ruby/dest/ruby' '--enable-shared' '--enable-mkmf-verbose' 'build_only_flags=-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1' 'CFLAGS=-O2'"
  CONFIG["build_only_flags"] = "-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1"

$ make install V=1 ECHO=echo 2>&1 | tee make_install.log

$ vi make_install.log
...
echo Extracting bundle gem debug-1.5.0...
Extracting bundle gem debug-1.5.0...
/usr/local/ruby-3.1.1/bin/ruby --disable=gems -C "." \
    -Itool -rgem-unpack \
    -e 'Gem.unpack("gems/debug-1.5.0.gem", ".bundle/gems", ".bundle/specifications")'
Unpacked gems/debug-1.5.0.gem
rm -fr "./.bundle/gems/debug-1.5.0/".git*
./miniruby -I./lib -I. -I.ext/common  ./tool/runruby.rb --extout=.ext  -- --disable-gems -r./x86_64-linux-fake ./tool/rbinstall.rb --make="make" --dest-dir="" --extout=".ext" --ext-build-dir="./ext" --mflags="" --make-flags=" -- ECHO=echo V=1" --data-mode=0644 --prog-mode=0755 --installed-list .installed.list --mantype="doc"  --gnumake --install=all --rdoc-output=".ext/rdoc" --html-output=".ext/html"
...

$ grep debug.c dest/ruby/lib/ruby/gems/3.2.0+1/extensions/x86_64-linux/3.2.0+1/debug-1.5.0/gem_make.out
echo compiling debug.c
compiling debug.c
gcc -I. -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/x86_64-linux -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/ruby/backward -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1 -I. -DHAVE_RB_ISEQ_PARAMETERS -DHAVE_RB_ISEQ_CODE_LOCATION -DHAVE_RB_ISEQ_TYPE    -fPIC -O2 -fPIC  -o debug.o -c debug.c

$ grep redhat dest/ruby/lib/ruby/gems/3.2.0+1/extensions/x86_64-linux/3.2.0+1/debug-1.5.0/gem_make.out
  => empty
```

> jaruga (Jun Aruga) wrote in #note-11:
> > > So, I want to add the ARCH_FLAG to configure script environment variables for convenience.
> > 
> > Just note. I tried to implement build only flags to inject ARCH_FLAG in Makefile with the patch below (the last commit on <https://github.com/junaruga/ruby/commits/wip/configure-arch-flag>)
> 
> Now this flag will appear in rbconfig.rb file, I think.

Right. The new `CONFIG["build_only_flags"]` appears in the `rbconfig.rb`. Perhaps it might be better to remove the line in the `rbconfig.rb`? I am not sure.

```
$ grep redhat rbconfig.rb
  CONFIG["configure_args"] = " '--prefix=/home/jaruga/git/ruby/ruby/dest/ruby' '--enable-shared' '--enable-mkmf-verbose' 'build_only_flags=-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1' 'CFLAGS=-O2'"
  CONFIG["build_only_flags"] = "-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1"
```


----------------------------------------
Misc #18691: An option to run `make rbconfig.rb` in a different directory
https://bugs.ruby-lang.org/issues/18691#change-98244

* Author: jaruga (Jun Aruga)
* Status: Feedback
* Priority: Normal
----------------------------------------
In a Fedora Ruby RPM packaging, I have a challenge for the current `make rbconfig.rb`. Currently when we build Ruby from source to create the Ruby RPM package, we set compiler flag `CFLAGS` including `--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1'`. The file `/usr/lib/rpm/redhat/redhat-hardened-cc1` is managed in `redhat-rpm-config` RPM package. This is a problem when end users run `gem install <a_gem_with_native_extension>`. Because the `config.status` created by `configure` script and `rbconfig.rb` created by `make` (`make rbconfig`) includes `CFLAGS` including `--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1'`. And the `gem install` checks flags in `rbconfig.rb`. Then end users need to install the redhat-rpm-config RPM to run the `gem install`.

So, we want to create another `rbconfig.rb` file with flags not depending on files in the redhat-rpm-config RPM, and ship in the Ruby RPM, so that end users can run `gem install` without installing `redhat-rpm-config` RPM.

Here are ideal steps to create another `rbconfig.rb` when building Ruby to create the Ruby RPM package. I tested it on the current latest master `381475f02e6b44ae729f9403637b30c445b622e5`.

```
$ autoconf
$ CFLAGS='-O2 --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' ./configure
$ make

$ mkdir -p build/rbconfig
$ cd build/rbconfig
$ CFLAGS='-O2' ../../configure
$ make rbconfig.rb
```

Then we can ship the `build/rbconfig/rbconfig.rb`.

But right now it seems that the `make rbconfig.rb` triggers the process to build `miniruby`, and I want to skip the process.

```
$ pwd
/home/jaruga/git/ruby/ruby/build/rbconfig

$ ls
config.log  config.status*  .ext/  GNUmakefile  Makefile  uncommon.mk

$ make rbconfig.rb
compiling ../../main.c
compiling ../../dmydln.c
...
```

I executed the following commands, to create `miniruby` on the current working directory and not to remake `miniruby` (`make -o miniruby`) to skip the process to build the `miniruby`. But ideally I want to execute only `make rbconfig` to create the `rbconfig.rb` in `build/rbconfig` directory. Do you have any idea about how to improve files such as `configure.ac`, `common.mk` or `tool/mkconfig.rb` to achieve this?

```
$ cp -p ../../miniruby .

$ make -o miniruby rbconfig.rb
/bin/sh ../../tool/ifchange "--timestamp=.rbconfig.time" rbconfig.rb rbconfig.tmp
rbconfig.rb updated

$ diff -u ../../rbconfig.rb rbconfig.rb
--- ../../rbconfig.rb	2022-04-12 18:03:45.484465916 +0200
+++ rbconfig.rb	2022-04-12 18:13:09.248334704 +0200
@@ -44,7 +44,7 @@
   CONFIG["RUBY_SEARCH_PATH"] = ""
   CONFIG["UNIVERSAL_INTS"] = ""
   CONFIG["UNIVERSAL_ARCHNAMES"] = ""
-  CONFIG["configure_args"] = " 'CFLAGS=-O2 --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1'"
+  CONFIG["configure_args"] = " 'CFLAGS=-O2'"
   CONFIG["CONFIGURE"] = "configure"
   CONFIG["vendorarchdir"] = "$(vendorlibdir)/$(sitearch)"
   CONFIG["vendorlibdir"] = "$(vendordir)/$(ruby_version)"
@@ -173,7 +173,7 @@
   CONFIG["OBJEXT"] = "o"
   CONFIG["CPPFLAGS"] = " $(DEFS) $(cppflags)"
   CONFIG["LDFLAGS"] = "-L. -fstack-protector-strong -rdynamic -Wl,-export-dynamic"
-  CONFIG["CFLAGS"] = "-O2 --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1"
+  CONFIG["CFLAGS"] = "-O2"
   CONFIG["STRIP"] = "strip -S -x"
   CONFIG["RANLIB"] = "gcc-ranlib"
   CONFIG["OBJDUMP"] = "objdump"
```




-- 
https://bugs.ruby-lang.org/