Gnulib use of -Wl,-rpath

Gnulib use of -Wl,-rpath

[Jeffrey Walton]

Hi Everyone,

I'm trying to track down a failure in building libidn. IDN uses
Gnulib, and IDN's 'make check' is failing on the Gnulib test runner.
IDN's earlier test runners pass, and I think the issue is related
Gnulib.

First, here is a successful runner. IDN's runners 1 and 2 do this:

libtool: link: gcc -g2 -O2 -fsanitize=address -fno-omit-frame-pointer
-march=native -fPIC -pthread -fsanitize=address -Wl,-R
-Wl,/var/sanitize/lib64 -Wl,--enable-new-dtags -o test-verify
test-verify.o -Wl,--no-as-needed  -L/var/sanitize/lib64 libtests.a
../../lib/gl/.libs/libgnu.a libtests.a -ldl -lpthread -pthread
gmake[4]: Leaving directory '/home/build/libidn-1.35/lib/gltests'
gmake  check-TESTS
gmake[4]: Entering directory '/home/build/libidn-1.35/lib/gltests'
gmake[5]: Entering directory '/home/build/libidn-1.35/lib/gltests'
PASS: test-alloca-opt
PASS: test-c-ctype
...

Second, here is a failed Gnulib runner. IDN's runner 3 does this:

libtool: link: gcc -g2 -O2 -fsanitize=address -fno-omit-frame-pointer
-march=native -fPIC -pthread -fsanitize=address -Wl,-R
-Wl,/var/sanitize/lib64 -Wl,--enable-new-dtags -o tst_tld tst_tld.o
-Wl,--no-as-needed  -L/var/sanitize/lib64 libutils.a
../lib/.libs/libidn.so ../gl/.libs/libgnu.a -ldl -lpthread -pthread
-Wl,-rpath -Wl,/home/build/libidn-1.35/lib/.libs -Wl,-rpath
-Wl,/var/sanitize/lib64
gmake[2]: Leaving directory '/home/build/libidn-1.35/tests'
gmake  check-TESTS
gmake[2]: Entering directory '/home/build/libidn-1.35/tests'
gmake[3]: Entering directory '/home/build/libidn-1.35/tests'
FAIL: tst_stringprep
FAIL: tst_punycode
...

In the failed run, notice two things.

(1) my LDFLAGS (Wl,-R -Wl,/var/sanitize/lib64 -Wl,--enable-new-dtags)
got overridden by Gnulib's LDFLAGS (-Wl,-rpath
-Wl,/var/sanitize/lib64).
(2) Gnulib's path does not include /home/build/libidn-1.35/gl/.libs,
which is where Gnulib is located

As I understand things an RPATH cannoth be overridden by
LD_LIBRARY_PATH, while a RUNPATH can be overridden by a
LD_LIBRARY_PATH.

My question are,

(1) why id Gnulib using a rpath instead of a runpath?
(2) why is Gnulib omitting <locations>/gl/.libs frm the runpath?
And maybe a third...
(3) why are runpaths being set for a static archive?

Thanks in advance

==========

This may be relevant:

$ cd libidn-1.35
$ find . -name 'libgnu.*'
./lib/gl/.libs/libgnu.a
./lib/gl/.libs/libgnu.la
./lib/gl/libgnu.la
./gl/.libs/libgnu.a
./gl/.libs/libgnu.la
./gl/libgnu.la

Re: Gnulib use of -Wl,-rpath

[Bruno Haible]

Hi,

Jeffrey Walton wrote:
> Second, here is a failed Gnulib runner. IDN's runner 3 does this:
> 
> libtool: link: gcc -g2 -O2 -fsanitize=address -fno-omit-frame-pointer
> -march=native -fPIC -pthread -fsanitize=address -Wl,-R
> -Wl,/var/sanitize/lib64 -Wl,--enable-new-dtags -o tst_tld tst_tld.o
> -Wl,--no-as-needed  -L/var/sanitize/lib64 libutils.a
> ../lib/.libs/libidn.so ../gl/.libs/libgnu.a -ldl -lpthread -pthread
> -Wl,-rpath -Wl,/home/build/libidn-1.35/lib/.libs -Wl,-rpath
> -Wl,/var/sanitize/lib64
> gmake[2]: Leaving directory '/home/build/libidn-1.35/tests'
> gmake  check-TESTS
> gmake[2]: Entering directory '/home/build/libidn-1.35/tests'
> gmake[3]: Entering directory '/home/build/libidn-1.35/tests'
> FAIL: tst_stringprep
> FAIL: tst_punycode
> ...

In these cases, the following tools are useful (on ELF systems):
  - ldd BINARY
    which tells you how the runtime linker resolves the shared libraries
    in the current situation,
  - objdump -p BINARY | grep '\(NEEDED\|RPATH\|RUNPATH\)'
    which tells you what information is encoded in the binary for the
    runtime linker,
  - readelf -d BINARY | grep '\(NEEDED\|RPATH\|RUNPATH\)'
    likewise.

> As I understand things an RPATH cannoth be overridden by
> LD_LIBRARY_PATH, while a RUNPATH can be overridden by a
> LD_LIBRARY_PATH.

At least that's what [1] says. See also [2].

> In the failed run, notice two things.
> 
> (1) my LDFLAGS (Wl,-R -Wl,/var/sanitize/lib64 -Wl,--enable-new-dtags)
> got overridden by Gnulib's LDFLAGS (-Wl,-rpath
> -Wl,/var/sanitize/lib64).

They get combined, not overridden. Multiple -R or -rpath options are
cumulative.

Did you also try -Wl,--disable-new-dtags instead of -Wl,--enable-new-dtags?
What is the outcome? Do the tests pass or fail?

> (2) Gnulib's path does not include /home/build/libidn-1.35/gl/.libs,
> which is where Gnulib is located

Yes, the LD_LIBRARY_PATH gets set in wrapper scripts that libtool
installs in the build tree, outside the .lib directories.

> My question are,
> 
> (1) why id Gnulib using a rpath instead of a runpath?

The question (raised by [2]) is more: Why are some distros using a runpath
instead of an rpath, breaking 20 years of existing practice (see [3][5]
and [4])?

> (2) why is Gnulib omitting <locations>/gl/.libs frm the runpath?

Answered above.

> (3) why are runpaths being set for a static archive?

This option has no effect on static archives, I guess. Please verify this
using 'objdump' (see above).

Bruno

[1] https://news.ycombinator.com/item?id=14222349
[2] https://stackoverflow.com/questions/52018092/how-to-set-rpath-and-runpath-with-gcc-ld
[3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=847298
[4] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=859732
[5] https://bugs.launchpad.net/ubuntu/+source/eglibc/+bug/1253638

Re: Gnulib use of -Wl,-rpath

[Jeffrey Walton]

On Tue, May 14, 2019 at 5:22 AM Bruno Haible <bruno@clisp.org> wrote:
>
> Hi,
>
> Jeffrey Walton wrote:
> > Second, here is a failed Gnulib runner. IDN's runner 3 does this:
> >
> > libtool: link: gcc -g2 -O2 -fsanitize=address -fno-omit-frame-pointer
> > -march=native -fPIC -pthread -fsanitize=address -Wl,-R
> > -Wl,/var/sanitize/lib64 -Wl,--enable-new-dtags -o tst_tld tst_tld.o
> > -Wl,--no-as-needed  -L/var/sanitize/lib64 libutils.a
> > ../lib/.libs/libidn.so ../gl/.libs/libgnu.a -ldl -lpthread -pthread
> > -Wl,-rpath -Wl,/home/build/libidn-1.35/lib/.libs -Wl,-rpath
> > -Wl,/var/sanitize/lib64
> > gmake[2]: Leaving directory '/home/build/libidn-1.35/tests'
> > gmake  check-TESTS
> > gmake[2]: Entering directory '/home/build/libidn-1.35/tests'
> > gmake[3]: Entering directory '/home/build/libidn-1.35/tests'
> > FAIL: tst_stringprep
> > FAIL: tst_punycode
> > ...
>
> In these cases, the following tools are useful (on ELF systems):
>   - ldd BINARY
>     which tells you how the runtime linker resolves the shared libraries
>     in the current situation,
>   - objdump -p BINARY | grep '\(NEEDED\|RPATH\|RUNPATH\)'
>     which tells you what information is encoded in the binary for the
>     runtime linker,
>   - readelf -d BINARY | grep '\(NEEDED\|RPATH\|RUNPATH\)'
>     likewise.
>
> > As I understand things an RPATH cannoth be overridden by
> > LD_LIBRARY_PATH, while a RUNPATH can be overridden by a
> > LD_LIBRARY_PATH.
>
> At least that's what [1] says. See also [2].
>
> > In the failed run, notice two things.
> >
> > (1) my LDFLAGS (Wl,-R -Wl,/var/sanitize/lib64 -Wl,--enable-new-dtags)
> > got overridden by Gnulib's LDFLAGS (-Wl,-rpath
> > -Wl,/var/sanitize/lib64).
>
> They get combined, not overridden. Multiple -R or -rpath options are
> cumulative.
>
> Did you also try -Wl,--disable-new-dtags instead of -Wl,--enable-new-dtags?
> What is the outcome? Do the tests pass or fail?
>
> > (2) Gnulib's path does not include /home/build/libidn-1.35/gl/.libs,
> > which is where Gnulib is located
>
> Yes, the LD_LIBRARY_PATH gets set in wrapper scripts that libtool
> installs in the build tree, outside the .lib directories.
>
> > My question are,
> >
> > (1) why id Gnulib using a rpath instead of a runpath?
>
> The question (raised by [2]) is more: Why are some distros using a runpath
> instead of an rpath, breaking 20 years of existing practice (see [3][5]
> and [4])?
>
> > (2) why is Gnulib omitting <locations>/gl/.libs frm the runpath?
>
> Answered above.
>
> > (3) why are runpaths being set for a static archive?
>
> This option has no effect on static archives, I guess. Please verify this
> using 'objdump' (see above).

Thanks Bruno.

I worked on this some more this morning.

I ran '$ LD_DEBUG=libs tests/tst_stringprep' and it produced:

     ...
     15300:     calling preinit: tests/tst_stringprep
     15300:
     15300:     /lib64/libasan.so.5: error: symbol lookup error:
undefined symbol: __isoc99_printf (fatal)
     15300:     /lib64/libasan.so.5: error: symbol lookup error:
undefined symbol: __isoc99_sprintf (fatal)
     15300:     /lib64/libasan.so.5: error: symbol lookup error:
undefined symbol: __isoc99_snprintf (fatal)
     15300:     /lib64/libasan.so.5: error: symbol lookup error:
undefined symbol: __isoc99_fprintf (fatal)
     15300:     /lib64/libasan.so.5: error: symbol lookup error:
undefined symbol: __isoc99_vprintf (fatal)
     15300:     /lib64/libasan.so.5: error: symbol lookup error:
undefined symbol: __isoc99_vsprintf (fatal)
     15300:     /lib64/libasan.so.5: error: symbol lookup error:
undefined symbol: __isoc99_vsnprintf (fatal)
     15300:     /lib64/libasan.so.5: error: symbol lookup error:
undefined symbol: __isoc99_vfprintf (fatal)
     15300:     /lib64/libasan.so.5: error: symbol lookup error:
undefined symbol: xdr_quad_t (fatal)
     15300:     /lib64/libasan.so.5: error: symbol lookup error:
undefined symbol: xdr_u_quad_t (fatal)
     15300:     /lib64/libasan.so.5: error: symbol lookup error:
undefined symbol: swift_demangle (fatal)

So I'm thinking I'm getting a bs error message, and I followed it down
a rabbit hole.

I still don't quite understand the why the presence of IDN in
/usr/local affects it, though. They are nearly the same machines. Both
are x86_64, both are Fedora, both have C/C++ developer tools, etc. I
use build scripts so I have reproducible process on all machines I
test on.

The failing machine is Skylake, and the passing machine is Goldmont.
The one difference I see is, failing machine in Fedroa 29, and passing
machine is Fedora 28. It is counter intuitive to me - it seems like
the F28 machine should be missing patches and broken.

Here is a similar report. I don't know if it has something to do with
it: https://github.com/google/sanitizers/issues/957. I've got 64-bit
elf files, so I am not clear on the relation to a 32-bit bug report.

Jeff