* test-rwlock1 failing on latest Fedora Rawhide @ 2019-01-23 19:37 Richard W.M. Jones 2019-01-23 20:01 ` Bruno Haible 0 siblings, 1 reply; 14+ messages in thread From: Richard W.M. Jones @ 2019-01-23 19:37 UTC (permalink / raw) To: bug-gnulib I cannot reproduce this locally, hence my bug report is rather devoid of details. However, it's 100% reproducible in Koji (the Fedora Rawhide build system) on *all* architectures except armv7: FAIL: test-rwlock1 ================== Unexpected outcome 3 FAIL: test-thread_create The full log is here: https://kojipkgs.fedoraproject.org//work/tasks/4382/32214382/build.log and the full logs on all other architectures are linked from: https://koji.fedoraproject.org/koji/taskinfo?taskID=32214341 Oddly the exact same thing was reported last year, but *only* on armv7. Unfortunately there was no resolution: https://lists.gnu.org/archive/html/coreutils/2018-02/msg00023.html Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: test-rwlock1 failing on latest Fedora Rawhide 2019-01-23 19:37 test-rwlock1 failing on latest Fedora Rawhide Richard W.M. Jones @ 2019-01-23 20:01 ` Bruno Haible 2019-01-23 20:45 ` Richard W.M. Jones 0 siblings, 1 reply; 14+ messages in thread From: Bruno Haible @ 2019-01-23 20:01 UTC (permalink / raw) To: bug-gnulib Hi Rich, > I cannot reproduce this locally, hence my bug report is rather devoid > of details. However, it's 100% reproducible in Koji (the Fedora > Rawhide build system) on *all* architectures except armv7: > > FAIL: test-rwlock1 > ================== > Unexpected outcome 3 > FAIL: test-thread_create As written in [1]: "Unexpected outcome 3" means that the test program could not create a second thread (other than the main thread). On glibc systems, this typically means that the 'test-rwlock1' program has not been linked with '-lpthread'. > The full log is here: > > https://kojipkgs.fedoraproject.org//work/tasks/4382/32214382/build.log The configure command-line looks right. I see LDFLAGS='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld' A global "-Wl,--as-needed" has the effect of removing libraries like -lpthread from the link. Which would explain the symptom. What do the other elements of LDFLAGS do? Bruno [1] http://lists.gnu.org/archive/html/coreutils/2018-02/msg00020.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: test-rwlock1 failing on latest Fedora Rawhide 2019-01-23 20:01 ` Bruno Haible @ 2019-01-23 20:45 ` Richard W.M. Jones 2019-01-23 21:03 ` Dmitry V. Levin 0 siblings, 1 reply; 14+ messages in thread From: Richard W.M. Jones @ 2019-01-23 20:45 UTC (permalink / raw) To: Bruno Haible; +Cc: bug-gnulib On Wed, Jan 23, 2019 at 09:01:19PM +0100, Bruno Haible wrote: > Hi Rich, > > > I cannot reproduce this locally, hence my bug report is rather devoid > > of details. However, it's 100% reproducible in Koji (the Fedora > > Rawhide build system) on *all* architectures except armv7: > > > > FAIL: test-rwlock1 > > ================== > > Unexpected outcome 3 > > FAIL: test-thread_create > > As written in [1]: > "Unexpected outcome 3" means that the test program could not create a second > thread (other than the main thread). > > On glibc systems, this typically means that the 'test-rwlock1' program has not > been linked with '-lpthread'. > > > The full log is here: > > > > https://kojipkgs.fedoraproject.org//work/tasks/4382/32214382/build.log > > The configure command-line looks right. > > I see > LDFLAGS='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld' > > A global "-Wl,--as-needed" has the effect of removing libraries like > -lpthread from the link. Which would explain the symptom. Ahhhhhhh, that would also explain why I couldn't reproduce it. I was using the default flags when I was testing locally. Using the Fedora flags, I *can* reproduce it locally, thanks! > What do the other elements of LDFLAGS do? All the flags AFAIK come from Fedora and are needed for hardening or other Fedora policy purposes. I checked the history of the Fedora package which adds these flags, and it seems like --as-needed was added for: https://fedoraproject.org/wiki/Changes/RemoveExcessiveLinking Removing -Wl,--as-needed fixes the problem. However I'm still unclear about this. Is Fedora wrong? Is hivex using gnulib wrongly? Is gnulib wrong? Rich. > Bruno > > [1] http://lists.gnu.org/archive/html/coreutils/2018-02/msg00020.html -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: test-rwlock1 failing on latest Fedora Rawhide 2019-01-23 20:45 ` Richard W.M. Jones @ 2019-01-23 21:03 ` Dmitry V. Levin 2019-01-23 21:14 ` Richard W.M. Jones 2019-01-23 21:17 ` Michael Hudson-Doyle 0 siblings, 2 replies; 14+ messages in thread From: Dmitry V. Levin @ 2019-01-23 21:03 UTC (permalink / raw) To: Richard W.M. Jones; +Cc: bug-gnulib, Bruno Haible [-- Attachment #1: Type: text/plain, Size: 714 bytes --] Hi, On Wed, Jan 23, 2019 at 08:45:06PM +0000, Richard W.M. Jones wrote: > On Wed, Jan 23, 2019 at 09:01:19PM +0100, Bruno Haible wrote: [...] > I checked the history of the Fedora package which adds these flags, and > it seems like --as-needed was added for: > > https://fedoraproject.org/wiki/Changes/RemoveExcessiveLinking > > Removing -Wl,--as-needed fixes the problem. However I'm still unclear > about this. Is Fedora wrong? Is hivex using gnulib wrongly? Is > gnulib wrong? We also build everything with --as-needed by default, but gnulib does the right thing nowadays by using -Wl,--push-state -Wl,--no-as-needed -lpthread -Wl,--pop-state instead of plain -lpthread. -- ldv [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 801 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: test-rwlock1 failing on latest Fedora Rawhide 2019-01-23 21:03 ` Dmitry V. Levin @ 2019-01-23 21:14 ` Richard W.M. Jones 2019-01-23 22:09 ` Dmitry V. Levin 2019-01-23 21:17 ` Michael Hudson-Doyle 1 sibling, 1 reply; 14+ messages in thread From: Richard W.M. Jones @ 2019-01-23 21:14 UTC (permalink / raw) To: Dmitry V. Levin; +Cc: bug-gnulib, Bruno Haible On Thu, Jan 24, 2019 at 12:03:21AM +0300, Dmitry V. Levin wrote: > Hi, > > On Wed, Jan 23, 2019 at 08:45:06PM +0000, Richard W.M. Jones wrote: > > On Wed, Jan 23, 2019 at 09:01:19PM +0100, Bruno Haible wrote: > [...] > > I checked the history of the Fedora package which adds these flags, and > > it seems like --as-needed was added for: > > > > https://fedoraproject.org/wiki/Changes/RemoveExcessiveLinking > > > > Removing -Wl,--as-needed fixes the problem. However I'm still unclear > > about this. Is Fedora wrong? Is hivex using gnulib wrongly? Is > > gnulib wrong? > > We also build everything with --as-needed by default, but > gnulib does the right thing nowadays by using > -Wl,--push-state -Wl,--no-as-needed -lpthread -Wl,--pop-state > instead of plain -lpthread. hivex is using a very recent gnulib (34290cb926). The test is linked with: /bin/sh ../../libtool --tag=CC --preserve-dup-deps --mode=link gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -o test-thread_create test-thread_create.o libtests.a ../../gnulib/lib/libgnu.la libtests.a -pthread -Wl,--push-state -Wl,--no-as-needed -lpthread -Wl,--pop-state As you can see from the last bit, the --push-state ... --pop-state seems correct. Yet the test still fails. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: test-rwlock1 failing on latest Fedora Rawhide 2019-01-23 21:14 ` Richard W.M. Jones @ 2019-01-23 22:09 ` Dmitry V. Levin 2019-01-23 22:24 ` Richard W.M. Jones 0 siblings, 1 reply; 14+ messages in thread From: Dmitry V. Levin @ 2019-01-23 22:09 UTC (permalink / raw) To: Richard W.M. Jones; +Cc: bug-gnulib, Bruno Haible [-- Attachment #1: Type: text/plain, Size: 2110 bytes --] On Wed, Jan 23, 2019 at 09:14:30PM +0000, Richard W.M. Jones wrote: > On Thu, Jan 24, 2019 at 12:03:21AM +0300, Dmitry V. Levin wrote: > > Hi, > > > > On Wed, Jan 23, 2019 at 08:45:06PM +0000, Richard W.M. Jones wrote: > > > On Wed, Jan 23, 2019 at 09:01:19PM +0100, Bruno Haible wrote: > > [...] > > > I checked the history of the Fedora package which adds these flags, and > > > it seems like --as-needed was added for: > > > > > > https://fedoraproject.org/wiki/Changes/RemoveExcessiveLinking > > > > > > Removing -Wl,--as-needed fixes the problem. However I'm still unclear > > > about this. Is Fedora wrong? Is hivex using gnulib wrongly? Is > > > gnulib wrong? > > > > We also build everything with --as-needed by default, but > > gnulib does the right thing nowadays by using > > -Wl,--push-state -Wl,--no-as-needed -lpthread -Wl,--pop-state > > instead of plain -lpthread. > > hivex is using a very recent gnulib (34290cb926). > > The test is linked with: > > /bin/sh ../../libtool --tag=CC --preserve-dup-deps --mode=link gcc -O2 > -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 > -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong > -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 > -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic > -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection > -Wl,-z,relro -Wl,--as-needed -Wl,-z,now > -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -o test-thread_create > test-thread_create.o libtests.a ../../gnulib/lib/libgnu.la libtests.a > -pthread -Wl,--push-state -Wl,--no-as-needed -lpthread -Wl,--pop-state > > As you can see from the last bit, the --push-state ... --pop-state > seems correct. Yet the test still fails. I see libtool is involved, and libtool is known to reorder arguments in a very unhelpful way. Could you show the actual link command invoked by libtool, please? I expect to see something like -Wl,--push-state -Wl,--no-as-needed -Wl,--pop-state ... -lpthread which doesn't make sense. -- ldv [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 801 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: test-rwlock1 failing on latest Fedora Rawhide 2019-01-23 22:09 ` Dmitry V. Levin @ 2019-01-23 22:24 ` Richard W.M. Jones 2019-01-23 22:38 ` Dmitry V. Levin 0 siblings, 1 reply; 14+ messages in thread From: Richard W.M. Jones @ 2019-01-23 22:24 UTC (permalink / raw) To: Dmitry V. Levin; +Cc: bug-gnulib, Bruno Haible On Thu, Jan 24, 2019 at 01:09:52AM +0300, Dmitry V. Levin wrote: > On Wed, Jan 23, 2019 at 09:14:30PM +0000, Richard W.M. Jones wrote: > > On Thu, Jan 24, 2019 at 12:03:21AM +0300, Dmitry V. Levin wrote: > > > Hi, > > > > > > On Wed, Jan 23, 2019 at 08:45:06PM +0000, Richard W.M. Jones wrote: > > > > On Wed, Jan 23, 2019 at 09:01:19PM +0100, Bruno Haible wrote: > > > [...] > > > > I checked the history of the Fedora package which adds these flags, and > > > > it seems like --as-needed was added for: > > > > > > > > https://fedoraproject.org/wiki/Changes/RemoveExcessiveLinking > > > > > > > > Removing -Wl,--as-needed fixes the problem. However I'm still unclear > > > > about this. Is Fedora wrong? Is hivex using gnulib wrongly? Is > > > > gnulib wrong? > > > > > > We also build everything with --as-needed by default, but > > > gnulib does the right thing nowadays by using > > > -Wl,--push-state -Wl,--no-as-needed -lpthread -Wl,--pop-state > > > instead of plain -lpthread. > > > > hivex is using a very recent gnulib (34290cb926). > > > > The test is linked with: > > > > /bin/sh ../../libtool --tag=CC --preserve-dup-deps --mode=link gcc -O2 > > -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 > > -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong > > -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 > > -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic > > -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection > > -Wl,-z,relro -Wl,--as-needed -Wl,-z,now > > -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -o test-thread_create > > test-thread_create.o libtests.a ../../gnulib/lib/libgnu.la libtests.a > > -pthread -Wl,--push-state -Wl,--no-as-needed -lpthread -Wl,--pop-state > > > > As you can see from the last bit, the --push-state ... --pop-state > > seems correct. Yet the test still fails. > > I see libtool is involved, and libtool is known to reorder arguments > in a very unhelpful way. Could you show the actual link command invoked > by libtool, please? I expect to see something like > -Wl,--push-state -Wl,--no-as-needed -Wl,--pop-state ... -lpthread > which doesn't make sense. I've posted the entire build/check with V=1 here: http://oirase.annexia.org/tmp/2019-01-gnulib-hivex.txt If you grep through that for "test-thread_create" you should find the appropriate lines. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: test-rwlock1 failing on latest Fedora Rawhide 2019-01-23 22:24 ` Richard W.M. Jones @ 2019-01-23 22:38 ` Dmitry V. Levin 2019-01-24 3:18 ` Bruno Haible 2019-01-24 8:50 ` Richard W.M. Jones 0 siblings, 2 replies; 14+ messages in thread From: Dmitry V. Levin @ 2019-01-23 22:38 UTC (permalink / raw) To: Richard W.M. Jones; +Cc: bug-gnulib, Bruno Haible [-- Attachment #1: Type: text/plain, Size: 4069 bytes --] On Wed, Jan 23, 2019 at 10:24:26PM +0000, Richard W.M. Jones wrote: > On Thu, Jan 24, 2019 at 01:09:52AM +0300, Dmitry V. Levin wrote: > > On Wed, Jan 23, 2019 at 09:14:30PM +0000, Richard W.M. Jones wrote: > > > On Thu, Jan 24, 2019 at 12:03:21AM +0300, Dmitry V. Levin wrote: > > > > Hi, > > > > > > > > On Wed, Jan 23, 2019 at 08:45:06PM +0000, Richard W.M. Jones wrote: > > > > > On Wed, Jan 23, 2019 at 09:01:19PM +0100, Bruno Haible wrote: > > > > [...] > > > > > I checked the history of the Fedora package which adds these flags, and > > > > > it seems like --as-needed was added for: > > > > > > > > > > https://fedoraproject.org/wiki/Changes/RemoveExcessiveLinking > > > > > > > > > > Removing -Wl,--as-needed fixes the problem. However I'm still unclear > > > > > about this. Is Fedora wrong? Is hivex using gnulib wrongly? Is > > > > > gnulib wrong? > > > > > > > > We also build everything with --as-needed by default, but > > > > gnulib does the right thing nowadays by using > > > > -Wl,--push-state -Wl,--no-as-needed -lpthread -Wl,--pop-state > > > > instead of plain -lpthread. > > > > > > hivex is using a very recent gnulib (34290cb926). > > > > > > The test is linked with: > > > > > > /bin/sh ../../libtool --tag=CC --preserve-dup-deps --mode=link gcc -O2 > > > -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 > > > -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong > > > -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 > > > -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic > > > -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection > > > -Wl,-z,relro -Wl,--as-needed -Wl,-z,now > > > -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -o test-thread_create > > > test-thread_create.o libtests.a ../../gnulib/lib/libgnu.la libtests.a > > > -pthread -Wl,--push-state -Wl,--no-as-needed -lpthread -Wl,--pop-state > > > > > > As you can see from the last bit, the --push-state ... --pop-state > > > seems correct. Yet the test still fails. > > > > I see libtool is involved, and libtool is known to reorder arguments > > in a very unhelpful way. Could you show the actual link command invoked > > by libtool, please? I expect to see something like > > -Wl,--push-state -Wl,--no-as-needed -Wl,--pop-state ... -lpthread > > which doesn't make sense. > > I've posted the entire build/check with V=1 here: > > http://oirase.annexia.org/tmp/2019-01-gnulib-hivex.txt > > If you grep through that for "test-thread_create" you should find the > appropriate lines. According to the log, it was libtool that broke a valid command /bin/sh ../../libtool --tag=CC --preserve-dup-deps --mode=link gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -o test-rwlock1 test-rwlock1.o libtests.a ../../gnulib/lib/libgnu.la libtests.a -pthread -Wl,--push-state -Wl,--no-as-needed -lpthread -Wl,--pop-state -lrt into an invalid command libtool: link: gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wl,-z -Wl,relro -Wl,--as-needed -Wl,-z -Wl,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -o test-rwlock1 test-rwlock1.o -pthread -Wl,--push-state -Wl,--no-as-needed -Wl,--pop-state libtests.a ../../gnulib/lib/.libs/libgnu.a libtests.a -lpthread -lrt -pthread What's the version of libtool that's produced such a mess? -- ldv [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 801 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: test-rwlock1 failing on latest Fedora Rawhide 2019-01-23 22:38 ` Dmitry V. Levin @ 2019-01-24 3:18 ` Bruno Haible 2019-01-24 3:31 ` Bruno Haible 2019-01-24 8:50 ` Richard W.M. Jones 1 sibling, 1 reply; 14+ messages in thread From: Bruno Haible @ 2019-01-24 3:18 UTC (permalink / raw) To: Dmitry V. Levin; +Cc: bug-gnulib Dmitry V. Levin wrote: > According to the log, it was libtool that broke a valid command > > /bin/sh ../../libtool --tag=CC --preserve-dup-deps --mode=link gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -o test-rwlock1 test-rwlock1.o libtests.a ../../gnulib/lib/libgnu.la libtests.a -pthread -Wl,--push-state -Wl,--no-as-needed -lpthread -Wl,--pop-state -lrt > > into an invalid command > > libtool: link: gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wl,-z -Wl,relro -Wl,--as-needed -Wl,-z -Wl,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -o test-rwlock1 test-rwlock1.o -pthread -Wl,--push-state -Wl,--no-as-needed -Wl,--pop-state libtests.a ../../gnulib/lib/.libs/libgnu.a libtests.a -lpthread -lrt -pthread > > What's the version of libtool that's produced such a mess? Even the latest released libtool versions do this. This patch should fix the issue. Pushed. 2019-01-23 Bruno Haible <bruno@clisp.org> thread: Force linking with -lpthread, even when --as-needed is in use. Reported by Richard W.M. Jones <rjones@redhat.com> in <https://lists.gnu.org/archive/html/bug-gnulib/2019-01/msg00123.html>. * lib/glthread/thread.h (pthread_create): Don't declare weak. diff --git a/lib/glthread/thread.h b/lib/glthread/thread.h index ca40508..1d2a544 100644 --- a/lib/glthread/thread.h +++ b/lib/glthread/thread.h @@ -133,8 +133,6 @@ extern int glthread_in_use (void); call to foo(...) in the same function. To avoid this, we test the address of a function in libpthread that we don't use. */ -# pragma weak pthread_create - # ifndef pthread_sigmask /* Do not declare rpl_pthread_sigmask weak. */ # pragma weak pthread_sigmask # endif ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: test-rwlock1 failing on latest Fedora Rawhide 2019-01-24 3:18 ` Bruno Haible @ 2019-01-24 3:31 ` Bruno Haible 2019-01-24 3:49 ` Bruno Haible 0 siblings, 1 reply; 14+ messages in thread From: Bruno Haible @ 2019-01-24 3:31 UTC (permalink / raw) To: Dmitry V. Levin; +Cc: bug-gnulib Let me summarize the issue. (1) The distro sets LDFLAGS="-Wl,--as-needed". So, libraries that not are not required through at least 1 symbol are not linked in. (2) threadlib.m4 defines LIBMULTITHREAD in a way that is meant to defeat -Wl,--as-needed. But it does not work, due to the reordering of options done by libtool. So, --as-needed is still in effect. (3) threadlib.m4 enables weak symbols, and thread.h and lock.h define specific symbols as weak. Weak symbols are ignored in (1). So, the fix is to define fewer symbols as weak. pthread_create was defined as weak, so that users of gnulib could use glthread_create without linking with -lpthread; it would then return ENOSYS. Now, the use of glthread_create will require linking with -lpthread, on platforms like glibc. It may be argued that this is actually a good thing. Bruno ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: test-rwlock1 failing on latest Fedora Rawhide 2019-01-24 3:31 ` Bruno Haible @ 2019-01-24 3:49 ` Bruno Haible 2019-01-24 8:57 ` Richard W.M. Jones 0 siblings, 1 reply; 14+ messages in thread From: Bruno Haible @ 2019-01-24 3:49 UTC (permalink / raw) To: Dmitry V. Levin; +Cc: bug-gnulib > (2) threadlib.m4 defines LIBMULTITHREAD in a way that is meant to defeat > -Wl,--as-needed. But it does not work, due to the reordering of > options done by libtool. Let me take out the code that does not work. 2019-01-23 Bruno Haible <bruno@clisp.org> threadlib: Revert commit from 2018-06-25. We now have a better fix. * m4/threadlib.m4 (gl_THREADLIB_BODY): Don't attempt to defeat a preceding -Wl,--as-needed option. Don't check whether the linker supports --as-needed/--no-as-needed and --push-state/--pop-state. diff --git a/m4/threadlib.m4 b/m4/threadlib.m4 index f7841b6..bfc3bac 100644 --- a/m4/threadlib.m4 +++ b/m4/threadlib.m4 @@ -1,4 +1,4 @@ -# threadlib.m4 serial 15 +# threadlib.m4 serial 16 dnl Copyright (C) 2005-2019 Free Software Foundation, Inc. dnl This file is free software; the Free Software Foundation dnl gives unlimited permission to copy and/or distribute it, @@ -151,36 +151,6 @@ int main () *" -static "*) gl_cv_have_weak=no ;; esac ]) - dnl Check whether the linker supports the --as-needed/--no-as-needed options. - dnl Assume GCC, so that we can use the -Wl option. - AC_CACHE_CHECK([whether the linker supports --as-needed], - [gl_cv_linker_have_as_needed], - [if test -n "$GCC"; then - gl_saved_ldflags="$LDFLAGS" - LDFLAGS="$gl_saved_ldflags -Wl,--as-needed -Wl,--no-as-needed" - AC_LINK_IFELSE([AC_LANG_PROGRAM()], - [gl_cv_linker_have_as_needed=yes], - [gl_cv_linker_have_as_needed=no]) - LDFLAGS="$gl_saved_ldflags" - else - gl_cv_linker_have_as_needed=no - fi - ]) - dnl Check whether the linker supports the --push-state/--pop-state options. - dnl Assume GCC, so that we can use the -Wl option. - AC_CACHE_CHECK([whether the linker supports --push-state], - [gl_cv_linker_have_push_state], - [if test -n "$GCC"; then - gl_saved_ldflags="$LDFLAGS" - LDFLAGS="$gl_saved_ldflags -Wl,--push-state -Wl,--pop-state" - AC_LINK_IFELSE([AC_LANG_PROGRAM()], - [gl_cv_linker_have_push_state=yes], - [gl_cv_linker_have_push_state=no]) - LDFLAGS="$gl_saved_ldflags" - else - gl_cv_linker_have_push_state=no - fi - ]) if test "$gl_use_threads" = yes || test "$gl_use_threads" = posix; then # On OSF/1, the compiler needs the flag -pthread or -D_REENTRANT so that # it groks <pthread.h>. It's added above, in gl_THREADLIB_EARLY_BODY. @@ -259,32 +229,6 @@ int main () [Define if references to the POSIX multithreading library should be made weak.]) LIBTHREAD= LTLIBTHREAD= - dnl On platforms where GCC enables --as-needed by default, attempt - dnl to make sure that LIBMULTITHREAD really links with -lpthread. - dnl Otherwise linking with LIBMULTITHREAD has no effect; then - dnl the weak symbols are not defined and thus evaluate to NULL. - case "$LIBMULTITHREAD" in - "") ;; - -pthread) - if test $gl_cv_linker_have_as_needed = yes; then - if test $gl_cv_linker_have_push_state = yes; then - LIBMULTITHREAD="$LIBMULTITHREAD -Wl,--push-state -Wl,--no-as-needed -lpthread -Wl,--pop-state" - else - LIBMULTITHREAD="$LIBMULTITHREAD -Wl,--no-as-needed -lpthread" - fi - fi - ;; - *) - if test $gl_cv_linker_have_as_needed = yes; then - if test $gl_cv_linker_have_push_state = yes; then - LIBMULTITHREAD="-Wl,--push-state -Wl,--no-as-needed $LIBMULTITHREAD -Wl,--pop-state" - else - LIBMULTITHREAD="-Wl,--no-as-needed $LIBMULTITHREAD" - fi - fi - ;; - esac - # TODO: May need to modify LTLIBMULTITHREAD similarly. fi fi fi @@ -317,18 +261,6 @@ int main () [Define if references to the old Solaris multithreading library should be made weak.]) LIBTHREAD= LTLIBTHREAD= - dnl On platforms where GCC enables --as-needed by default, attempt - dnl to make sure that LIBMULTITHREAD really links with -lthread. - dnl Otherwise linking with LIBMULTITHREAD has no effect; then - dnl the weak symbols are not defined and thus evaluate to NULL. - if test $gl_cv_linker_have_as_needed = yes; then - if test $gl_cv_linker_have_push_state = yes; then - LIBMULTITHREAD="-Wl,--push-state -Wl,--no-as-needed $LIBMULTITHREAD -Wl,--pop-state" - else - LIBMULTITHREAD="-Wl,--no-as-needed $LIBMULTITHREAD" - fi - fi - # TODO: May need to modify LTLIBMULTITHREAD similarly. fi fi fi @@ -357,18 +289,6 @@ int main () [Define if references to the GNU Pth multithreading library should be made weak.]) LIBTHREAD= LTLIBTHREAD= - dnl On platforms where GCC enables --as-needed by default, attempt - dnl to make sure that LIBMULTITHREAD really links with -lpth. - dnl Otherwise linking with LIBMULTITHREAD has no effect; then - dnl the weak symbols are not defined and thus evaluate to NULL. - if test $gl_cv_linker_have_as_needed = yes; then - if test $gl_cv_linker_have_push_state = yes; then - LIBMULTITHREAD="-Wl,--push-state -Wl,--no-as-needed $LIBMULTITHREAD -Wl,--pop-state" - else - LIBMULTITHREAD="-Wl,--no-as-needed $LIBMULTITHREAD" - fi - fi - # TODO: May need to modify LTLIBMULTITHREAD similarly. fi fi else ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: test-rwlock1 failing on latest Fedora Rawhide 2019-01-24 3:49 ` Bruno Haible @ 2019-01-24 8:57 ` Richard W.M. Jones 0 siblings, 0 replies; 14+ messages in thread From: Richard W.M. Jones @ 2019-01-24 8:57 UTC (permalink / raw) To: Bruno Haible; +Cc: bug-gnulib I see two patches that went into gnulib overnight. I have now tried compiling hivex with gnulib at commit 34881aff4043, and that seems to have fixed the problem, thanks all. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: test-rwlock1 failing on latest Fedora Rawhide 2019-01-23 22:38 ` Dmitry V. Levin 2019-01-24 3:18 ` Bruno Haible @ 2019-01-24 8:50 ` Richard W.M. Jones 1 sibling, 0 replies; 14+ messages in thread From: Richard W.M. Jones @ 2019-01-24 8:50 UTC (permalink / raw) To: Dmitry V. Levin; +Cc: bug-gnulib, Bruno Haible On Thu, Jan 24, 2019 at 01:38:28AM +0300, Dmitry V. Levin wrote: > On Wed, Jan 23, 2019 at 10:24:26PM +0000, Richard W.M. Jones wrote: > > On Thu, Jan 24, 2019 at 01:09:52AM +0300, Dmitry V. Levin wrote: > > > On Wed, Jan 23, 2019 at 09:14:30PM +0000, Richard W.M. Jones wrote: > > > > On Thu, Jan 24, 2019 at 12:03:21AM +0300, Dmitry V. Levin wrote: > > > > > Hi, > > > > > > > > > > On Wed, Jan 23, 2019 at 08:45:06PM +0000, Richard W.M. Jones wrote: > > > > > > On Wed, Jan 23, 2019 at 09:01:19PM +0100, Bruno Haible wrote: > > > > > [...] > > > > > > I checked the history of the Fedora package which adds these flags, and > > > > > > it seems like --as-needed was added for: > > > > > > > > > > > > https://fedoraproject.org/wiki/Changes/RemoveExcessiveLinking > > > > > > > > > > > > Removing -Wl,--as-needed fixes the problem. However I'm still unclear > > > > > > about this. Is Fedora wrong? Is hivex using gnulib wrongly? Is > > > > > > gnulib wrong? > > > > > > > > > > We also build everything with --as-needed by default, but > > > > > gnulib does the right thing nowadays by using > > > > > -Wl,--push-state -Wl,--no-as-needed -lpthread -Wl,--pop-state > > > > > instead of plain -lpthread. > > > > > > > > hivex is using a very recent gnulib (34290cb926). > > > > > > > > The test is linked with: > > > > > > > > /bin/sh ../../libtool --tag=CC --preserve-dup-deps --mode=link gcc -O2 > > > > -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 > > > > -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong > > > > -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 > > > > -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic > > > > -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection > > > > -Wl,-z,relro -Wl,--as-needed -Wl,-z,now > > > > -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -o test-thread_create > > > > test-thread_create.o libtests.a ../../gnulib/lib/libgnu.la libtests.a > > > > -pthread -Wl,--push-state -Wl,--no-as-needed -lpthread -Wl,--pop-state > > > > > > > > As you can see from the last bit, the --push-state ... --pop-state > > > > seems correct. Yet the test still fails. > > > > > > I see libtool is involved, and libtool is known to reorder arguments > > > in a very unhelpful way. Could you show the actual link command invoked > > > by libtool, please? I expect to see something like > > > -Wl,--push-state -Wl,--no-as-needed -Wl,--pop-state ... -lpthread > > > which doesn't make sense. > > > > I've posted the entire build/check with V=1 here: > > > > http://oirase.annexia.org/tmp/2019-01-gnulib-hivex.txt > > > > If you grep through that for "test-thread_create" you should find the > > appropriate lines. > > According to the log, it was libtool that broke a valid command > > /bin/sh ../../libtool --tag=CC --preserve-dup-deps --mode=link gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -o test-rwlock1 test-rwlock1.o libtests.a ../../gnulib/lib/libgnu.la libtests.a -pthread -Wl,--push-state -Wl,--no-as-needed -lpthread -Wl,--pop-state -lrt > > into an invalid command > > libtool: link: gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wl,-z -Wl,relro -Wl,--as-needed -Wl,-z -Wl,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -o test-rwlock1 test-rwlock1.o -pthread -Wl,--push-state -Wl,--no-as-needed -Wl,--pop-state libtests.a ../../gnulib/lib/.libs/libgnu.a libtests.a -lpthread -lrt -pthread > > What's the version of libtool that's produced such a mess? libtool-2.4.6-28.fc30.x86_64 Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: test-rwlock1 failing on latest Fedora Rawhide 2019-01-23 21:03 ` Dmitry V. Levin 2019-01-23 21:14 ` Richard W.M. Jones @ 2019-01-23 21:17 ` Michael Hudson-Doyle 1 sibling, 0 replies; 14+ messages in thread From: Michael Hudson-Doyle @ 2019-01-23 21:17 UTC (permalink / raw) To: Dmitry V. Levin; +Cc: Bruno Haible, bug-gnulib [-- Attachment #1: Type: text/plain, Size: 881 bytes --] On Thu, 24 Jan 2019 at 10:03, Dmitry V. Levin <ldv@altlinux.org> wrote: > Hi, > > On Wed, Jan 23, 2019 at 08:45:06PM +0000, Richard W.M. Jones wrote: > > On Wed, Jan 23, 2019 at 09:01:19PM +0100, Bruno Haible wrote: > [...] > > I checked the history of the Fedora package which adds these flags, and > > it seems like --as-needed was added for: > > > > https://fedoraproject.org/wiki/Changes/RemoveExcessiveLinking > > > > Removing -Wl,--as-needed fixes the problem. However I'm still unclear > > about this. Is Fedora wrong? Is hivex using gnulib wrongly? Is > > gnulib wrong? > > We also build everything with --as-needed by default, but > gnulib does the right thing nowadays by using > -Wl,--push-state -Wl,--no-as-needed -lpthread -Wl,--pop-state > instead of plain -lpthread. > libtool can "helpfully" reorder those flags so they don't do anything though. Cheers, mwh [-- Attachment #2: Type: text/html, Size: 1456 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2019-01-24 9:11 UTC | newest] Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-01-23 19:37 test-rwlock1 failing on latest Fedora Rawhide Richard W.M. Jones 2019-01-23 20:01 ` Bruno Haible 2019-01-23 20:45 ` Richard W.M. Jones 2019-01-23 21:03 ` Dmitry V. Levin 2019-01-23 21:14 ` Richard W.M. Jones 2019-01-23 22:09 ` Dmitry V. Levin 2019-01-23 22:24 ` Richard W.M. Jones 2019-01-23 22:38 ` Dmitry V. Levin 2019-01-24 3:18 ` Bruno Haible 2019-01-24 3:31 ` Bruno Haible 2019-01-24 3:49 ` Bruno Haible 2019-01-24 8:57 ` Richard W.M. Jones 2019-01-24 8:50 ` Richard W.M. Jones 2019-01-23 21:17 ` Michael Hudson-Doyle
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).