FreeBSD 11.2: glthread build failure

FreeBSD 11.2: glthread build failure

[Tim Rühsen]

[-- Attachment #1.1: Type: text/plain, Size: 826 bytes --]

Hi,

with the latest gnulib (a7903da07d3d18c23314aa0815adbb4058fd7cec) on
FreeBSD 11.2-RELEASE-p4 (with clang 6.0):

  CCLD     wget2
/usr/bin/ld: undefined reference to symbol
`pthread_mutexattr_gettype@@FBSD_1.0' (try adding -lthr)
//lib/libthr.so.3: could not read symbols: Bad value
clang: error: linker command failed with exit code 1 (use -v to see
invocation)
gmake[2]: *** [Makefile:1697: wget2] Error 1

I have no direct access to that machine or to any other FreeBSD
installation.

LIBMULTITHREAD=''
LIBPMULTITHREAD='-lpthread'
LIBPTHREAD=''
LIBSTDTHREAD=''
LIBTHREAD=''
LIB_PTHREAD_SIGMASK=''

gnulib-tool tells me to add $(LIBMULTITHREAD), $(LIBTHREAD) and
$(LIB_PTHREAD_SIGMASK) to LDADD.

config.log is at
https://gitlab.com/gnuwget/wget2/-/jobs/395869770/artifacts/file/config.log


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: FreeBSD 11.2: glthread build failure

[Tim Rühsen]

[-- Attachment #1.1: Type: text/plain, Size: 1081 bytes --]

FYI, the issue below also happens with --disable-threads.

Using --enable-threads=isoc fixes it (found in an earlier post on this ML).

Regards, Tim

On 1/7/20 4:03 PM, Tim Rühsen wrote:
> Hi,
> 
> with the latest gnulib (a7903da07d3d18c23314aa0815adbb4058fd7cec) on
> FreeBSD 11.2-RELEASE-p4 (with clang 6.0):
> 
>   CCLD     wget2
> /usr/bin/ld: undefined reference to symbol
> `pthread_mutexattr_gettype@@FBSD_1.0' (try adding -lthr)
> //lib/libthr.so.3: could not read symbols: Bad value
> clang: error: linker command failed with exit code 1 (use -v to see
> invocation)
> gmake[2]: *** [Makefile:1697: wget2] Error 1
> 
> I have no direct access to that machine or to any other FreeBSD
> installation.
> 
> LIBMULTITHREAD=''
> LIBPMULTITHREAD='-lpthread'
> LIBPTHREAD=''
> LIBSTDTHREAD=''
> LIBTHREAD=''
> LIB_PTHREAD_SIGMASK=''
> 
> gnulib-tool tells me to add $(LIBMULTITHREAD), $(LIBTHREAD) and
> $(LIB_PTHREAD_SIGMASK) to LDADD.
> 
> config.log is at
> https://gitlab.com/gnuwget/wget2/-/jobs/395869770/artifacts/file/config.log
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: FreeBSD 11.2: glthread build failure

[Bruno Haible]

Hi Tim,

> > FreeBSD 11.2-RELEASE-p4 (with clang 6.0):
> > 
> >   CCLD     wget2

Please make it a habit to use "make V=1" when submitting a report.
I don't want to see "CCLD wget2"; I want to see the actual link
command line.

> > /usr/bin/ld: undefined reference to symbol
> > `pthread_mutexattr_gettype@@FBSD_1.0' (try adding -lthr)
> > //lib/libthr.so.3: could not read symbols: Bad value
> > clang: error: linker command failed with exit code 1 (use -v to see
> > invocation)
> > gmake[2]: *** [Makefile:1697: wget2] Error 1

I reproduce the issue
  - with a wget2-1.99.2.tar.gz tarball downloaded from
    https://gitlab.com/gnuwget/wget2/pipelines
  - when configuring with --disable-shared
  - on FreeBSD 11, NOT on FreeBSD 12,
  - NOT when configuring with --disable-shared --without-lzma

The problem is that, somehow, weak symbols don't work well when linking
with a library that has a dependency to libpthread.so (-> libthr.so)
- liblzma.so in this case.

Since gnulib cannot forbid any package from linking with liblzma, gnulib
has to avoid using weak symbols in this case.

This patch should fix the problem.


2020-01-19  Bruno Haible  <bruno@clisp.org>

	threadlib: Disable use of weak symbols on FreeBSD 11.
	Reported by Tim Rühsen <tim.ruehsen@gmx.de> in
	<https://lists.gnu.org/archive/html/bug-gnulib/2020-01/msg00061.html>.
	* m4/threadlib.m4 (gl_WEAK_SYMBOLS): Require AC_CANONICAL_HOST. Test
	against a bug in FreeBSD 11.

diff --git a/m4/threadlib.m4 b/m4/threadlib.m4
index e64ad62..94441f6 100644
--- a/m4/threadlib.m4
+++ b/m4/threadlib.m4
@@ -1,4 +1,4 @@
-# threadlib.m4 serial 26
+# threadlib.m4 serial 27
 dnl Copyright (C) 2005-2020 Free Software Foundation, Inc.
 dnl This file is free software; the Free Software Foundation
 dnl gives unlimited permission to copy and/or distribute it,
@@ -81,6 +81,7 @@ dnl Checks whether the compiler and linker support weak declarations of symbols.
 
 AC_DEFUN([gl_WEAK_SYMBOLS],
 [
+  AC_REQUIRE([AC_CANONICAL_HOST])
   AC_CACHE_CHECK([whether imported symbols can be declared weak],
     [gl_cv_have_weak],
     [gl_cv_have_weak=no
@@ -119,6 +120,30 @@ int main ()
      case " $LDFLAGS " in
        *" -static "*) gl_cv_have_weak=no ;;
      esac
+     dnl Test for a bug in FreeBSD 11: A link error occurs when using a weak
+     dnl symbol and linking against a shared library that has a dependency on
+     dnl the shared library that defines the symbol.
+     case "$gl_cv_have_weak" in
+       *yes)
+         case "$host_os" in
+           freebsd* | dragonfly*)
+             : > conftest1.c
+             $CC $CPPFLAGS $CFLAGS $LDFLAGS -fPIC -shared -o libempty.so conftest1.c -lpthread >&AS_MESSAGE_LOG_FD 2>&1
+             cat <<EOF > conftest2.c
+#include <pthread.h>
+#pragma weak pthread_mutexattr_gettype
+int main ()
+{
+  return (pthread_mutexattr_gettype != NULL);
+}
+EOF
+             $CC $CPPFLAGS $CFLAGS $LDFLAGS -o conftest conftest2.c libempty.so >&AS_MESSAGE_LOG_FD 2>&1 \
+               || gl_cv_have_weak=no
+             rm -f conftest1.c libempty.so conftest2.c conftest
+             ;;
+         esac
+         ;;
+     esac
     ])
   case "$gl_cv_have_weak" in
     *yes)

Re: FreeBSD 11.2: glthread build failure

[Tim Rühsen]

[-- Attachment #1.1: Type: text/plain, Size: 780 bytes --]

Hi Bruno,

thanks for your work !

I can confirm that your patch fixes the issue here.

On 1/20/20 4:01 AM, Bruno Haible wrote:
>>> FreeBSD 11.2-RELEASE-p4 (with clang 6.0):
>>>
>>>   CCLD     wget2
> 
> Please make it a habit to use "make V=1" when submitting a report.
> I don't want to see "CCLD wget2"; I want to see the actual link
> command line.

I don't have direct console/shell access - the issue is/was on a FreeBSD
instance running somewhere that we use for testing via Gitlab CI.
Despite from just having the output from that CI runner, it wasn't
obvious in that moment that 'make V=1' would be helpful.

Please let me know if something is missing in a report before you waste
time and i try to get that piece of info somehow.

Regards, Tim


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: FreeBSD 11.2: glthread build failure

[Bruno Haible]

> I can confirm that your patch fixes the issue here.

Great. Thanks.

Testing showed a regression: glthread_once is no longer working on FreeBSD 11,
because in this code

# define glthread_once(ONCE_CONTROL, INITFUNCTION) \
    (pthread_in_use ()                                                         \
     ? pthread_once (ONCE_CONTROL, INITFUNCTION)                               \
     : (glthread_once_singlethreaded (ONCE_CONTROL) ? (INITFUNCTION (), 0) : 0))

pthread_in_use () now evaluates to 1, and pthread_once from libc returns ENOSYS.
Thus glthread_once returns ENOSYS, and gl_once calls abort().

This patch fixes it.


2020-01-20  Bruno Haible  <bruno@clisp.org>

	lock: Fix test-once1 failure on FreeBSD 11 (regression from 2020-01-19).
	* lib/glthread/lock.c (glthread_once_multithreaded): New function.
	* lib/glthread/lock.h (glthread_once_multithreaded): New declaration.
	(glthread_once): Use it.

diff --git a/lib/glthread/lock.c b/lib/glthread/lock.c
index 6336591..1f6f713 100644
--- a/lib/glthread/lock.c
+++ b/lib/glthread/lock.c
@@ -718,6 +718,26 @@ glthread_once_singlethreaded (pthread_once_t *once_control)
     return 0;
 }
 
+# if !(PTHREAD_IN_USE_DETECTION_HARD || USE_POSIX_THREADS_WEAK)
+
+int
+glthread_once_multithreaded (pthread_once_t *once_control,
+                             void (*init_function) (void))
+{
+  int err = pthread_once (once_control, init_function);
+  if (err == ENOSYS)
+    {
+      /* This happens on FreeBSD 11: The pthread_once function in libc returns
+         ENOSYS.  */
+      if (glthread_once_singlethreaded (once_control))
+        init_function ();
+      return 0;
+    }
+  return err;
+}
+
+# endif
+
 #endif
 
 /* ========================================================================= */
diff --git a/lib/glthread/lock.h b/lib/glthread/lock.h
index 9c70f4e..1de86fe 100644
--- a/lib/glthread/lock.h
+++ b/lib/glthread/lock.h
@@ -505,10 +505,19 @@ extern int glthread_recursive_lock_destroy_multithreaded (gl_recursive_lock_t *l
 typedef pthread_once_t gl_once_t;
 # define gl_once_define(STORAGECLASS, NAME) \
     STORAGECLASS pthread_once_t NAME = PTHREAD_ONCE_INIT;
-# define glthread_once(ONCE_CONTROL, INITFUNCTION) \
-    (pthread_in_use ()                                                         \
-     ? pthread_once (ONCE_CONTROL, INITFUNCTION)                               \
-     : (glthread_once_singlethreaded (ONCE_CONTROL) ? (INITFUNCTION (), 0) : 0))
+# if PTHREAD_IN_USE_DETECTION_HARD || USE_POSIX_THREADS_WEAK
+#  define glthread_once(ONCE_CONTROL, INITFUNCTION) \
+     (pthread_in_use ()                                                        \
+      ? pthread_once (ONCE_CONTROL, INITFUNCTION)                              \
+      : (glthread_once_singlethreaded (ONCE_CONTROL) ? (INITFUNCTION (), 0) : 0))
+# else
+#  define glthread_once(ONCE_CONTROL, INITFUNCTION) \
+     (pthread_in_use ()                                                        \
+      ? glthread_once_multithreaded (ONCE_CONTROL, INITFUNCTION)               \
+      : (glthread_once_singlethreaded (ONCE_CONTROL) ? (INITFUNCTION (), 0) : 0))
+extern int glthread_once_multithreaded (pthread_once_t *once_control,
+                                        void (*init_function) (void));
+# endif
 extern int glthread_once_singlethreaded (pthread_once_t *once_control);
 
 # ifdef __cplusplus

Re: FreeBSD 11.2: glthread build failure

[Bruno Haible]

Hi Tim,

> >>>   CCLD     wget2
> > 
> > Please make it a habit to use "make V=1" when submitting a report.
> > I don't want to see "CCLD wget2"; I want to see the actual link
> > command line.
> 
> I don't have direct console/shell access - the issue is/was on a FreeBSD
> instance running somewhere that we use for testing via Gitlab CI.

You could, instead of running
   make
run
   make || { echo 'Retrying with verbosity'; make V=1; exit 1; }
This way, the overall log is compact, but it is verbose at the place
where we need it.

> it wasn't obvious in that moment that 'make V=1' would be helpful.

The best practice for bug reporters has always been - for years:
"Show the command that was executed, and show the results that it produced."

Bruno

Re: FreeBSD 11.2: glthread build failure

[Tim Rühsen]

[-- Attachment #1.1: Type: text/plain, Size: 640 bytes --]

On 1/20/20 7:04 PM, Bruno Haible wrote:
>>>>>   CCLD     wget2
>>>
>>> Please make it a habit to use "make V=1" when submitting a report.
>>> I don't want to see "CCLD wget2"; I want to see the actual link
>>> command line.
>>
>> I don't have direct console/shell access - the issue is/was on a FreeBSD
>> instance running somewhere that we use for testing via Gitlab CI.
> 
> You could, instead of running
>    make
> run
>    make || { echo 'Retrying with verbosity'; make V=1; exit 1; }
> This way, the overall log is compact, but it is verbose at the place
> where we need it.

Simple and brilliant !

Thanks, Tim


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]