Re: Undefined use of weak symbols in gnulib

[Bruno Haible <bruno@clisp.org>]

Hi Florian,

> > So, in the normal cases (link with '-lpthread', link without '-lpthread',
> > and even with dlopen()), everything will work fine. The only problematic
> > case thus is the the use of LD_PRELOAD. Right?
> 
> LD_PRELOAD and glibc 2.34 as originally planned.
> ...
> > In other words, the problem is that
> >   - there are some/many binaries out there, that were produced by an 'ld'
> >     that did not anticipate the changes in glibc 2.34, and
> >   - these binaries have a problem not when run directly, but only when
> >     run with LD_PRELOAD.
> >
> > Right?
> 
> No, glibc 2.34 won't need LD_PRELOAD to expose the bug.  LD_PRELOAD is
> just a development aid that reveals the problem with glibc 2.33 and
> earlier.

Ouch, then it means that a significant number of binaries out there
will crash when the system's libc is upgraded to 2.34.

Many programs use "#pragma weak ..." in the way it was advertised 20 years
ago. In fact, it was advertised to be useful for backward compatibility [1] —
and now we face a problem with it, precisely regarding backward compatibility.

Here are just some occurrences of "#pragma weak pthread_*":

- Gnulib - embedded in many GNU programs. [2]
- GCC / libstdc++ - gthr-posix.[hc] (gcc-4.6.4)
- libxml2 [3]
- prelude-siem [4]

> I spent today on coming up with a workaround in glibc.

These are the workarounds I can see:
  - Delay the planned changes in glibc by 5 years or so, to minimize
    the number of binaries out there that would crash. (Probably not what
    you want.)
  - Change glibc's ld.so to deal with the binaries that are out there
    and that have been produced by existing binutils (with or without the
    patches that H.J. Lu listed).
  - Play tricks with the preprocessor, such as
    '#define pthread_create pthread_create_in_libc'. (Probably not POSIX
    compliant.)
  - Make use of symbol versioning. Symbol versioning was invented to
    allow making big changes to libc without breaking binary backward
    compatibility. (I don't know about the interplay between weak and
    versioned symbols.)

> >> No, it's unrelated.  The crash or other undefined behavior is a
> >> consequence of actions of the link editor and cannot be reverted at run
> >> time.

I fear that you will need to touch the dynamic linker. The binaries that
could crash are out in the wild.

> However, you should really remove those weak symbol
> hacks.  They won't have any effect for glibc 2.34, and as explained

Yes, I will remove them from gnulib, when a glibc >= 2.34 is found,
simply to reduce complexity on glibc systems.
But it will not help fixing the crashes when glibc 2.34 is released,
because it takes time until
  1. the gnulib changes are integrated into the packages that use gnulib,
  2. these packages have had a tarball release on ftp.gnu.org,
  3. users have replaced their old self-compiled binaries with newer ones.
I am talking about a time frame from 3 to 10 years here.

Bruno

[1] https://en.wikipedia.org/wiki/Weak_symbol#Use_cases
[2] https://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=blob;f=lib/glthread/lock.h;h=a0b24b63224cf44f862bd8e181e79b037c969d58;hb=HEAD#l242
[3] https://gitlab.gnome.org/GNOME/libxml2/-/blob/master/threads.c#L50
[4] https://www.prelude-siem.org/attachments/download/119/pthread-weak.diff