glibc 2.33.9000 version checks and in memory file name change

glibc 2.33.9000 version checks and in memory file name change

[Mark Wielaard]

Hi,

valgrind is kind of special and does various things it really shouldn't
do, but... There is one change in particular in 2.34 pre-releases that
is causing us some headaches. It took a bit of time to understand that
this innocent looking changes had an impact:

   * Previously, glibc installed its various shared objects under
   versioned file names such as libc-2.33.so.  The ABI sonames
   (e.g., libc.so.6) were provided as symbolic links.  Starting
   with glibc 2.34, the shared objects are installed under their
   ABI sonames directly, without symbolic links.  This increases
   compatibility with distribution package managers that delete
   removed files late during the package upgrade or downgrade
   process.

This has an impact on anything that uses in-process mapped file names.
For example most programs using backtraces used to report something like:

#0  0x00007fd73bdcbaea     __GI___clock_nanosleep - /usr/lib64/libc-2.33.so
#1  0x00007fd73bdd0d57 - 1 __GI___nanosleep - /usr/lib64/libc-2.33.so
#2  0x000055988e913a00 - 1 main - /usr/bin/sleep
#3  0x00007fd73bd2bb75 - 1 __libc_start_main - /usr/lib64/libc-2.33.so
#4  0x000055988e913bbe - 1 _start - /usr/bin/sleep

But now report:

#0  0x00007f070b8a2c1a     __GI___clock_nanosleep - /usr/lib64/libc.so.6
#1  0x00007f070b8a7997 - 1 __GI___nanosleep - /usr/lib64/libc.so.6
#2  0x000055657fbf0a00 - 1 main - /usr/bin/sleep
#3  0x00007f070b7f57e0 - 1 __libc_start_call_main - /usr/lib64/libc.so.6
#4  0x00007f070b7f588c - 1 __libc_start_main_impl - /usr/lib64/libc.so.6
#5  0x000055657fbf0bb5 - 1 _start - /usr/bin/sleep

Unfortunately valgrind suppressions are based on the file name reported
in the backtrace when valgrind reports an issue. For example we have
some standard suppressions like:

{
   helgrind-glibc-io-xsputn-mempcpy
   Helgrind:Race
   fun:__GI_mempcpy
   fun:_IO_*xsputn*
   obj:*/lib*/libc-2.*so*
}

That tells to suppress an Helgrind Race condition if the backtrace
originated for an in memory object file matching */lib*/libc-2.*so*.
This suppression fails now since /usr/lib64/libc.so.6 doesn't match
that pattern.

I wonder if there are other programs which might depend on the in
memory file name of glibc and might be impacted by this change.

Now we can update our default suppressions and will do so.

Unfortunately our current mechanism for doing that relies on a
configure check for the glibc version. Specifically we check whether
features.h defines __GLIBC__ __GLIBC_MINOR__ and construct the glibc
version from that. But current development versions of glibc (like
Fedora rawhides glibc-2.33.9000) still report __GLIBC_MINOR__ as 33.

Is there a way to detect this is really 2.33.9000 aka almost 2.34? Or
could you up the __GLIBC_MINOR__ already now that people are starting
to ship test versions like 2.33.9000?

Thanks,

Mark

Re: glibc 2.33.9000 version checks and in memory file name change

[Adhemerval Zanella via Libc-alpha]

On 09/07/2021 13:37, Mark Wielaard wrote:
> Hi,
> 
> valgrind is kind of special and does various things it really shouldn't
> do, but... There is one change in particular in 2.34 pre-releases that
> is causing us some headaches. It took a bit of time to understand that
> this innocent looking changes had an impact:
> 
>    * Previously, glibc installed its various shared objects under
>    versioned file names such as libc-2.33.so.  The ABI sonames
>    (e.g., libc.so.6) were provided as symbolic links.  Starting
>    with glibc 2.34, the shared objects are installed under their
>    ABI sonames directly, without symbolic links.  This increases
>    compatibility with distribution package managers that delete
>    removed files late during the package upgrade or downgrade
>    process.
> 
> This has an impact on anything that uses in-process mapped file names.
> For example most programs using backtraces used to report something like:
> 
> #0  0x00007fd73bdcbaea     __GI___clock_nanosleep - /usr/lib64/libc-2.33.so
> #1  0x00007fd73bdd0d57 - 1 __GI___nanosleep - /usr/lib64/libc-2.33.so
> #2  0x000055988e913a00 - 1 main - /usr/bin/sleep
> #3  0x00007fd73bd2bb75 - 1 __libc_start_main - /usr/lib64/libc-2.33.so
> #4  0x000055988e913bbe - 1 _start - /usr/bin/sleep
> 
> But now report:
> 
> #0  0x00007f070b8a2c1a     __GI___clock_nanosleep - /usr/lib64/libc.so.6
> #1  0x00007f070b8a7997 - 1 __GI___nanosleep - /usr/lib64/libc.so.6
> #2  0x000055657fbf0a00 - 1 main - /usr/bin/sleep
> #3  0x00007f070b7f57e0 - 1 __libc_start_call_main - /usr/lib64/libc.so.6
> #4  0x00007f070b7f588c - 1 __libc_start_main_impl - /usr/lib64/libc.so.6
> #5  0x000055657fbf0bb5 - 1 _start - /usr/bin/sleep
> 
> Unfortunately valgrind suppressions are based on the file name reported
> in the backtrace when valgrind reports an issue. For example we have
> some standard suppressions like:
> 
> {
>    helgrind-glibc-io-xsputn-mempcpy
>    Helgrind:Race
>    fun:__GI_mempcpy
>    fun:_IO_*xsputn*
>    obj:*/lib*/libc-2.*so*
> }
> 
> That tells to suppress an Helgrind Race condition if the backtrace
> originated for an in memory object file matching */lib*/libc-2.*so*.
> This suppression fails now since /usr/lib64/libc.so.6 doesn't match
> that pattern.
> 
> I wonder if there are other programs which might depend on the in
> memory file name of glibc and might be impacted by this change.

I am not sure, but relying on such naming scheme is fragile in principle:
although the installation did use the embedded the version in the 
installed filename nothing preventes the distribution to rename it to
something complete different (since the DT_NEEDED will still point to
libc.so.6).

> 
> Now we can update our default suppressions and will do so.
> 
> Unfortunately our current mechanism for doing that relies on a
> configure check for the glibc version. Specifically we check whether
> features.h defines __GLIBC__ __GLIBC_MINOR__ and construct the glibc
> version from that. But current development versions of glibc (like
> Fedora rawhides glibc-2.33.9000) still report __GLIBC_MINOR__ as 33.

What kind of regular expression does valgrind suppression mechanism
support? Does it mimic the shell expansion used on glob(), for instance,
or is it something more restrict?

Because you could use use something like "libc*(-2.[0-9]*).so*" if you
can use a glob-like with some gnu extensions.

> 
> Is there a way to detect this is really 2.33.9000 aka almost 2.34? Or
> could you up the __GLIBC_MINOR__ already now that people are starting
> to ship test versions like 2.33.9000?

We now moved all libpthread/librt/libdl symbols to libc.so, so if you
must detect 2.34 glibc you may try use pthread_create without linking
to -lpthread.  Usually we do follow the release procedure to just bump
de version to 2.34 once the release is actually done.

Re: glibc 2.33.9000 version checks and in memory file name change

[Mark Wielaard]

Hi,

On Wed, Jul 14, 2021 at 01:55:27PM -0300, Adhemerval Zanella wrote:
> On 09/07/2021 13:37, Mark Wielaard wrote:
> > That tells to suppress an Helgrind Race condition if the backtrace
> > originated for an in memory object file matching */lib*/libc-2.*so*.
> > This suppression fails now since /usr/lib64/libc.so.6 doesn't match
> > that pattern.
> > 
> > I wonder if there are other programs which might depend on the in
> > memory file name of glibc and might be impacted by this change.
> 
> I am not sure, but relying on such naming scheme is fragile in principle:
> although the installation did use the embedded the version in the 
> installed filename nothing preventes the distribution to rename it to
> something complete different (since the DT_NEEDED will still point to
> libc.so.6).

Yes, it would be nicer to have used the so name, but people are more
familiar with the actual file name. It also used to be more
architecture independent. Because of the symlinks ld.so was named the
same on all arches. I certainly understand this was just by accident
and we shouldn't have relied on it, but...

> > Now we can update our default suppressions and will do so.
> > 
> > Unfortunately our current mechanism for doing that relies on a
> > configure check for the glibc version. Specifically we check whether
> > features.h defines __GLIBC__ __GLIBC_MINOR__ and construct the glibc
> > version from that. But current development versions of glibc (like
> > Fedora rawhides glibc-2.33.9000) still report __GLIBC_MINOR__ as 33.
> 
> What kind of regular expression does valgrind suppression mechanism
> support? Does it mimic the shell expansion used on glob(), for instance,
> or is it something more restrict?

It only knows about * (zero or more) and ? (zero or one) chars.

> > Is there a way to detect this is really 2.33.9000 aka almost 2.34? Or
> > could you up the __GLIBC_MINOR__ already now that people are starting
> > to ship test versions like 2.33.9000?
> 
> We now moved all libpthread/librt/libdl symbols to libc.so, so if you
> must detect 2.34 glibc you may try use pthread_create without linking
> to -lpthread.

Thanks, that is a nice check. I used that for now.
https://bugs.kde.org/show_bug.cgi?id=439590

> Usually we do follow the release procedure to just bump
> de version to 2.34 once the release is actually done.

Understood, and it is too late to change that for this release.  But
maybe for the next features.h could be changed at the freeze point
instead of at release? Then people will test against the actual
release number.

Thanks,

Mark

Re: glibc 2.33.9000 version checks and in memory file name change

[Adhemerval Zanella via Libc-alpha]

On 19/07/2021 06:18, Mark Wielaard wrote:
> Hi,
> 
> On Wed, Jul 14, 2021 at 01:55:27PM -0300, Adhemerval Zanella wrote:
>> On 09/07/2021 13:37, Mark Wielaard wrote:
>>> That tells to suppress an Helgrind Race condition if the backtrace
>>> originated for an in memory object file matching */lib*/libc-2.*so*.
>>> This suppression fails now since /usr/lib64/libc.so.6 doesn't match
>>> that pattern.
>>>
>>> I wonder if there are other programs which might depend on the in
>>> memory file name of glibc and might be impacted by this change.
>>
>> I am not sure, but relying on such naming scheme is fragile in principle:
>> although the installation did use the embedded the version in the 
>> installed filename nothing preventes the distribution to rename it to
>> something complete different (since the DT_NEEDED will still point to
>> libc.so.6).
> 
> Yes, it would be nicer to have used the so name, but people are more
> familiar with the actual file name. It also used to be more
> architecture independent. Because of the symlinks ld.so was named the
> same on all arches. I certainly understand this was just by accident
> and we shouldn't have relied on it, but...

We really care about ABI stability and since this is accomplished by
setting the expected soname on DT_NEEEDED, I think adding another
constrain, such the installed filename, does not really add much here.
The potential breakage is minimum imho and could be mitigated by using
a more expressive filter (such as the one I suggested) if filename is
really important by the project.

> 
>>> Now we can update our default suppressions and will do so.
>>>
>>> Unfortunately our current mechanism for doing that relies on a
>>> configure check for the glibc version. Specifically we check whether
>>> features.h defines __GLIBC__ __GLIBC_MINOR__ and construct the glibc
>>> version from that. But current development versions of glibc (like
>>> Fedora rawhides glibc-2.33.9000) still report __GLIBC_MINOR__ as 33.
>>
>> What kind of regular expression does valgrind suppression mechanism
>> support? Does it mimic the shell expansion used on glob(), for instance,
>> or is it something more restrict?
> 
> It only knows about * (zero or more) and ? (zero or one) chars.
> 
>>> Is there a way to detect this is really 2.33.9000 aka almost 2.34? Or
>>> could you up the __GLIBC_MINOR__ already now that people are starting
>>> to ship test versions like 2.33.9000?
>>
>> We now moved all libpthread/librt/libdl symbols to libc.so, so if you
>> must detect 2.34 glibc you may try use pthread_create without linking
>> to -lpthread.
> 
> Thanks, that is a nice check. I used that for now.
> https://bugs.kde.org/show_bug.cgi?id=439590
> 
>> Usually we do follow the release procedure to just bump
>> de version to 2.34 once the release is actually done.
> 
> Understood, and it is too late to change that for this release.  But
> maybe for the next features.h could be changed at the freeze point
> instead of at release? Then people will test against the actual
> release number.
> 

I don't have a strong opinion, maybe we could bump it on slushy freeze
that usually happens 4 weeks prior release.  Carlos, what do you think
about changing it to the next release?