Re: [PATCH] Hurd: Enable ifunc by default

Re: [PATCH] Hurd: Enable ifunc by default

[Joseph Myers]

On Wed, 13 Jan 2021, Thomas Schwinge wrote:

> Hi!
> 
> Thanks (and sorry for the delay), pushed "Hurd: Enable ifunc by default"
> to master branch in commit e9cb89b936f831a02318d45fc4ddb06f7be55ae4, and
> cherry-picked into releases/gcc-10 branch in commit
> 92b131491c22eb4e4b663d226e9d97f1fd693063, releases/gcc-9 branch in commit
> 0313ce139f4ca3c96db9dc82125ec9e4a167a224, releases/gcc-8 branch in commit
> 975b0fa0f43e84bed3cb1b2b593132bc219f962c, see attached.

I'm not sure what toolchain component the underlying bug is in, but this 
GCC commit (verified in the releases/gcc-10 case) results in a glibc build 
failure for i686-gnu with build-many-glibcs.py.

https://sourceware.org/pipermail/libc-testresults/2021q1/007378.html

The error is:

/scratch/jmyers/glibc-bot/install/compilers/i686-gnu/lib/gcc/i686-glibc-gnu/11.0.0/../../../../i686-glibc-gnu/bin/ld: /scratch/jmyers/glibc-bot/build/compilers/i686-gnu/glibc/i686-gnu/elf/librtld.os: in function `hurd_file_name_lookup_retry':
(.text+0x1e08e): undefined reference to `strncpy'
collect2: error: ld returned 1 exit status
Makefile:584: recipe for target '/scratch/jmyers/glibc-bot/build/compilers/i686-gnu/glibc/i686-gnu/elf/ld.so' failed
make[3]: *** [/scratch/jmyers/glibc-bot/build/compilers/i686-gnu/glibc/i686-gnu/elf/ld.so] Error 1

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [PATCH] Hurd: Enable ifunc by default

[Samuel Thibault via Libc-alpha]

Hello,

Joseph Myers, le lun. 18 janv. 2021 20:05:44 +0000, a ecrit:
> On Wed, 13 Jan 2021, Thomas Schwinge wrote:
> > Thanks (and sorry for the delay), pushed "Hurd: Enable ifunc by default"
> > to master branch in commit e9cb89b936f831a02318d45fc4ddb06f7be55ae4, and
> > cherry-picked into releases/gcc-10 branch in commit
> > 92b131491c22eb4e4b663d226e9d97f1fd693063, releases/gcc-9 branch in commit
> > 0313ce139f4ca3c96db9dc82125ec9e4a167a224, releases/gcc-8 branch in commit
> > 975b0fa0f43e84bed3cb1b2b593132bc219f962c, see attached.
> 
> I'm not sure what toolchain component the underlying bug is in, but this 
> GCC commit (verified in the releases/gcc-10 case) results in a glibc build 
> failure for i686-gnu with build-many-glibcs.py.
> 
> https://sourceware.org/pipermail/libc-testresults/2021q1/007378.html
> 
> The error is:
> 
> /scratch/jmyers/glibc-bot/install/compilers/i686-gnu/lib/gcc/i686-glibc-gnu/11.0.0/../../../../i686-glibc-gnu/bin/ld: /scratch/jmyers/glibc-bot/build/compilers/i686-gnu/glibc/i686-gnu/elf/librtld.os: in function `hurd_file_name_lookup_retry':
> (.text+0x1e08e): undefined reference to `strncpy'

Ah, I believe I had tested that (which is precisely why I asked for that
commit to be done in gcc), but I'll have a look, thanks.

Samuel

Re: [PATCH] Hurd: Enable ifunc by default

[Samuel Thibault via Libc-alpha]

[-- Attachment #1: Type: text/plain, Size: 1445 bytes --]

(leaving gcc out, it's really out of the story, it just happens to
expose support for ifunc)

Joseph Myers, le lun. 18 janv. 2021 20:05:44 +0000, a ecrit:
> /scratch/jmyers/glibc-bot/install/compilers/i686-gnu/lib/gcc/i686-glibc-gnu/11.0.0/../../../../i686-glibc-gnu/bin/ld: /scratch/jmyers/glibc-bot/build/compilers/i686-gnu/glibc/i686-gnu/elf/librtld.os: in function `hurd_file_name_lookup_retry':
> (.text+0x1e08e): undefined reference to `strncpy'

The story seems complex and related to the glibc rtld build rules.

- We need hurd/lookup-retry.c in the rtld
- It happens to use strncpy since ee11682d4f5 ("hurd: Fix strcpy calls")
- strncpy happens to have a hidden def in libc
- hurd/lookup-retry.os thus refers to __GI_strncpy rather than strncpy
- dl-allobjs.os thus includes strncpy-c.os (that provides __GI_strncpy),
and not strncpy.os (that provides strncpy)
- librtld.map thus contains strncpy-c.os only, and not strncpy.os
- thus no rtld-strncpy.os is getting built
- thus strncpy is not available in rtld
- but strncpy doesn't have a hidden def in rtld
- and thus hurd/rtld-lookup-retry.os refers to strncpy, not __GI_strncpy
- thus the link failure.

The base issue I see here is that dl-allobjs is based on libc-built
objects that might be using hidden defs while rtld-built objects might
not.

But we could also say that it's strncpy that should also have a hidden
def in rtld. The attached patch does this, is this ok?

Samuel

[-- Attachment #2: patch --]
[-- Type: text/plain, Size: 946 bytes --]

diff --git a/include/libc-symbols.h b/include/libc-symbols.h
index ea126ae70c..051d6b9492 100644
--- a/include/libc-symbols.h
+++ b/include/libc-symbols.h
@@ -856,6 +856,8 @@ for linking")
 # define HIDDEN_BUILTIN_JUMPTARGET(name) HIDDEN_JUMPTARGET(name)
 #endif
 
+#define rtld_hidden_builtin_proto(name, attrs...) rtld_hidden_proto (name, ##attrs)
+
 #if IS_IN (libutil)
 # define libutil_hidden_proto(name, attrs...) hidden_proto (name, ##attrs)
 # define libutil_hidden_tls_proto(name, attrs...) \
diff --git a/include/string.h b/include/string.h
index 81dab39891..27cbde47ef 100644
--- a/include/string.h
+++ b/include/string.h
@@ -144,6 +144,7 @@ libc_hidden_builtin_proto (strcspn)
 libc_hidden_builtin_proto (strlen)
 libc_hidden_builtin_proto (strncmp)
 libc_hidden_builtin_proto (strncpy)
+rtld_hidden_builtin_proto (strncpy)
 libc_hidden_builtin_proto (strpbrk)
 libc_hidden_builtin_proto (stpcpy)
 libc_hidden_builtin_proto (strrchr)

Re: [PATCH] Hurd: Enable ifunc by default

[H.J. Lu via Libc-alpha]

On Mon, Jan 18, 2021 at 4:04 PM Samuel Thibault via Libc-alpha
<libc-alpha@sourceware.org> wrote:
>
> (leaving gcc out, it's really out of the story, it just happens to
> expose support for ifunc)
>
> Joseph Myers, le lun. 18 janv. 2021 20:05:44 +0000, a ecrit:
> > /scratch/jmyers/glibc-bot/install/compilers/i686-gnu/lib/gcc/i686-glibc-gnu/11.0.0/../../../../i686-glibc-gnu/bin/ld: /scratch/jmyers/glibc-bot/build/compilers/i686-gnu/glibc/i686-gnu/elf/librtld.os: in function `hurd_file_name_lookup_retry':
> > (.text+0x1e08e): undefined reference to `strncpy'
>
> The story seems complex and related to the glibc rtld build rules.
>
> - We need hurd/lookup-retry.c in the rtld
> - It happens to use strncpy since ee11682d4f5 ("hurd: Fix strcpy calls")
> - strncpy happens to have a hidden def in libc
> - hurd/lookup-retry.os thus refers to __GI_strncpy rather than strncpy
> - dl-allobjs.os thus includes strncpy-c.os (that provides __GI_strncpy),
> and not strncpy.os (that provides strncpy)
> - librtld.map thus contains strncpy-c.os only, and not strncpy.os
> - thus no rtld-strncpy.os is getting built
> - thus strncpy is not available in rtld
> - but strncpy doesn't have a hidden def in rtld
> - and thus hurd/rtld-lookup-retry.os refers to strncpy, not __GI_strncpy
> - thus the link failure.
>
> The base issue I see here is that dl-allobjs is based on libc-built
> objects that might be using hidden defs while rtld-built objects might
> not.
>
> But we could also say that it's strncpy that should also have a hidden
> def in rtld. The attached patch does this, is this ok?
>

String function usage in ld.so must be carefully managed.  Why doesn't
hurd ld.so build script mark strncpy as needed?


-- 
H.J.

Re: [PATCH] Hurd: Enable ifunc by default

[Samuel Thibault via Libc-alpha]

H.J. Lu, le lun. 18 janv. 2021 16:17:16 -0800, a ecrit:
> String function usage in ld.so must be carefully managed.

Sure.

> Why doesn't hurd ld.so build script mark strncpy as needed?

I don't even know what "hurd ld.so build script" is. AFAIK there is not
much hurd-specific there except including the RPC stubs
($(objpfx)librtld.map: $(rpcuserlibs:.so=_pic.a)).

Samuel

Re: [PATCH] Hurd: Enable ifunc by default

[Samuel Thibault via Libc-alpha]

Samuel Thibault, le mar. 19 janv. 2021 01:21:05 +0100, a ecrit:
> H.J. Lu, le lun. 18 janv. 2021 16:17:16 -0800, a ecrit:
> > Why doesn't hurd ld.so build script mark strncpy as needed?
> 
> I don't even know what "hurd ld.so build script" is.

More precisely, I don't see why hurd should be doing anything special
while everything is already taken care of by elf/Makefile's librtld.map
file scripts.

Samuel

Re: [PATCH] Hurd: Enable ifunc by default

[H.J. Lu via Libc-alpha]

On Mon, Jan 18, 2021 at 4:24 PM Samuel Thibault <samuel.thibault@gnu.org> wrote:
>
> Samuel Thibault, le mar. 19 janv. 2021 01:21:05 +0100, a ecrit:
> > H.J. Lu, le lun. 18 janv. 2021 16:17:16 -0800, a ecrit:
> > > Why doesn't hurd ld.so build script mark strncpy as needed?
> >
> > I don't even know what "hurd ld.so build script" is.
>
> More precisely, I don't see why hurd should be doing anything special
> while everything is already taken care of by elf/Makefile's librtld.map
> file scripts.
>

I am taking a look.


-- 
H.J.

[PATCH] Hurd: Add rtld-strncpy-c.c

[H.J. Lu via Libc-alpha]

[-- Attachment #1: Type: text/plain, Size: 689 bytes --]

On Mon, Jan 18, 2021 at 4:40 PM H.J. Lu <hjl.tools@gmail.com> wrote:
>
> On Mon, Jan 18, 2021 at 4:24 PM Samuel Thibault <samuel.thibault@gnu.org> wrote:
> >
> > Samuel Thibault, le mar. 19 janv. 2021 01:21:05 +0100, a ecrit:
> > > H.J. Lu, le lun. 18 janv. 2021 16:17:16 -0800, a ecrit:
> > > > Why doesn't hurd ld.so build script mark strncpy as needed?
> > >
> > > I don't even know what "hurd ld.so build script" is.
> >
> > More precisely, I don't see why hurd should be doing anything special
> > while everything is already taken care of by elf/Makefile's librtld.map
> > file scripts.
> >
>
> I am taking a look.
>

Try this.   I have no idea if it really works on hurd.

-- 
H.J.

[-- Attachment #2: 0001-Hurd-Add-rtld-strncpy-c.c.patch --]
[-- Type: text/x-patch, Size: 789 bytes --]

From a73010aaab13c5cd67ded77091601b8cd01e3fcf Mon Sep 17 00:00:00 2001
From: "H.J. Lu" <hjl.tools@gmail.com>
Date: Mon, 18 Jan 2021 18:19:16 -0800
Subject: [PATCH] Hurd: Add rtld-strncpy-c.c

All IFUNC functions which are used in ld.so must have a rtld version if
the IFUNC version isn't safe to use in ld.so.
---
 sysdeps/mach/hurd/i386/i686/multiarch/rtld-strncpy-c.c | 1 +
 1 file changed, 1 insertion(+)
 create mode 100644 sysdeps/mach/hurd/i386/i686/multiarch/rtld-strncpy-c.c

diff --git a/sysdeps/mach/hurd/i386/i686/multiarch/rtld-strncpy-c.c b/sysdeps/mach/hurd/i386/i686/multiarch/rtld-strncpy-c.c
new file mode 100644
index 0000000000..b759c999eb
--- /dev/null
+++ b/sysdeps/mach/hurd/i386/i686/multiarch/rtld-strncpy-c.c
@@ -0,0 +1 @@
+#include <string/strncpy.c>
-- 
2.29.2

Re: [PATCH] Hurd: Add rtld-strncpy-c.c

[Samuel Thibault via Libc-alpha]

H.J. Lu via Libc-alpha, le lun. 18 janv. 2021 18:24:48 -0800, a ecrit:
> On Mon, Jan 18, 2021 at 4:40 PM H.J. Lu <hjl.tools@gmail.com> wrote:
> >
> > On Mon, Jan 18, 2021 at 4:24 PM Samuel Thibault <samuel.thibault@gnu.org> wrote:
> > >
> > > Samuel Thibault, le mar. 19 janv. 2021 01:21:05 +0100, a ecrit:
> > > > H.J. Lu, le lun. 18 janv. 2021 16:17:16 -0800, a ecrit:
> > > > > Why doesn't hurd ld.so build script mark strncpy as needed?
> > > >
> > > > I don't even know what "hurd ld.so build script" is.
> > >
> > > More precisely, I don't see why hurd should be doing anything special
> > > while everything is already taken care of by elf/Makefile's librtld.map
> > > file scripts.
> >
> > I am taking a look.
> 
> Try this.   I have no idea if it really works on hurd.

It is working indeed.

Samuel

Re: [PATCH] Hurd: Add rtld-strncpy-c.c

[H.J. Lu via Libc-alpha]

On Tue, Jan 19, 2021 at 1:02 AM Samuel Thibault <samuel.thibault@gnu.org> wrote:
>
> H.J. Lu via Libc-alpha, le lun. 18 janv. 2021 18:24:48 -0800, a ecrit:
> > On Mon, Jan 18, 2021 at 4:40 PM H.J. Lu <hjl.tools@gmail.com> wrote:
> > >
> > > On Mon, Jan 18, 2021 at 4:24 PM Samuel Thibault <samuel.thibault@gnu.org> wrote:
> > > >
> > > > Samuel Thibault, le mar. 19 janv. 2021 01:21:05 +0100, a ecrit:
> > > > > H.J. Lu, le lun. 18 janv. 2021 16:17:16 -0800, a ecrit:
> > > > > > Why doesn't hurd ld.so build script mark strncpy as needed?
> > > > >
> > > > > I don't even know what "hurd ld.so build script" is.
> > > >
> > > > More precisely, I don't see why hurd should be doing anything special
> > > > while everything is already taken care of by elf/Makefile's librtld.map
> > > > file scripts.
> > >
> > > I am taking a look.
> >
> > Try this.   I have no idea if it really works on hurd.
>
> It is working indeed.
>
> Samuel

Should I check it in?

-- 
H.J.

Re: [PATCH] Hurd: Add rtld-strncpy-c.c

[Adhemerval Zanella via Libc-alpha]

On 19/01/2021 09:21, H.J. Lu via Libc-alpha wrote:
> On Tue, Jan 19, 2021 at 1:02 AM Samuel Thibault <samuel.thibault@gnu.org> wrote:
>>
>> H.J. Lu via Libc-alpha, le lun. 18 janv. 2021 18:24:48 -0800, a ecrit:
>>> On Mon, Jan 18, 2021 at 4:40 PM H.J. Lu <hjl.tools@gmail.com> wrote:
>>>>
>>>> On Mon, Jan 18, 2021 at 4:24 PM Samuel Thibault <samuel.thibault@gnu.org> wrote:
>>>>>
>>>>> Samuel Thibault, le mar. 19 janv. 2021 01:21:05 +0100, a ecrit:
>>>>>> H.J. Lu, le lun. 18 janv. 2021 16:17:16 -0800, a ecrit:
>>>>>>> Why doesn't hurd ld.so build script mark strncpy as needed?
>>>>>>
>>>>>> I don't even know what "hurd ld.so build script" is.
>>>>>
>>>>> More precisely, I don't see why hurd should be doing anything special
>>>>> while everything is already taken care of by elf/Makefile's librtld.map
>>>>> file scripts.
>>>>
>>>> I am taking a look.
>>>
>>> Try this.   I have no idea if it really works on hurd.
>>
>> It is working indeed.
>>
>> Samuel
> 
> Should I check it in?
> 
> 
Yes please, the loader should not use ifunc on its own string usage and
it is a build issue.

Re: [PATCH] Hurd: Add rtld-strncpy-c.c

[H.J. Lu via Libc-alpha]

On Tue, Jan 19, 2021 at 4:26 AM Adhemerval Zanella
<adhemerval.zanella@linaro.org> wrote:
>
>
>
> On 19/01/2021 09:21, H.J. Lu via Libc-alpha wrote:
> > On Tue, Jan 19, 2021 at 1:02 AM Samuel Thibault <samuel.thibault@gnu.org> wrote:
> >>
> >> H.J. Lu via Libc-alpha, le lun. 18 janv. 2021 18:24:48 -0800, a ecrit:
> >>> On Mon, Jan 18, 2021 at 4:40 PM H.J. Lu <hjl.tools@gmail.com> wrote:
> >>>>
> >>>> On Mon, Jan 18, 2021 at 4:24 PM Samuel Thibault <samuel.thibault@gnu.org> wrote:
> >>>>>
> >>>>> Samuel Thibault, le mar. 19 janv. 2021 01:21:05 +0100, a ecrit:
> >>>>>> H.J. Lu, le lun. 18 janv. 2021 16:17:16 -0800, a ecrit:
> >>>>>>> Why doesn't hurd ld.so build script mark strncpy as needed?
> >>>>>>
> >>>>>> I don't even know what "hurd ld.so build script" is.
> >>>>>
> >>>>> More precisely, I don't see why hurd should be doing anything special
> >>>>> while everything is already taken care of by elf/Makefile's librtld.map
> >>>>> file scripts.
> >>>>
> >>>> I am taking a look.
> >>>
> >>> Try this.   I have no idea if it really works on hurd.
> >>
> >> It is working indeed.
> >>
> >> Samuel
> >
> > Should I check it in?
> >
> >
> Yes please, the loader should not use ifunc on its own string usage and
> it is a build issue.

Done.

-- 
H.J.

Re: [PATCH] Hurd: Add rtld-strncpy-c.c

[Samuel Thibault via Libc-alpha]

H.J. Lu via Libc-alpha, le mar. 19 janv. 2021 04:35:09 -0800, a ecrit:
> On Tue, Jan 19, 2021 at 4:26 AM Adhemerval Zanella
> <adhemerval.zanella@linaro.org> wrote:
> > On 19/01/2021 09:21, H.J. Lu via Libc-alpha wrote:
> > > On Tue, Jan 19, 2021 at 1:02 AM Samuel Thibault <samuel.thibault@gnu.org> wrote:
> > >> H.J. Lu via Libc-alpha, le lun. 18 janv. 2021 18:24:48 -0800, a ecrit:
> > >>> On Mon, Jan 18, 2021 at 4:40 PM H.J. Lu <hjl.tools@gmail.com> wrote:
> > >>>> On Mon, Jan 18, 2021 at 4:24 PM Samuel Thibault <samuel.thibault@gnu.org> wrote:
> > >>>>> More precisely, I don't see why hurd should be doing anything special
> > >>>>> while everything is already taken care of by elf/Makefile's librtld.map
> > >>>>> file scripts.
> > >>>>
> > >>>> I am taking a look.
> > >>>
> > >>> Try this.   I have no idea if it really works on hurd.
> > >>
> > >> It is working indeed.
> > >
> > > Should I check it in?
> > >
> >
> > Yes please, the loader should not use ifunc on its own string usage and
> > it is a build issue.
> 
> Done.

Thanks!