[Question] ifunc odering in ELF

[Question] ifunc odering in ELF

[Juchan Kim]

Hello,

My name is Juchan Kim and working for QNX.
I'm writing this to ask a question regarding IFUNC.

We get a segmentation fault and here is the scenario.

We have two ifuncs foo1 and foo2 in our library

foo1 has foo1_resolver() 
foo2 has foo2_resolver()


foo1_resolver() {
	...
}

foo2_resolver() {
	foo1()
	...
}

As you can see foo2_reoslver calls foo1() inside and this is the order
in ELF
000000010fc0  000000000408 R_AARCH64_IRELATI         ac4 // foo2()
000000010fc8  000000000408 R_AARCH64_IRELATI         aec // foo1() 

When the library is loaded in our dynamic linker, it resolves foo2()
first by running foo2_resolver(), but since foo1() is not reoslvered at
that time, we get a segfault.

We can resolve it by modifying the library, but I'm looking for ways on
ELF or ld sides.

What I mean is that...
Is there any way to re-order R_AARCH64_IRELATI by dependencies using
link scrpit or 'ld'?

or I would appreiciate if you have other suggestions

Thanks,
Juchan Kim

Re: [Question] ifunc odering in ELF

[Andrew Pinski]

On Thu, Sep 12, 2019 at 12:03 PM Juchan Kim <juckim@blackberry.com> wrote:
>
> Hello,
>
> My name is Juchan Kim and working for QNX.
> I'm writing this to ask a question regarding IFUNC.
>
> We get a segmentation fault and here is the scenario.
>
> We have two ifuncs foo1 and foo2 in our library
>
> foo1 has foo1_resolver()
> foo2 has foo2_resolver()
>
>
> foo1_resolver() {
>         ...
> }
>
> foo2_resolver() {
>         foo1()
>         ...
> }
>
> As you can see foo2_reoslver calls foo1() inside and this is the order
> in ELF
> 000000010fc0  000000000408 R_AARCH64_IRELATI         ac4 // foo2()
> 000000010fc8  000000000408 R_AARCH64_IRELATI         aec // foo1()
>
> When the library is loaded in our dynamic linker, it resolves foo2()
> first by running foo2_resolver(), but since foo1() is not reoslvered at
> that time, we get a segfault.
>
> We can resolve it by modifying the library, but I'm looking for ways on
> ELF or ld sides.
>
> What I mean is that...
> Is there any way to re-order R_AARCH64_IRELATI by dependencies using
> link scrpit or 'ld'?
>
> or I would appreiciate if you have other suggestions

What normally glibc does is lazy bindings.  But what can be done in an
ifunc resolver is usually limited anyways.  There has been discussions
about what can and cannot be done in a resolver function.  Most of
them just die out with nothing changing.  These discussion have been
about using floating point (or simd registers) in them.

Thanks,
Andrew Pinski

>
> Thanks,
> Juchan Kim

Re: [Question] ifunc odering in ELF

[Florian Weimer]

* Juchan Kim:

> We have two ifuncs foo1 and foo2 in our library
>
> foo1 has foo1_resolver() 
> foo2 has foo2_resolver()
>
>
> foo1_resolver() {
> 	...
> }
>
> foo2_resolver() {
> 	foo1()
> 	...
> }

IFUNC resolvers must not depend on run-time relocations.  If you follow
that rule, this cannot happen.

binutils ld sorts IFUNC relocations last, which helps, but to cover more
cases (but not all of them), run-time reordering of relocations is
needed.  We have some patches for that, but my impression is that the
glibc project does not want them.  Instead, we're probably going to
remove all IFUNC resolvers with relocation dependencies from glibc, and
tell others to do so as well.

Thanks,
Florian

Re: [Question] ifunc odering in ELF

[Juchan Kim]

Hi Florian,

OK. That's what I also expected. 
Thanks for the reply.

Cheers,
Juchan

On Mon, 2019-09-16 at 22:43 -0400, Florian Weimer wrote:
> * Juchan Kim:
> 
> > 
> > We have two ifuncs foo1 and foo2 in our library
> > 
> > foo1 has foo1_resolver() 
> > foo2 has foo2_resolver()
> > 
> > 
> > foo1_resolver() {
> > 	...
> > }
> > 
> > foo2_resolver() {
> > 	foo1()
> > 	...
> > }
> IFUNC resolvers must not depend on run-time relocations.  If you
> follow
> that rule, this cannot happen.
> 
> binutils ld sorts IFUNC relocations last, which helps, but to cover
> more
> cases (but not all of them), run-time reordering of relocations is
> needed.  We have some patches for that, but my impression is that the
> glibc project does not want them.  Instead, we're probably going to
> remove all IFUNC resolvers with relocation dependencies from glibc,
> and
> tell others to do so as well.
> 
> Thanks,
> Florian

Re: [Question] ifunc odering in ELF

[Adhemerval Zanella]

On 16/09/2019 23:43, Florian Weimer wrote:
> * Juchan Kim:
> 
>> We have two ifuncs foo1 and foo2 in our library
>>
>> foo1 has foo1_resolver() 
>> foo2 has foo2_resolver()
>>
>>
>> foo1_resolver() {
>> 	...
>> }
>>
>> foo2_resolver() {
>> 	foo1()
>> 	...
>> }
> 
> IFUNC resolvers must not depend on run-time relocations.  If you follow
> that rule, this cannot happen.
> 
> binutils ld sorts IFUNC relocations last, which helps, but to cover more
> cases (but not all of them), run-time reordering of relocations is
> needed.  We have some patches for that, but my impression is that the
> glibc project does not want them.  Instead, we're probably going to
> remove all IFUNC resolvers with relocation dependencies from glibc, and
> tell others to do so as well.
> 

That's was not my impression, I this we should to go this way iff we can't
really make IFUNC reordering work on loader. From your initial work on
the two-phase IFUNC relocation, which prevents us to move forward?

Re: [Question] ifunc odering in ELF

[Florian Weimer]

* Adhemerval Zanella:

> That's was not my impression, I this we should to go this way iff we can't
> really make IFUNC reordering work on loader. From your initial work on
> the two-phase IFUNC relocation, which prevents us to move forward?

No one said, “yes, we do need this feature in the loader”.

(Keep in mind that it does not solve all ordering problems, the example
in this thread would still break.)

Thanks,
Florian

Re: [Question] ifunc odering in ELF

[Szabolcs Nagy]

On 20/09/2019 16:51, Florian Weimer wrote:
> * Adhemerval Zanella:
> 
>> That's was not my impression, I this we should to go this way iff we can't
>> really make IFUNC reordering work on loader. From your initial work on
>> the two-phase IFUNC relocation, which prevents us to move forward?
> 
> No one said, “yes, we do need this feature in the loader”.
> 
> (Keep in mind that it does not solve all ordering problems, the example
> in this thread would still break.)

if most "reasonable" ordering issues can be fixed, then
i think the ordering should be fixed.

but i seem to remember that for it to work it's not enough
to consider the relocations of a single module, but other
modules have to be considered (in case of copy relocated
function pointers? or extern ifunc code in not yet relocated
modules?).

if relocations have to be processed twice across multiple
modules then i think the complexity is too much (we need to
allocate space for the list of relocs that need reprocessing
etc).

Re: [Question] ifunc odering in ELF

[Florian Weimer]

* Szabolcs Nagy:

> On 20/09/2019 16:51, Florian Weimer wrote:
>> * Adhemerval Zanella:
>> 
>>> That's was not my impression, I this we should to go this way iff we can't
>>> really make IFUNC reordering work on loader. From your initial work on
>>> the two-phase IFUNC relocation, which prevents us to move forward?
>> 
>> No one said, “yes, we do need this feature in the loader”.
>> 
>> (Keep in mind that it does not solve all ordering problems, the example
>> in this thread would still break.)
>
> if most "reasonable" ordering issues can be fixed, then
> i think the ordering should be fixed.
>
> but i seem to remember that for it to work it's not enough
> to consider the relocations of a single module,

Current binutils already orders IRELATIVE relocations after all other
relocations.  I haven't checked (or don't recall) if the sorting also
treats likely self-bindings specially (ordering them after other
relocations).  Beyond that there is little we can do in the link editor
because the problematic scenarios involve preloading, so there's just
not a complete picture and link time.

> but other modules have to be considered (in case of copy relocated
> function pointers? or extern ifunc code in not yet relocated
> modules?).

The patch I posted is global in effect:

  <https://sourceware.org/ml/libc-alpha/2018-06/msg00077.html>

> if relocations have to be processed twice across multiple
> modules then i think the complexity is too much (we need to
> allocate space for the list of relocs that need reprocessing
> etc).

Right, the patch does that.  Looks like I read the consensus on the list
correctly. 8-/

I guess I'll keep working on eliminating the remaining relocation
dependencies (and others) in glibc's IFUNC resolvers.

Thanks,
Florian