Re: [PATCH v8 03/10] Remove __morecore and __default_morecore

[Guillaume Morin <guillaume@morinfr.org>]

On 14 Jul 12:31, Siddhesh Poyarekar wrote:
> FYI, there's at least one use case[1] that is adversely affected by
> __morecore removal.  I have closed it as WONTFIX citing that malloc does not
> always do the right thing with arbitrary morecore (and we don't even test
> it) anyway and it's a net win to remove them, but I mention it here too in
> the interest of a wider discussion.
> 
> Guillaume, would you like to elaborate on the use case a bit more so that we
> know exactly what we're dealing with?
> 
> Siddhesh
> 
> [1] https://sourceware.org/bugzilla/show_bug.cgi?id=20646

Hello Siddesh,

I replied on the bug report becuase I had not seen this message.

But basically, this breaks https://github.com/libhugetlbfs/libhugetlbfs/
which is old (started in 2006) and commonly used library.  The library
is plugging its own morecore() implementation to use hugetlb pages to
back the malloc heap.

This is definitely not a win for all libhugetlbfs users. We have no
equivalent solution. You're asking to find us an entire new malloc
implementation if there is one (or write one?). We have no way of keep
using glibc's malloc and hugetlb. I am not even sure there exists
an equivalent replacement.

I understand there is some security concern about malloc hooks but then
why not allow morecore() substitution with a properly
documented/supported interface? Most users already LD_PRELOAD
libhugetlbfs so that would be an easy fix.

You're claiming that it's subtly broken. I'd like to understand how and
why this can't be fixed. Afaik people have been using libhugetlbfs's
morecore() in production for 15+ years without any issue.

The only issue I am aware of is the one I reported about 5 years ago
(along with a reproducer and a patch). This problem can only be reached
if trimming is enabled in the morecore implementation (it's disabled in
libhugetlbfs for this very reason).

-- 
Guillaume Morin <guillaume@morinfr.org>