[ruby-core:86978] Re: [Ruby trunk Feature#14718] Use jemalloc by default?

[Eric Wong <normalperson@yhbt.net>]

shyouhei@ruby-lang.org wrote:
> So, I think it's clear people are interested in replacing _glibc_ malloc, not everything that exist in the whole universe.
> 
> Let's discuss how we achieve that.  There can be several ways:
> 
> - Just enable `--with-jemalloc` default on, only for Linux.
>     - pro: This is the easiest to implement.
>     - pro: Arguably works well. Already field proven.
>     - con: Mandates runtime dependency for libjemalloc on those systems.

How about only link against it by default if detected.  It will
be like our optional dependency on GMP.  We will depend on
distros to enable/disable the dependency on it.

I'm not 100% sure jemalloc is the best solution today, nor will
be in the future.  With some of my projects, glibc malloc is
often slightly smaller and faster.  But I realize typical Ruby
code is not written with low memory usage in mind :<

jemalloc itself is a LOT of code for a malloc implementation,
so the icache footprint is non-trivial and that impacts
startup time (but that is currently overshadowed by Rubygems).

> - Detect glibc on startup and try linking jemalloc then.
>     - pro: Even works on systems without jemalloc.
>     - con: Sacrifices process bootup time, which is a bad thing.
>     - con: Tricky to implement, prone to bug.

I'm not sure how that would work since malloc seems to get
called before main() by the linker.

Perhaps we can try mallopt(M_ARENA_MAX, 2) for glibc by
default; along with auto-enabling jemalloc by iff installed.

> - Bundle jemalloc and link it statically.

NAK.  The cons your list are huge and makes life difficult for
distros and code auditing.

>     - pro: No runtime hell.
>     - con: Bloats source distribution.  Costs non-glibc users.
>     - con: Also costs the core devs because they have to sync the bundled jemalloc with the upstream.