[ruby-core:87229] [Ruby trunk Feature#14718] Use jemalloc by default?

[dennisb55@hotmail.com]

Issue #14718 has been updated by bluz71 (Dennis B).


mperham (Mike Perham) wrote:
> If jemalloc 5.1.0 is using too much memory, you can tune its arenas in the same way as glibc:
> 
> ~~~
> MALLOC_CONF=narenas:2,print_stats:true
> ~~~
> 
> https://github.com/jemalloc/jemalloc/blob/dev/TUNING.md

Interesting.

So can we make the observation that low arena count equals low fragmentation but higher thread/malloc contention.

Maybe jemalloc 3.6 used one (or low) arena counts. This gives low fragmentation, but also with certain scripts (such as Yusuke's frag2.rb IO program) low performance.

It may be the case that jemalloc vs glibc performance/fragmentation will not differ that much once arena count are equalised?

Ideally I would like a new Ruby runtime flag `--long-lived` that was tuned for long run times (e.g low malloc arena count, JIT enabled) vs script/short usages where I want maximum performance.

Sidekiq, Sinatra and Rails usually want low arena counts and JIT.
Scripts want maximum immediate performance always (max arenas and no JIT).

----------------------------------------
Feature #14718: Use jemalloc by default?
https://bugs.ruby-lang.org/issues/14718#change-72219

* Author: mperham (Mike Perham)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
I know Sam opened #9113 4 years ago to suggest this but I'm revisiting the topic to see if there's any movement here for Ruby 2.6 or 2.7.  I supply a major piece of Ruby infrastructure (Sidekiq) and I keep hearing over and over how Ruby is terrible with memory, a huge memory hog with their Rails apps.  My users switch to jemalloc and a miracle occurs: their memory usage drops massively.  Some data points:

https://twitter.com/brandonhilkert/status/987400365627801601
https://twitter.com/d_jones/status/989866391787335680
https://github.com/mperham/sidekiq/issues/3824#issuecomment-383072469

Redis moved to jemalloc many years ago and it solved all of their memory issues too.  Their conclusion: the glibc allocator "sucks really really hard". http://oldblog.antirez.com/post/everything-about-redis-24.html

This is a real pain point for the entire Rails community and would improve Ruby's reputation immensely if we can solve this problem.



-- 
https://bugs.ruby-lang.org/