[ruby-core:92027] [Ruby trunk Feature#15667] Introduce malloc_trim(0) in full gc cycles

[sam.saffron@gmail.com]

Issue #15667 has been updated by sam.saffron (Sam Saffron).

File Screenshot_2019-03-28 Grafana - Compare Discourse Perf.png added

@mame / @carlos attached is a screenshot of side by side testing on live traffic patterns

containers run multiple ruby processes (a few unicorn workers and a sidekiq worker)

standard14 = Ruby 2.6.2 + jemalloc
standard14_a = Ruby 2.6.2 + glibc malloc
standard14_b = Ruby 2.6.2 + glibc malloc + patch

My conclusions from these graphs:

- Memory is clearly down with the patch
- 99th percentile performance is slightly impacted
- cpu is very slightly higher
- jemalloc still fairs better than glibc even after the patch


I think I would support a slightly amended patch that only does the trim once every say 10 minutes (maybe even in a background thread), happy to test that out as well. 

That said... selfishly for Discourse this does not matter that much we will still stick with jemalloc cause memory is better and performance is better under jemalloc. 

For the wider ruby community though a safer default is very appealing. 

----------------------------------------
Feature #15667: Introduce malloc_trim(0) in full gc cycles
https://bugs.ruby-lang.org/issues/15667#change-77352

* Author: sam.saffron (Sam Saffron)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
Per Hongli's excellent article it looks like malloc_trim can help tremendously with memory bloat issues. 


https://www.joyfulbikeshedding.com/blog/2019-03-14-what-causes-ruby-memory-bloat.html#a-magic-trick-trimming


I would like to get this patch tested side-by-side at Discourse, GitHub and Shopify. If it looks good I think this is both a great candidate for 2.7 and and 2.4,2.5,2.6 backports. 


Will coordinate with Shopify and GitHub to see if we can get numbers posted here, I will run tests on a live Discourse instance over the next week and report numbers here. 

Koichi, what are your thoughts, to me this looks like an incredibly safe patch, the amount of work added to major GCs is tiny compared to the potential benefit, walking all pages is a very cheap operation.



---Files--------------------------------
ruby_gc_malloc_trim.patch (1011 Bytes)
Screenshot_2019-03-28 Grafana - Compare Discourse Perf.png (530 KB)


-- 
https://bugs.ruby-lang.org/