From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-2.7 required=3.0 tests=BAYES_00,FORGED_HOTMAIL_RCVD2, FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_PASS shortcircuit=no autolearn=no autolearn_force=no version=3.4.0 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 871D91F51C for ; Wed, 23 May 2018 00:38:56 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id DE13012091D; Wed, 23 May 2018 09:38:54 +0900 (JST) Received: from o1678916x28.outbound-mail.sendgrid.net (o1678916x28.outbound-mail.sendgrid.net [167.89.16.28]) by neon.ruby-lang.org (Postfix) with ESMTPS id D229C12090C for ; Wed, 23 May 2018 09:38:52 +0900 (JST) Received: by filter0043p3las1.sendgrid.net with SMTP id filter0043p3las1-5903-5B04B81A-15 2018-05-23 00:38:50.341428458 +0000 UTC Received: from herokuapp.com (ec2-54-227-104-222.compute-1.amazonaws.com [54.227.104.222]) by ismtpd0004p1iad1.sendgrid.net (SG) with ESMTP id _zY670xVRSmG0v1Byv2g7g Wed, 23 May 2018 00:38:50.245 +0000 (UTC) Date: Wed, 23 May 2018 00:38:50 +0000 (UTC) From: dennisb55@hotmail.com To: ruby-core@ruby-lang.org Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 62594 X-Redmine-Project: ruby-trunk X-Redmine-Issue-Id: 14718 X-Redmine-Issue-Author: mperham X-Redmine-Sender: bluz71 X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: ync6xU2WACa70kv/Ymy4QrNMhiuLXJG8OTL2vJD1yS4Wv33xnvxII6TUgrrz5BNwZOl9niVd11Pwt9 Y7MDV9e9NChO6YowpiIZdG+/HMTnSeRGIy9IAhyRK5s1uHzSKqOHiStqm+vjYAF4jWDvnspOqp2sOI N6IWpnJv6m7i6HUxWiAysrYLo5oqI601bUWF7flzYhgk3jNYnaRLCQ63JA== X-ML-Name: ruby-core X-Mail-Count: 87229 Subject: [ruby-core:87229] [Ruby trunk Feature#14718] Use jemalloc by default? X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #14718 has been updated by bluz71 (Dennis B). mperham (Mike Perham) wrote: > If jemalloc 5.1.0 is using too much memory, you can tune its arenas in the same way as glibc: > > ~~~ > MALLOC_CONF=narenas:2,print_stats:true > ~~~ > > https://github.com/jemalloc/jemalloc/blob/dev/TUNING.md Interesting. So can we make the observation that low arena count equals low fragmentation but higher thread/malloc contention. Maybe jemalloc 3.6 used one (or low) arena counts. This gives low fragmentation, but also with certain scripts (such as Yusuke's frag2.rb IO program) low performance. It may be the case that jemalloc vs glibc performance/fragmentation will not differ that much once arena count are equalised? Ideally I would like a new Ruby runtime flag `--long-lived` that was tuned for long run times (e.g low malloc arena count, JIT enabled) vs script/short usages where I want maximum performance. Sidekiq, Sinatra and Rails usually want low arena counts and JIT. Scripts want maximum immediate performance always (max arenas and no JIT). ---------------------------------------- Feature #14718: Use jemalloc by default? https://bugs.ruby-lang.org/issues/14718#change-72219 * Author: mperham (Mike Perham) * Status: Open * Priority: Normal * Assignee: * Target version: ---------------------------------------- I know Sam opened #9113 4 years ago to suggest this but I'm revisiting the topic to see if there's any movement here for Ruby 2.6 or 2.7. I supply a major piece of Ruby infrastructure (Sidekiq) and I keep hearing over and over how Ruby is terrible with memory, a huge memory hog with their Rails apps. My users switch to jemalloc and a miracle occurs: their memory usage drops massively. Some data points: https://twitter.com/brandonhilkert/status/987400365627801601 https://twitter.com/d_jones/status/989866391787335680 https://github.com/mperham/sidekiq/issues/3824#issuecomment-383072469 Redis moved to jemalloc many years ago and it solved all of their memory issues too. Their conclusion: the glibc allocator "sucks really really hard". http://oldblog.antirez.com/post/everything-about-redis-24.html This is a real pain point for the entire Rails community and would improve Ruby's reputation immensely if we can solve this problem. -- https://bugs.ruby-lang.org/