From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-2.7 required=3.0 tests=BAYES_00,FORGED_HOTMAIL_RCVD2, FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_PASS shortcircuit=no autolearn=no autolearn_force=no version=3.4.1 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 3214F1F597 for ; Sun, 29 Jul 2018 04:27:16 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 172C5120AD5; Sun, 29 Jul 2018 13:27:12 +0900 (JST) Received: from o1678916x28.outbound-mail.sendgrid.net (o1678916x28.outbound-mail.sendgrid.net [167.89.16.28]) by neon.ruby-lang.org (Postfix) with ESMTPS id 51EE6120A1F for ; Sun, 29 Jul 2018 13:27:09 +0900 (JST) Received: by filter0081p3las1.sendgrid.net with SMTP id filter0081p3las1-5256-5B5D421A-5 2018-07-29 04:27:06.100878628 +0000 UTC m=+280967.254319862 Received: from herokuapp.com (ec2-54-156-52-34.compute-1.amazonaws.com [54.156.52.34]) by ismtpd0024p1mdw1.sendgrid.net (SG) with ESMTP id GCAj5TxFTcCZbgU5qJQQFw Sun, 29 Jul 2018 04:27:06.019 +0000 (UTC) Date: Sun, 29 Jul 2018 04:27:06 +0000 (UTC) From: dennisb55@hotmail.com To: ruby-core@ruby-lang.org Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 63540 X-Redmine-Project: ruby-trunk X-Redmine-Issue-Id: 14718 X-Redmine-Issue-Author: mperham X-Redmine-Sender: bluz71 X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: ync6xU2WACa70kv/Ymy4QrNMhiuLXJG8OTL2vJD1yS4jWj+HqQDaLhSdi+qnmyu64MfzmCma+amhkJ Gl/49dyjXk92VJcbrWTgQDR7GyKyd51Uo+tqISaBpOf0VFrtspLyAuYbUtNctTzuua2aGwjIJIVNvv Xn0NSm44Ancc24KBEEd8ntP9x/vf96YhNxd29K6gMUlAvlheYQnpz9ul5Q== X-ML-Name: ruby-core X-Mail-Count: 88161 Subject: [ruby-core:88161] [Ruby trunk Feature#14718] Use jemalloc by default? X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #14718 has been updated by bluz71 (Dennis B). @andresakata, Which version of jemalloc? Ubuntu 16.04 provides jemalloc 3.6.0 in the repos, is that the version you are testing? Simple testing some of us have done indicates that newer jemalloc versions will behave **very differently** to 3.6.0. Can you test jemalloc 5.1.0, either compile it by hand or install it via [linuxbrew](http://linuxbrew.sh), and post the results please. Can you also test glibc with `MALLOC_ARENA_MAX=2`. Early on with this issue I thought the solution was simple, just move across to jemalloc, but now I realise that there is no easy fix, only headaches whichever way one goes. Ultimately we need the glibc allocator to fragment less since I doubt Ruby core will switch over to jemalloc; and if I were in their shoes now I would not switch over either (just yet). BUT, lest we sweep this under the carpet, memory fragmentation for long-lived Ruby applications (such as Rails) is a **real** problem that effects the broader Ruby community. This memory behaviour genuinely harms Ruby; but I don't know how we deal with it, maybe hard-coding `MALLOC_ARENA_MAX=2` whilst Guilds don't exist yet? ---------------------------------------- Feature #14718: Use jemalloc by default? https://bugs.ruby-lang.org/issues/14718#change-73186 * Author: mperham (Mike Perham) * Status: Open * Priority: Normal * Assignee: * Target version: ---------------------------------------- I know Sam opened #9113 4 years ago to suggest this but I'm revisiting the topic to see if there's any movement here for Ruby 2.6 or 2.7. I supply a major piece of Ruby infrastructure (Sidekiq) and I keep hearing over and over how Ruby is terrible with memory, a huge memory hog with their Rails apps. My users switch to jemalloc and a miracle occurs: their memory usage drops massively. Some data points: https://twitter.com/brandonhilkert/status/987400365627801601 https://twitter.com/d_jones/status/989866391787335680 https://github.com/mperham/sidekiq/issues/3824#issuecomment-383072469 Redis moved to jemalloc many years ago and it solved all of their memory issues too. Their conclusion: the glibc allocator "sucks really really hard". http://oldblog.antirez.com/post/everything-about-redis-24.html This is a real pain point for the entire Rails community and would improve Ruby's reputation immensely if we can solve this problem. -- https://bugs.ruby-lang.org/