From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-3.6 required=3.0 tests=AWL,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.0 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 3BB5B1F51C for ; Thu, 17 May 2018 17:31:26 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 3716E120AFE; Fri, 18 May 2018 02:31:22 +0900 (JST) Received: from o1678916x28.outbound-mail.sendgrid.net (o1678916x28.outbound-mail.sendgrid.net [167.89.16.28]) by neon.ruby-lang.org (Postfix) with ESMTPS id 41B09120AF9 for ; Fri, 18 May 2018 02:31:20 +0900 (JST) Received: by filter0027p3iad2.sendgrid.net with SMTP id filter0027p3iad2-19790-5AFDBC63-97 2018-05-17 17:31:15.901431748 +0000 UTC Received: from herokuapp.com (ec2-54-226-155-145.compute-1.amazonaws.com [54.226.155.145]) by ismtpd0007p1iad1.sendgrid.net (SG) with ESMTP id E6L772EjQHa4q7IRgE7TtQ Thu, 17 May 2018 17:31:15.808 +0000 (UTC) Date: Thu, 17 May 2018 17:31:16 +0000 (UTC) From: mperham@gmail.com To: ruby-core@ruby-lang.org Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 62508 X-Redmine-Project: ruby-trunk X-Redmine-Issue-Id: 14718 X-Redmine-Issue-Author: mperham X-Redmine-Sender: mperham X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: ync6xU2WACa70kv/Ymy4QrNMhiuLXJG8OTL2vJD1yS5zOH/QiVaSPgGdMM1Yc+JDveU5oZw66nQG1l HqURFv2+5nYaZBWOMz80Tw8i8Sm2ALbj23J2T5tUa1wsuN800OxLnDQZgSWtAOph0oFn9xaG0Pe+zO aad1uHgL45JFoY9pZfue/f+oxwXAm26+8V+dWHY6qtF+M+zvUuajrrV1Qw== X-ML-Name: ruby-core X-Mail-Count: 87145 Subject: [ruby-core:87145] [Ruby trunk Feature#14718] Use jemalloc by default? X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #14718 has been updated by mperham (Mike Perham). Yusuke, I'm not sure, I can't explain that. It does get worse as the machine gets larger. A machine with more cores will see larger bloat, which is what that graph above shows (36 cores, 40GB -> 9GB); the GVL does not appear to help. The glibc memory allocation internals are documented here: https://sourceware.org/glibc/wiki/MallocInternals ---------------------------------------- Feature #14718: Use jemalloc by default? https://bugs.ruby-lang.org/issues/14718#change-72136 * Author: mperham (Mike Perham) * Status: Open * Priority: Normal * Assignee: * Target version: ---------------------------------------- I know Sam opened #9113 4 years ago to suggest this but I'm revisiting the topic to see if there's any movement here for Ruby 2.6 or 2.7. I supply a major piece of Ruby infrastructure (Sidekiq) and I keep hearing over and over how Ruby is terrible with memory, a huge memory hog with their Rails apps. My users switch to jemalloc and a miracle occurs: their memory usage drops massively. Some data points: https://twitter.com/brandonhilkert/status/987400365627801601 https://twitter.com/d_jones/status/989866391787335680 https://github.com/mperham/sidekiq/issues/3824#issuecomment-383072469 Redis moved to jemalloc many years ago and it solved all of their memory issues too. Their conclusion: the glibc allocator "sucks really really hard". http://oldblog.antirez.com/post/everything-about-redis-24.html This is a real pain point for the entire Rails community and would improve Ruby's reputation immensely if we can solve this problem. -- https://bugs.ruby-lang.org/