From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-4.1 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 9BED720248 for ; Sun, 31 Mar 2019 01:52:43 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id B2922120FFA; Sun, 31 Mar 2019 10:52:38 +0900 (JST) Received: from o1678916x28.outbound-mail.sendgrid.net (o1678916x28.outbound-mail.sendgrid.net [167.89.16.28]) by neon.ruby-lang.org (Postfix) with ESMTPS id 7D278120C35 for ; Sun, 31 Mar 2019 10:52:36 +0900 (JST) Received: by filter0062p3las1.sendgrid.net with SMTP id filter0062p3las1-15753-5CA01D63-7 2019-03-31 01:52:35.489681445 +0000 UTC m=+120413.281765577 Received: from herokuapp.com (unknown [3.85.129.45]) by ismtpd0029p1mdw1.sendgrid.net (SG) with ESMTP id wtLWrf-8QGOGLre3w2y-Pg for ; Sun, 31 Mar 2019 01:52:35.289 +0000 (UTC) Date: Sun, 31 Mar 2019 01:52:36 +0000 (UTC) From: philipp@tessenow.org Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 67546 X-Redmine-Project: ruby-trunk X-Redmine-Issue-Id: 15667 X-Redmine-Issue-Author: sam.saffron X-Redmine-Sender: tessi X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: =?us-ascii?Q?Z9gEu+R3P8Zk6I7xV14clx0bvwfClD2mXVPKEzUO=2FRTAQA6nuuOnvLqgPqbsj8?= =?us-ascii?Q?VSQjmWF7B5QcjQZQlESocba+m7dnAQPLbQTNVyt?= =?us-ascii?Q?xOlZx7nwaBNi+X52=2FjSAiIlHhqc38aB=2FO0FeNgt?= =?us-ascii?Q?+FAol7SsP2TNjHzFkdMdtMr+MWEQJd4Rm1bZ3Q6?= =?us-ascii?Q?noRgzMw2=2FSwUs0TwKnAXEM4xX0gm2bM8zpw=3D=3D?= To: ruby-core@ruby-lang.org X-ML-Name: ruby-core X-Mail-Count: 92060 Subject: [ruby-core:92060] [Ruby trunk Feature#15667] Introduce malloc_trim(0) in full gc cycles X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #15667 has been updated by tessi (Philipp Tessenow). FYI: To easy testing this change, I just pushed a small gem to rubygems `malloc_trim` (https://github.com/tessi/malloc_trim). This gives access to `malloc_trim` to ruby land to let us play with it without the need to re-compile ruby and/or deploying a custom patches ruby. With `MallocTrim.enable_trimming`, there is also a built in way to run `malloc_trim(0)` after every GC MARK (the most relevant internal event I found to hook into). It probably calls malloc_trim somewhat too often -- I'd be happy for suggestions to find a better hook. In any way, doing a manual `MallocTrim.trim` after a `GC.start` (e.g. between two rails requests) is still possible with this gem. ---------------------------------------- Feature #15667: Introduce malloc_trim(0) in full gc cycles https://bugs.ruby-lang.org/issues/15667#change-77383 * Author: sam.saffron (Sam Saffron) * Status: Open * Priority: Normal * Assignee: * Target version: ---------------------------------------- Per Hongli's excellent article it looks like malloc_trim can help tremendously with memory bloat issues. https://www.joyfulbikeshedding.com/blog/2019-03-14-what-causes-ruby-memory-bloat.html#a-magic-trick-trimming I would like to get this patch tested side-by-side at Discourse, GitHub and Shopify. If it looks good I think this is both a great candidate for 2.7 and and 2.4,2.5,2.6 backports. Will coordinate with Shopify and GitHub to see if we can get numbers posted here, I will run tests on a live Discourse instance over the next week and report numbers here. Koichi, what are your thoughts, to me this looks like an incredibly safe patch, the amount of work added to major GCs is tiny compared to the potential benefit, walking all pages is a very cheap operation. ---Files-------------------------------- ruby_gc_malloc_trim.patch (1011 Bytes) Screenshot_2019-03-28 Grafana - Compare Discourse Perf.png (530 KB) -- https://bugs.ruby-lang.org/