From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-3.4 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS, HTML_MESSAGE,MAILING_LIST_MULTI,MIME_HTML_MOSTLY,RCVD_IN_DNSWL_MED, SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id A86ED20248 for ; Mon, 1 Apr 2019 09:28:34 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 6995412154D; Mon, 1 Apr 2019 18:28:29 +0900 (JST) Received: from mail-it1-f172.google.com (mail-it1-f172.google.com [209.85.166.172]) by neon.ruby-lang.org (Postfix) with ESMTPS id 562761216F0 for ; Mon, 1 Apr 2019 18:28:24 +0900 (JST) Received: by mail-it1-f172.google.com with SMTP id u65so14132180itc.2 for ; Mon, 01 Apr 2019 02:28:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=uTeqUCI9mlAY7/3chWS7J0tWP/HKagQsv6hgc4LGbCY=; b=faS7R7GwmEjqzoFuyuUH6GQWcTnXMvIKVQiWoTCX+THolxvuoxyq4DGQraBZ9utRsJ ju6MHmimt03rJ7sVHRF2Spqup4H4E3EzJnsXhBzWXcPd7ACgiTwl7dXev1UV/OW5ZQNp ERM4IpDzdlE2fDJI6qz7LoQrBlrQVKEq0EeWINXxsyeCQuMOg/OQYyzJuexYYvNENSty jW9xk5bqV23+KG4yFJojYbzS+TGW9oBg3qnftK8xTw5FsQ8t3z5dPlQblJ4cfTWlRYLD 9ktTxA3tDN2MOWQTMinpUl3nDt33fd9fklKkxqY/QncgYa3iKKdupwtxBtyqLYaPbiiA 3PrQ== X-Gm-Message-State: APjAAAX5pC/Z30q8jg5NOrRMMKOjMJlgVHCc9RTi/7M111OMxrABm0zm dv3EqekC73nn1TPx9GQisCCERs0xWkvUPCfTJIrjmfgk5xU= X-Google-Smtp-Source: APXvYqypyReCfBp3jyE7FnPlftsyyKP7Y+/wpoEfFgq1U0aC/9CE/SrZq0VmI35XwYG1X6mU9y5vZL63oegygVRK9RY= X-Received: by 2002:a05:660c:68a:: with SMTP id n10mr14959084itk.154.1554110904367; Mon, 01 Apr 2019 02:28:24 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Waheed Al-Barghouthi Date: Mon, 1 Apr 2019 11:28:12 +0200 Message-ID: To: Ruby developers X-ML-Name: ruby-core X-Mail-Count: 92087 Subject: [ruby-core:92087] Re: [Ruby trunk Feature#15667] Introduce malloc_trim(0) in full gc cycles X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0909483433==" Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Content-Transfer-Encoding: 7bit --===============0909483433== Content-Type: multipart/alternative; boundary="000000000000657f74058574a4f9" --000000000000657f74058574a4f9 Content-Type: text/plain; charset="UTF-8" There is some decent amount of information in that article, it's a great food for thought, for me I actually decided to use jmalloc for a production service and the problem were specifically related to sidekiq workers not releasing memory effectively when jobs are consumed. Anyone have an idea if setting M_TRIM_THRESHOLD for a value like 40 or 80 bytes would actually help in this case, as I believe it should automatically return memory to the system. I will try using its environment variable and test if there are any improvement by just using that variable. All in all, great job everyone. On Mon, Apr 1, 2019 at 6:05 AM wrote: > Issue #15667 has been updated by sam.saffron (Sam Saffron). > > File crash.png added > > @tessi > > My tests were with 3.6.0, I will do a side by side now that I have all the > infrastructure of 5 vs 3.6 and even tcmalloc. > > Nice to see that gem! Looking at my graphs I think best bang would just be > to spin a thread that does trimming every 10-30 minutes or something, > especially if you can release the GVL prior to calling it (provided this > thing is thread safe, which @carlos should know) > > > ---------------------------------------- > Feature #15667: Introduce malloc_trim(0) in full gc cycles > https://bugs.ruby-lang.org/issues/15667#change-77394 > > * Author: sam.saffron (Sam Saffron) > * Status: Open > * Priority: Normal > * Assignee: > * Target version: > ---------------------------------------- > Per Hongli's excellent article it looks like malloc_trim can help > tremendously with memory bloat issues. > > > > https://www.joyfulbikeshedding.com/blog/2019-03-14-what-causes-ruby-memory-bloat.html#a-magic-trick-trimming > > > I would like to get this patch tested side-by-side at Discourse, GitHub > and Shopify. If it looks good I think this is both a great candidate for > 2.7 and and 2.4,2.5,2.6 backports. > > > Will coordinate with Shopify and GitHub to see if we can get numbers > posted here, I will run tests on a live Discourse instance over the next > week and report numbers here. > > Koichi, what are your thoughts, to me this looks like an incredibly safe > patch, the amount of work added to major GCs is tiny compared to the > potential benefit, walking all pages is a very cheap operation. > > > > ---Files-------------------------------- > ruby_gc_malloc_trim.patch (1011 Bytes) > Screenshot_2019-03-28 Grafana - Compare Discourse Perf.png (530 KB) > crash.png (85.9 KB) > > > -- > https://bugs.ruby-lang.org/ > > Unsubscribe: > > --000000000000657f74058574a4f9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
There is some decent amount of information in that article= , it's a great food for thought, for me I actually decided to use jmall= oc for a production service and the problem were specifically related to si= dekiq workers not releasing memory effectively when jobs are consumed.=C2= =A0

Anyone have an idea if setting=C2=A0M_TRIM_THRESHOLD for a value like 40 or 80 bytes would actually help in this case, as I= believe it should automatically return memory to the system. I will try us= ing its environment variable and test if there are any improvement by just = using that variable.

All in all, great job everyon= e.

On Mon, Apr 1, 2019 at 6:05 AM <sam.saffron@gmail.com> wrote:
Issue #15667 has been updated by sam.saffron = (Sam Saffron).

File crash.png added

@tessi

My tests were with 3.6.0, I will do a side by side now that I have all the = infrastructure of 5 vs 3.6 and even tcmalloc.

Nice to see that gem! Looking at my graphs I think best bang would just be = to spin a thread that does trimming every 10-30 minutes or something, espec= ially if you can release the GVL prior to calling it (provided this thing i= s thread safe, which @carlos should know)


----------------------------------------
Feature #15667: Introduce malloc_trim(0) in full gc cycles
https://bugs.ruby-lang.org/issues/15667#change-7= 7394

* Author: sam.saffron (Sam Saffron)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
----------------------------------------
Per Hongli's excellent article it looks like malloc_trim can help treme= ndously with memory bloat issues.


https://www.joyfulbikeshedding.com/blog/2019-03-14-what-causes-ruby= -memory-bloat.html#a-magic-trick-trimming


I would like to get this patch tested side-by-side at Discourse, GitHub and= Shopify. If it looks good I think this is both a great candidate for 2.7 a= nd and 2.4,2.5,2.6 backports.


Will coordinate with Shopify and GitHub to see if we can get numbers posted= here, I will run tests on a live Discourse instance over the next week and= report numbers here.

Koichi, what are your thoughts, to me this looks like an incredibly safe pa= tch, the amount of work added to major GCs is tiny compared to the potentia= l benefit, walking all pages is a very cheap operation.



---Files--------------------------------
ruby_gc_malloc_trim.patch (1011 Bytes)
Screenshot_2019-03-28 Grafana - Compare Discourse Perf.png (530 KB)
crash.png (85.9 KB)


--
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=3Dunsubscribe= >
<http://lists.ruby-lang.org/cgi-bin/m= ailman/options/ruby-core>
--000000000000657f74058574a4f9-- --===============0909483433== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============0909483433==--