From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: poffice@blade.nagaokaut.ac.jp Delivered-To: poffice@blade.nagaokaut.ac.jp Received: from kankan.nagaokaut.ac.jp (kankan.nagaokaut.ac.jp [133.44.2.24]) by blade.nagaokaut.ac.jp (Postfix) with ESMTP id DB6DA17DEE4A for ; Tue, 16 Jun 2015 12:38:15 +0900 (JST) Received: from funfun.nagaokaut.ac.jp (funfun.nagaokaut.ac.jp [133.44.2.201]) by kankan.nagaokaut.ac.jp (Postfix) with ESMTP id 81154B5D964 for ; Tue, 16 Jun 2015 13:00:30 +0900 (JST) Received: from funfun.nagaokaut.ac.jp (localhost.nagaokaut.ac.jp [127.0.0.1]) by funfun.nagaokaut.ac.jp (Postfix) with ESMTP id EE00797A82B for ; Tue, 16 Jun 2015 13:00:31 +0900 (JST) X-Virus-Scanned: amavisd-new at nagaokaut.ac.jp Authentication-Results: funfun.nagaokaut.ac.jp (amavisd-new); dkim=fail (1024-bit key) reason="fail (message has been altered)" header.d=sendgrid.me Received: from funfun.nagaokaut.ac.jp ([127.0.0.1]) by funfun.nagaokaut.ac.jp (funfun.nagaokaut.ac.jp [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qaz93-YBsQpP for ; Tue, 16 Jun 2015 13:00:31 +0900 (JST) Received: from voscc.nagaokaut.ac.jp (voscc.nagaokaut.ac.jp [133.44.1.100]) by funfun.nagaokaut.ac.jp (Postfix) with ESMTP id A861697A827 for ; Tue, 16 Jun 2015 13:00:31 +0900 (JST) Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by voscc.nagaokaut.ac.jp (Postfix) with ESMTP id 23D2295243E for ; Tue, 16 Jun 2015 13:00:29 +0900 (JST) Received: from [221.186.184.76] (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id DCB75120461; Tue, 16 Jun 2015 13:00:27 +0900 (JST) X-Original-To: ruby-core@ruby-lang.org Delivered-To: ruby-core@ruby-lang.org Received: from o10.shared.sendgrid.net (o10.shared.sendgrid.net [173.193.132.135]) by neon.ruby-lang.org (Postfix) with ESMTPS id 07A1F120454 for ; Tue, 16 Jun 2015 13:00:22 +0900 (JST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.me; h=from:to:references:subject:mime-version:content-type:content-transfer-encoding:list-id; s=smtpapi; bh=jtTonlEeSouGWuNtGJ/yemoQ9fs=; b=dVxcF1YUSLZwpUg2Br GPlM5BOxZpt6saeLVH84BrZCSxls5YCHuAf8PWsa20dDzijO4kJq2ZKimZSEwHwa o/Z46RTMUQztspmR4rT1KOaEd2TERCcabH3sqLZvSmYUOQK1qfse8/nuqsgZxak/ Qan+qJcU7UDacXWTfo4W80C1M= Received: by filter0461p1mdw1.sendgrid.net with SMTP id filter0461p1mdw1.10924.557F9F511 2015-06-16 04:00:18.470682406 +0000 UTC Received: from herokuapp.com (ec2-54-81-39-135.compute-1.amazonaws.com [54.81.39.135]) by ismtpd-039 (SG) with ESMTP id 14dfa8659b6.e13.1b0fbd for ; Tue, 16 Jun 2015 04:00:18 +0000 (UTC) Date: Tue, 16 Jun 2015 04:00:18 +0000 From: nobu@ruby-lang.org To: ruby-core@ruby-lang.org Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Redmine-MailingListIntegration-Message-Ids: 44124 X-Redmine-Project: ruby-trunk X-Redmine-Issue-Id: 11264 X-Redmine-Issue-Author: luke-gru X-Redmine-Sender: nobu X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: ync6xU2WACa70kv/Ymy4QrNMhiuLXJG8OTL2vJD1yS5zGVnKVQPek2rV3yC5ZBVj/1Tfw/ihvp5C6u UgVKBb7lD7Sa6h9VC5ykS3YrCCCyJuMy+0BcaE5B2TEqT7n6Pf+547J72nrf4XYBmNs026Qqd/AQZM EPPTZOAemidl7JCZnd9vmk5iTEIwVtvbhl19 X-ML-Name: ruby-core X-Mail-Count: 69601 Subject: [ruby-core:69601] [Ruby trunk - Bug #11264] [Third Party's Issue] Memory leak in JSON stdlib ext (JSON generation) X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #11264 has been updated by Nobuyoshi Nakada. Description updated Status changed from Open to Third Party's Issue Luke Gruber wrote: > Once again, not sure if this is a bug or if we should never raise errors from custom to_json methods (ie: always wrap them in a begin... rescue block. It doesn't work enough, as exceptions can raise even inside `cState_prepare_buffer()`. ---------------------------------------- Bug #11264: Memory leak in JSON stdlib ext (JSON generation) https://bugs.ruby-lang.org/issues/11264#change-52942 * Author: Luke Gruber * Status: Third Party's Issue * Priority: Normal * Assignee: * ruby -v: 2.2-head * Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN ---------------------------------------- Hi, I'm not sure if this is a bug, or just undocumented behaviour, but here's a script to reproduce the memory leak: ~~~ruby require 'json' class MyClass def to_json(*) "a" * 1048576 # 1 megabytes of chars end end class MyOther def to_json(*) raise "OMG" end end 1000.times do |i| # will leak up to ~ 4 gigs puts i JSON.dump([MyClass.new, MyClass.new, MyClass.new, MyOther.new]) rescue nil end ~~~ What's happening is that the C extension is iterating over the array to eventually dump it out to JSON. It's going through the array in order, appending to the `fbuffer` as needed. The problem is that that the API extension point of adding a `to_json` method to a class (or object), without wrapping the code in some sort of 'begin...rescue , free(buffer), re-raise' block results in the buffer never being freed. Normally this isn't too bad, except if a lot of data was appended to the buffer before the error got raised. To test it against normal behaviour in the above script, take out the offending `MyOther.new` in the array. It should run much more smoothly this way :) Note that since the `fbuffer`s aren't GC marked (not that they should be), it isn't possible to trace this leak using `GC.stat`. Once again, not sure if this is a bug or if we should never raise errors from custom `to_json` methods (ie: always wrap them in a begin... rescue block. Thanks, I also reported this to the JSON gem maintainer here: https://github.com/flori/json/issues/251 -- https://bugs.ruby-lang.org/