From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: poffice@blade.nagaokaut.ac.jp Delivered-To: poffice@blade.nagaokaut.ac.jp Received: from kankan.nagaokaut.ac.jp (kankan.nagaokaut.ac.jp [133.44.2.24]) by blade.nagaokaut.ac.jp (Postfix) with ESMTP id 914B917D759E for ; Tue, 16 Jun 2015 11:32:19 +0900 (JST) Received: from funfun.nagaokaut.ac.jp (smtp.nagaokaut.ac.jp [133.44.2.201]) by kankan.nagaokaut.ac.jp (Postfix) with ESMTP id AE476B5D890 for ; Tue, 16 Jun 2015 11:54:33 +0900 (JST) Received: from funfun.nagaokaut.ac.jp (localhost.nagaokaut.ac.jp [127.0.0.1]) by funfun.nagaokaut.ac.jp (Postfix) with ESMTP id 04E9297A82B for ; Tue, 16 Jun 2015 11:54:35 +0900 (JST) X-Virus-Scanned: amavisd-new at nagaokaut.ac.jp Authentication-Results: funfun.nagaokaut.ac.jp (amavisd-new); dkim=fail (1024-bit key) reason="fail (message has been altered)" header.d=sendgrid.me Received: from funfun.nagaokaut.ac.jp ([127.0.0.1]) by funfun.nagaokaut.ac.jp (funfun.nagaokaut.ac.jp [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CQ2fYh1Z0kbY for ; Tue, 16 Jun 2015 11:54:34 +0900 (JST) Received: from voscc.nagaokaut.ac.jp (voscc.nagaokaut.ac.jp [133.44.1.100]) by funfun.nagaokaut.ac.jp (Postfix) with ESMTP id B436F97A820 for ; Tue, 16 Jun 2015 11:54:34 +0900 (JST) Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by voscc.nagaokaut.ac.jp (Postfix) with ESMTP id 51CD0952439 for ; Tue, 16 Jun 2015 11:54:33 +0900 (JST) Received: from [221.186.184.76] (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 8B9DB12045B; Tue, 16 Jun 2015 11:54:31 +0900 (JST) X-Original-To: ruby-core@ruby-lang.org Delivered-To: ruby-core@ruby-lang.org Received: from o2.heroku.sendgrid.net (o2.heroku.sendgrid.net [67.228.50.55]) by neon.ruby-lang.org (Postfix) with ESMTPS id 796D0120451 for ; Tue, 16 Jun 2015 11:54:25 +0900 (JST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.me; h=from:to:references:subject:mime-version:content-type:content-transfer-encoding:list-id; s=smtpapi; bh=f4faDosUeGQRrJQcAGqHiDWNBaI=; b=Fu1cXTCkpd+ewsXF80 xT7twLLlz9iee82waR4g01OFqLbs2JjL75tmy3XQFSi82efb8/lx5OnVUSx3il6D nH/mFl0F5JqwFVNPP1EBmgv+0KFuCCAv/SXHV8XsJQtRPB+2/J5tcezO+DGvvQ9L zaca4CIwtn4GgNR/k+KnPn2pc= Received: by filter0523p1mdw1.sendgrid.net with SMTP id filter0523p1mdw1.24563.557F8FDA1C 2015-06-16 02:54:18.958453083 +0000 UTC Received: from herokuapp.com (ec2-54-159-247-118.compute-1.amazonaws.com [54.159.247.118]) by ismtpd-001 (SG) with ESMTP id 14dfa49eefb.21ce.224c76 for ; Tue, 16 Jun 2015 02:54:18 +0000 (UTC) Date: Tue, 16 Jun 2015 02:54:18 +0000 From: luke.gru@gmail.com To: ruby-core@ruby-lang.org Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Redmine-MailingListIntegration-Message-Ids: 44122 X-Redmine-Project: ruby-trunk X-Redmine-Issue-Id: 11264 X-Redmine-Issue-Author: luke-gru X-Redmine-Sender: luke-gru X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: ync6xU2WACa70kv/Ymy4QrNMhiuLXJG8OTL2vJD1yS4iSZP7jdGCh/747IWcflFmqohVKfAMTYrmkj MkN7t7xakMJpsvFgHxjHzJ3+cBwsEjHiEEiQXFiXIE49T3ruL2c1/F1Z0AsYi6vgouvcbbmrYbileW qlzUwBHGA6ts/ho0iX2ju/AAhdYfsJmTErld X-ML-Name: ruby-core X-Mail-Count: 69599 Subject: [ruby-core:69599] [Ruby trunk - Bug #11264] [Open] Memory leak in JSON stdlib ext (JSON generation) X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #11264 has been reported by Luke Gruber. ---------------------------------------- Bug #11264: Memory leak in JSON stdlib ext (JSON generation) https://bugs.ruby-lang.org/issues/11264 * Author: Luke Gruber * Status: Open * Priority: Normal * Assignee: * ruby -v: 2.2-head * Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN ---------------------------------------- Hi, I'm not sure if this is a bug, or just undocumented behaviour, but here's a script to reproduce the memory leak: ------------------------------ require 'json' class MyClass def to_json(*) "a" * 1048576 # 1 megabytes of chars end end class MyOther def to_json(*) raise "OMG" end end 1000.times do |i| # will leak up to ~ 4 gigs puts i JSON.dump([MyClass.new, MyClass.new, MyClass.new, MyOther.new]) rescue nil end ------------------------------ What's happening is that the C extension is iterating over the array to eventually dump it out to JSON. It's going through the array in order, appending to the fbuffer as needed. The problem is that that the API extension point of adding a to_json method to a class (or object), without wrapping the code in some sort of 'begin...rescue , free(buffer), re-raise' block results in the buffer never being freed. Normally this isn't too bad, except if a lot of data was appended to the buffer before the error got raised. To test it against normal behaviour in the above script, take out the offending MyOther.new in the array. It should run much more smoothly this way :) Note that since the fbuffers aren't GC marked (not that they should be), it isn't possible to trace this leak using GC.stat. Once again, not sure if this is a bug or if we should never raise errors from custom to_json methods (ie: always wrap them in a begin... rescue block. Thanks, I also reported this to the JSON gem maintainer here: https://github.com/flori/json/issues/251 -- https://bugs.ruby-lang.org/