From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: poffice@blade.nagaokaut.ac.jp Delivered-To: poffice@blade.nagaokaut.ac.jp Received: from kankan.nagaokaut.ac.jp (kankan.nagaokaut.ac.jp [133.44.2.24]) by blade.nagaokaut.ac.jp (Postfix) with ESMTP id EACB017D6C89 for ; Wed, 17 Jun 2015 09:31:11 +0900 (JST) Received: from funfun.nagaokaut.ac.jp (smtp.nagaokaut.ac.jp [133.44.2.201]) by kankan.nagaokaut.ac.jp (Postfix) with ESMTP id 1C438B5D898 for ; Wed, 17 Jun 2015 09:53:38 +0900 (JST) Received: from funfun.nagaokaut.ac.jp (localhost.nagaokaut.ac.jp [127.0.0.1]) by funfun.nagaokaut.ac.jp (Postfix) with ESMTP id 2845097A82B for ; Wed, 17 Jun 2015 09:53:39 +0900 (JST) X-Virus-Scanned: amavisd-new at nagaokaut.ac.jp Authentication-Results: funfun.nagaokaut.ac.jp (amavisd-new); dkim=fail (1024-bit key) reason="fail (message has been altered)" header.d=sendgrid.me Received: from funfun.nagaokaut.ac.jp ([127.0.0.1]) by funfun.nagaokaut.ac.jp (funfun.nagaokaut.ac.jp [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eLH-83pju63U for ; Wed, 17 Jun 2015 09:53:39 +0900 (JST) Received: from voscc.nagaokaut.ac.jp (voscc.nagaokaut.ac.jp [133.44.1.100]) by funfun.nagaokaut.ac.jp (Postfix) with ESMTP id E10CF97A827 for ; Wed, 17 Jun 2015 09:53:38 +0900 (JST) Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by voscc.nagaokaut.ac.jp (Postfix) with ESMTP id B1F45952439 for ; Wed, 17 Jun 2015 09:53:37 +0900 (JST) Received: from [221.186.184.76] (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 3B8F912046F; Wed, 17 Jun 2015 09:53:36 +0900 (JST) X-Original-To: ruby-core@ruby-lang.org Delivered-To: ruby-core@ruby-lang.org Received: from o10.shared.sendgrid.net (o10.shared.sendgrid.net [173.193.132.135]) by neon.ruby-lang.org (Postfix) with ESMTPS id 78033120458 for ; Wed, 17 Jun 2015 09:53:30 +0900 (JST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.me; h=from:to:references:subject:mime-version:content-type:content-transfer-encoding:list-id; s=smtpapi; bh=uaIKZi95h6cbY9S/9/hoNby1D3E=; b=AmakZnieaKYUx/j4Ld 7+8NR0UG7aMHA7Y+PQvriXEwpWUSVsc+AkSZcZ4CYOxSF7LEnGvv8yv2E3UjAjuf xz7lLPHCQppXLW00gK68MJxINrQIh2Quni7WrLQCDi/dLQQqwC33qxD8GkONjZ7j XLamyzyipBk4j7aupjum2RXYk= Received: by filter0644p1mdw1.sendgrid.net with SMTP id filter0644p1mdw1.19971.5580C505D 2015-06-17 00:53:25.417404129 +0000 UTC Received: from herokuapp.com (ec2-54-90-247-40.compute-1.amazonaws.com [54.90.247.40]) by ismtpd-010 (SG) with ESMTP id 14dff019c61.5aef.281723 Wed, 17 Jun 2015 00:53:25 +0000 (UTC) Date: Wed, 17 Jun 2015 00:53:24 +0000 From: nobu@ruby-lang.org To: ruby-core@ruby-lang.org Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Redmine-MailingListIntegration-Message-Ids: 44155 X-Redmine-Project: ruby-trunk X-Redmine-Issue-Id: 11264 X-Redmine-Issue-Author: luke-gru X-Redmine-Sender: nobu X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: ync6xU2WACa70kv/Ymy4QrNMhiuLXJG8OTL2vJD1yS6CzQHjbb+SxyK+C9wzR/a8X1J3Z3ipHYupVb 687kBFjEj6PzNYlmsE/4b5QFD5i4GiXISnK4CtlYa09C1j5pRv03n5ICstbUUNbxcL4hep3LBVO2IE 96O94QaxnuQgfvlp9teQA4pUKbmmiQz98qX2 X-ML-Name: ruby-core X-Mail-Count: 69623 Subject: [ruby-core:69623] [Ruby trunk - Bug #11264] Memory leak in JSON stdlib ext (JSON generation) X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #11264 has been updated by Nobuyoshi Nakada. Luke Gruber wrote: > Should we continue this here or on https://github.com/flori/json/issues/251 ? I don't know what the right approach is > for std libs that are maintained separately as gems. Yes, I've posted a patch there. ---------------------------------------- Bug #11264: Memory leak in JSON stdlib ext (JSON generation) https://bugs.ruby-lang.org/issues/11264#change-52968 * Author: Luke Gruber * Status: Third Party's Issue * Priority: Normal * Assignee: * ruby -v: 2.2-head * Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN ---------------------------------------- Hi, I'm not sure if this is a bug, or just undocumented behaviour, but here's a script to reproduce the memory leak: ~~~ruby require 'json' class MyClass def to_json(*) "a" * 1048576 # 1 megabytes of chars end end class MyOther def to_json(*) raise "OMG" end end 1000.times do |i| # will leak up to ~ 4 gigs puts i JSON.dump([MyClass.new, MyClass.new, MyClass.new, MyOther.new]) rescue nil end ~~~ What's happening is that the C extension is iterating over the array to eventually dump it out to JSON. It's going through the array in order, appending to the `fbuffer` as needed. The problem is that that the API extension point of adding a `to_json` method to a class (or object), without wrapping the code in some sort of 'begin...rescue , free(buffer), re-raise' block results in the buffer never being freed. Normally this isn't too bad, except if a lot of data was appended to the buffer before the error got raised. To test it against normal behaviour in the above script, take out the offending `MyOther.new` in the array. It should run much more smoothly this way :) Note that since the `fbuffer`s aren't GC marked (not that they should be), it isn't possible to trace this leak using `GC.stat`. Once again, not sure if this is a bug or if we should never raise errors from custom `to_json` methods (ie: always wrap them in a begin... rescue block. Thanks, I also reported this to the JSON gem maintainer here: https://github.com/flori/json/issues/251 -- https://bugs.ruby-lang.org/