From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: poffice@blade.nagaokaut.ac.jp Delivered-To: poffice@blade.nagaokaut.ac.jp Received: from kankan.nagaokaut.ac.jp (kankan.nagaokaut.ac.jp [133.44.2.24]) by blade.nagaokaut.ac.jp (Postfix) with ESMTP id CD2B41F03A2 for ; Mon, 20 Apr 2009 23:30:12 +0900 (JST) Received: from funfun.nagaokaut.ac.jp (funfun.nagaokaut.ac.jp [133.44.2.201]) by kankan.nagaokaut.ac.jp (Postfix) with ESMTP id 815EDEA5191 for ; Mon, 20 Apr 2009 23:23:07 +0900 (JST) Received: from localhost (localhost.nagaokaut.ac.jp [127.0.0.1]) by funfun.nagaokaut.ac.jp (Postfix) with ESMTP id 683061FA002 for ; Mon, 20 Apr 2009 23:23:08 +0900 (JST) X-Virus-Scanned: amavisd-new at funfun.nagaokaut.ac.jp Received: from funfun.nagaokaut.ac.jp ([127.0.0.1]) by localhost (funfun.nagaokaut.ac.jp [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Tubqngp3a0IF for ; Mon, 20 Apr 2009 23:23:08 +0900 (JST) Received: from voscc.nagaokaut.ac.jp (voscc.nagaokaut.ac.jp [133.44.1.100]) by funfun.nagaokaut.ac.jp (Postfix) with ESMTP id 40C8E1FA001 for ; Mon, 20 Apr 2009 23:23:08 +0900 (JST) Received: from carbon.ruby-lang.org (carbon.ruby-lang.org [221.186.184.68]) by voscc.nagaokaut.ac.jp (Postfix) with ESMTP id 27FFD95243A for ; Mon, 20 Apr 2009 23:23:03 +0900 (JST) Received: from beryllium.ruby-lang.org (beryllium.ruby-lang.org [127.0.0.1]) by carbon.ruby-lang.org (Postfix) with ESMTP id AD5893C21EB6B; Mon, 20 Apr 2009 23:23:03 +0900 (JST) Received: from mail-fx0-f170.google.com (mail-fx0-f170.google.com [209.85.220.170]) by carbon.ruby-lang.org (Postfix) with ESMTP id 7C50D3C21EB5C for ; Mon, 20 Apr 2009 23:22:57 +0900 (JST) Received: by fxm18 with SMTP id 18so2107035fxm.7 for ; Mon, 20 Apr 2009 07:22:55 -0700 (PDT) Received: by 10.239.130.196 with SMTP id 4mr246479hbk.17.1240237375566; Mon, 20 Apr 2009 07:22:55 -0700 (PDT) Delivered-To: ruby-core@ruby-lang.org Date: Mon, 20 Apr 2009 23:22:58 +0900 Posted: Mon, 20 Apr 2009 16:22:55 +0200 From: "Muhammad A. Ali" Reply-To: ruby-core@ruby-lang.org Subject: [ruby-core:23263] Re: [Bug #1392] Object#extend leaks memory on Ruby 1.9.1 To: ruby-core@ruby-lang.org Message-Id: <4cb4766d0904200722x20f2ed08y250036d0684d39b@mail.gmail.com> In-Reply-To: <4cb4766d0904200623p3f255cf6r852b76c39cd437d4@mail.gmail.com> References: <49ea51a1deec_323f5bb5b52104e@redmine.ruby-lang.org> <4cb4766d0904200623p3f255cf6r852b76c39cd437d4@mail.gmail.com> X-ML-Name: ruby-core X-Mail-Count: 23263 X-MLServer: fml [fml 4.0.3 release (20011202/4.0.3)]; post only (only members can post) X-ML-Info: If you have a question, send e-mail with the body "help" (without quotes) to the address ruby-core-ctl@ruby-lang.org; help= X-Spam-Checker-Version: SpamAssassin 3.1.7-deb3 (2006-10-05) on carbon.ruby-lang.org X-Spam-Level: ** X-Spam-Status: No, score=2.3 required=7.0 tests=ARIN,BAYES_20, CONTENT_TYPE_PRESENT,FAKEDWORD_ATMARK,FAKEDWORD_VERTICALLINE, HTML_30_40,HTML_MESSAGE,MIMEQENC,MULTIPART_ALTERNATIVE,QENCPTR1, QENCPTR2 autolearn=disabled version=3.1.7-deb3 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=mF/WXDg8rgxd/0JRrMfnWdueMUC700TXCz06BZuhkbw=; b=p6CVB0at4xeRpeJX1tvZza4W/oRfKN/JgqBAjlK8jlaoJZQTKXedp86mtZ2vD2S7rT F/8w3NCziw3mE2NXoft0BpjsgXfjvrq/CmJsM2M/w/JKPA3kK7YP8UDObFBuNXEIy7Q1 AviFHLTh+acyVBEr/VYvZYJzG1GUVjEs5exdA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=ieuP/AP0/oxkT5Uj9wOwNjb3I8b2ssKGhcZUVYHvzw9opdPdyI96T13tYMav6e4Dc9 Sx0nnWHySXLdm2M7ErAkdIvDsuBywpmfVnK41rcwSac6ypTGtrEBcbalpFF2rra7LzDu 2c5sS8CaZSMmX+/9jGBXTL0s/PGpvRMQv26ss= Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=00163649906935341d0467fd4493 Precedence: bulk List-Id: ruby-core.ruby-lang.org List-Software: fml [fml 4.0.3 release (20011202/4.0.3)] List-Post: List-Owner: List-Help: List-Unsubscribe: --00163649906935341d0467fd4493 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On second thoughts, I still believe there is a leak. I modified the test as follows: @extend = ARGV[0] module BetterHash def blabla end end unless @extend class Hash include BetterHash end end t = Time.now 100.times do #split the allocation process 10000.times do s = {} s.extend BetterHash if @extend end GC.start # force a gc after each allocation batch end after = Time.now - t puts "done with #{GC.count} gc runs after #{after} seconds" This way the allocation is done in batches and the GC should be able to free the objects after each batch. After the first few batches the vm should have enough free space for any of the next ones. Not what happens though as it keeps growing till it reaches like ~18 MB on my machine. (the include version maxes at 2.8). If I add a delay between batches I could watch the process as it uses more memory gradually. To further prove I added the following code before the end: @strings = [] 10240.times do @strings << "a"*1024 end This will alocate around 10MB worth of small strings (1K each). The include version reports an increase of 10MB in its RAM usage. The troubling thing is that the extend version also reports a 10MB increase in its RAM usage (up to ~29) when it has supposedly more than enough heap space to allocate those small strings. thoughts? oldmoe oldmoe.blogspot.com On Mon, Apr 20, 2009 at 3:23 PM, Muhammad A. Ali wrote: > Hi, > > Thanks for the clarification. The strange thing though (which lead me to > this conclusion) is that 1.8 consistently maintains > > the same memory usage for both the include and the extend versions. Could > this be attributed to the frequency of GC runs? > > Regards > > oldmoe > oldmoe.blogspot.com > > On Mon, Apr 20, 2009 at 5:43 AM, Yukihiro Matsumoto wrote: > >> Hi, >> >> In message "Re: [ruby-core:23252] [Bug #1392] Object#extend leaks memory >> on Ruby 1.9.1" >> on Sun, 19 Apr 2009 07:18:18 +0900, Muhammad Ali < >> redmine@ruby-lang.org> writes: >> | >> |Bug #1392: Object#extend leaks memory on Ruby 1.9.1 >> |http://redmine.ruby-lang.org/issues/show/1392 >> >> |A few bytes are leaked every time Object#extend is called, here is a >> sample 1.9.1 script >> >> It does not leak memory. the version that use #extend allocates >> internal class-like object for each hash object, so that it allocates >> lot more objects than #include version, thus requires more heap space >> to be allocated. The GC does reclaim the unused objects, but it may >> not be able to return heap space to the underlying OS. >> >> To confirm there's no memory leak, replace sleep with >> >> GC.start >> p ObjectSpace.count_objects >> >> It prints the number of live objects, e.g. >> >> {:TOTAL=>17185, :FREE=>9590, :T_OBJECT=>5, :T_CLASS=>474, :T_MODULE=>20, >> :T_FLOAT=>6, :T_STRING=>1551, :T_REGEXP=>10, :T_ARRAY=>23, :T_HASH=>3, >> :T_BIGNUM=>2, :T_FILE=>3, :T_DATA=>28, :T_COMPLEX=>1, :T_NODE=>5451, >> :T_ICLASS=>18} >> >> matz. >> >> > --00163649906935341d0467fd4493 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

On second thoughts, I still believe there is a leak. I = modified the test as follows:

@extend =3D ARGV[0]

module B= etterHash
=A0 def blabla
=A0 end
end

unless @extend
=A0 = class Hash
=A0 include BetterHash
=A0 end
end

t =3D Time.now
100.tim= es do #split the allocation process
=A0 10000.times do=A0
=A0 =A0 = s =3D {}
=A0 =A0 s.extend BetterHash if @extend=A0
=A0 end
=A0 = GC.start # force a gc after each allocation batch
end

after =3D Time.now - t
puts "done with #{GC.count} gc= runs after #{after} seconds"

This way the allocation is done i= n batches and the GC should be able to free the objects after each batch. <= /p>

After the first few batches the vm should have enough free space for any= of the next ones. Not what happens

though as it keeps growing ti= ll it reaches like ~18 MB on my machine. (the include version maxes at 2.8)= .

If I add a delay between batches I could watch the process as it uses mo= re memory gradually.

To further prove I added the following code befo= re the end:

@strings =3D []
10240.times do
=A0@strings <<= "a"*1024=A0
end

This will alocate around 10MB worth of small strings (1K each). = The include version reports an increase of 10MB in its RAM usage.

The troubling thing is that the extend version also reports a 10MB increas= e in its RAM usage (up to ~29) when it has supposedly

more than enough heap space to allocate those small strings.

thoughts?
oldmoe
oldmoe.blogspot.com
On Mon, Apr 20,= 2009 at 3:23 PM, Muhammad A. Ali <oldmoe@gmail.com> wrote:

Hi,

Thanks for the clarificatio= n. The strange thing though (which lead me to this conclusion) is that 1.8 = consistently maintains

the same memory usage for both the include= and the extend versions. Could this be attributed to the frequency of GC r= uns?

Regards

oldmoe
oldmoe.blogspot.com


On Mon, Apr 20, 2009 at 5:43 AM, Yukihiro Matsumoto <= matz@ruby-lang.org> wrote:
Hi,

In message "Re: [ruby-core:23252] [Bug #1392] Object#extend leaks memo= ry on Ruby 1.9.1"
|A few bytes are leaked every time Object#extend is called, here= is a sample 1.9.1 script

It does not leak memory. =A0the version that use #extend allocates internal class-like object for each hash object, so that it allocates
lot more objects than #include version, thus requires more heap space
to be allocated. =A0The GC does reclaim the unused objects, but it may
not be able to return heap space to the underlying OS.

To confirm there's no memory leak, replace sleep with

=A0GC.start
=A0p ObjectSpace.count_objects

It prints the number of live objects, e.g.

{:TOTAL=3D>17185, :FREE=3D>9590, :T_OBJECT=3D>5, :T_CLASS=3D>47= 4, :T_MODULE=3D>20, :T_FLOAT=3D>6, :T_STRING=3D>1551, :T_REGEXP=3D= >10, :T_ARRAY=3D>23, :T_HASH=3D>3, :T_BIGNUM=3D>2, :T_FILE=3D&g= t;3, :T_DATA=3D>28, :T_COMPLEX=3D>1, :T_NODE=3D>5451, :T_ICLASS=3D= >18}

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0matz.



--00163649906935341d0467fd4493--