[ruby-core:69341] [Ruby trunk - Bug #11174] [Open] threads memory leak

[ruby-core:69341] [Ruby trunk - Bug #11174] [Open] threads memory leak

[from-ruby-lang]

Issue #11174 has been reported by Cyril Vechera.

----------------------------------------
Bug #11174: threads memory leak
https://bugs.ruby-lang.org/issues/11174

* Author: Cyril Vechera
* Status: Open
* Priority: Normal
* Assignee: 
* ruby -v: 2.2.0
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------
There's strong memory growth during intensive thread using. 

Script to demonstrate the problem (on x86_64-linux):

~~~
loop {
	10.times { 1000.times.map { Thread.new { } }.each(&:join) }
	GC.start # not necessary, just to be sure
	puts File.open('/proc/self/status').grep(/VmRSS:/).first
}
~~~

Running this script shows RSS growing from 45 Mb at the start time to 700 Mb after few minutes.

~~~
$ ruby thread_memleak4.rb
VmRSS:     45036 kB
VmRSS:     66748 kB
VmRSS:     87024 kB
...
VmRSS:    678052 kB
~~~

---Files--------------------------------
thread_memleak4.rb (138 Bytes)


-- 
https://bugs.ruby-lang.org/

[ruby-core:69845] [Ruby trunk - Bug #11174] threads memory leak

[from-ruby-lang]

Issue #11174 has been updated by Cyril Vechera.

ruby -v changed from 2.2.0 to 2.2.3, 2.2.0, 2.1.0, 2.0.0, 1.9.3

I've checked the bug on older versions and the 2.2-head: leaks all but 1.8.7

~~~
$ ruby --version
ruby 2.2.3p139 (2015-07-01) [x86_64-linux]
$ ruby /tmp/thread_memleak4.rb
VmRSS:     47152 kB
VmRSS:     72872 kB
VmRSS:     90448 kB
VmRSS:    105752 kB

~~~

~~~
$ ruby --version
ruby 1.8.7 (2014-01-28 patchlevel 376) [x86_64-linux]
$ ruby /tmp/thread_memleak4.rb
VmRSS:      2496 kB
VmRSS:      3688 kB
VmRSS:      2768 kB
VmRSS:      2536 kB
VmRSS:      2536 kB

~~~

----------------------------------------
Bug #11174: threads memory leak
https://bugs.ruby-lang.org/issues/11174#change-53244

* Author: Cyril Vechera
* Status: Open
* Priority: Normal
* Assignee: 
* ruby -v: 2.2.3, 2.2.0, 2.1.0, 2.0.0, 1.9.3
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------
There's strong memory growth during intensive thread using. 

Script to demonstrate the problem (on x86_64-linux):

~~~
loop {
	10.times { 1000.times.map { Thread.new { } }.each(&:join) }
	GC.start # not necessary, just to be sure
	puts File.open('/proc/self/status').grep(/VmRSS:/).first
}
~~~

Running this script shows RSS growing from 45 Mb at the start time to 700 Mb after few minutes.

~~~
$ ruby thread_memleak4.rb
VmRSS:     45036 kB
VmRSS:     66748 kB
VmRSS:     87024 kB
...
VmRSS:    678052 kB
~~~

---Files--------------------------------
thread_memleak4.rb (138 Bytes)


-- 
https://bugs.ruby-lang.org/

[ruby-core:82499] [Ruby trunk Bug#11174] threads memory leak

[kadmos]

Issue #11174 has been updated by parhs (Gibon M).


I am having similar issue. Even calling open3.capture2 which uses threads results into 250mb of rss memory after a day. 2.3.4 and all 2.3.x.. ,2.4.x

----------------------------------------
Bug #11174: threads memory leak
https://bugs.ruby-lang.org/issues/11174#change-66308

* Author: cvss (Cyril Vechera)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 2.2.3, 2.2.0, 2.1.0, 2.0.0, 1.9.3
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------
There's strong memory growth during intensive thread using. 

Script to demonstrate the problem (on x86_64-linux):

~~~
loop {
	10.times { 1000.times.map { Thread.new { } }.each(&:join) }
	GC.start # not necessary, just to be sure
	puts File.open('/proc/self/status').grep(/VmRSS:/).first
}
~~~

Running this script shows RSS growing from 45 Mb at the start time to 700 Mb after few minutes.

~~~
$ ruby thread_memleak4.rb
VmRSS:     45036 kB
VmRSS:     66748 kB
VmRSS:     87024 kB
...
VmRSS:    678052 kB
~~~

---Files--------------------------------
thread_memleak4.rb (138 Bytes)


-- 
https://bugs.ruby-lang.org/

[ruby-core:82802] [Ruby trunk Bug#11174] threads memory leak

[ko1]

Issue #11174 has been updated by ko1 (Koichi Sasada).

Assignee set to ko1 (Koichi Sasada)

Sorry I missed it.

----------------------------------------
Bug #11174: threads memory leak
https://bugs.ruby-lang.org/issues/11174#change-66678

* Author: cvss (Cyril Vechera)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* Target version: 
* ruby -v: 2.2.3, 2.2.0, 2.1.0, 2.0.0, 1.9.3
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------
There's strong memory growth during intensive thread using. 

Script to demonstrate the problem (on x86_64-linux):

~~~
loop {
	10.times { 1000.times.map { Thread.new { } }.each(&:join) }
	GC.start # not necessary, just to be sure
	puts File.open('/proc/self/status').grep(/VmRSS:/).first
}
~~~

Running this script shows RSS growing from 45 Mb at the start time to 700 Mb after few minutes.

~~~
$ ruby thread_memleak4.rb
VmRSS:     45036 kB
VmRSS:     66748 kB
VmRSS:     87024 kB
...
VmRSS:    678052 kB
~~~

---Files--------------------------------
thread_memleak4.rb (138 Bytes)


-- 
https://bugs.ruby-lang.org/

[ruby-core:89004] [Ruby trunk Bug#11174] threads memory leak

[hartator]

Issue #11174 has been updated by hartator (Julien Khaleghy).


ko1 (Koichi Sasada) wrote:
> Sorry I missed it.

Thanks for the awesome work on MRI.

I still have the same issue on Ruby 2.5.1 (ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]):

~~~ ruby
VmRSS:	   60116 kB
VmRSS:	   79448 kB
VmRSS:	   89768 kB
VmRSS:	   99276 kB
VmRSS:	  107916 kB
VmRSS:	  113728 kB
VmRSS:	  118736 kB
VmRSS:	  126600 kB
...
after 5 minutes
....
VmRSS:	  754552 kB
VmRSS:	  757876 kB
VmRSS:	  760132 kB
~~~


Any idea if it's already fixed somewhere else, or you guys still working on it?

----------------------------------------
Bug #11174: threads memory leak
https://bugs.ruby-lang.org/issues/11174#change-74032

* Author: cvss (Cyril Vechera)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* Target version: 
* ruby -v: 2.2.3, 2.2.0, 2.1.0, 2.0.0, 1.9.3
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------
There's strong memory growth during intensive thread using. 

Script to demonstrate the problem (on x86_64-linux):

~~~
loop {
	10.times { 1000.times.map { Thread.new { } }.each(&:join) }
	GC.start # not necessary, just to be sure
	puts File.open('/proc/self/status').grep(/VmRSS:/).first
}
~~~

Running this script shows RSS growing from 45 Mb at the start time to 700 Mb after few minutes.

~~~
$ ruby thread_memleak4.rb
VmRSS:     45036 kB
VmRSS:     66748 kB
VmRSS:     87024 kB
...
VmRSS:    678052 kB
~~~

---Files--------------------------------
thread_memleak4.rb (138 Bytes)


-- 
https://bugs.ruby-lang.org/

[ruby-core:89008] Re: [Ruby trunk Bug#11174] threads memory leak

[Eric Wong]

hartator@gmail.com wrote:
> Issue #11174 has been updated by hartator (Julien Khaleghy).

> Any idea if it's already fixed somewhere else, or you guys
> still working on it?

I missed it, too :x (and now I'm too sleepy to continue...).

It seems specific to glibc (tested 2.24-11+deb9 on Debian
stable) and does not happen with jemalloc (3.6.0).

And it is NOT because of memalign fragmentation, either(*); I
disabled memalign usage to fallback to the aligned malloc+free
fallback; but can still reproduce it.

Finally, it needs multiple threads to reproduce the issue,
a single thread is not enough.


(*) https://sourceware.org/bugzilla/show_bug.cgi?id=14581

[ruby-core:89010] Re: [Ruby trunk Bug#11174] threads memory leak

[Eric Wong]

> It seems specific to glibc

That should be "glibc malloc"; and only old versions.  The bug
is fixed in glibc 2.28, at least; likely earlier.  (glibc takes
forever to build) It's present in 2.24-11+deb9 on Debian.

MALLOC_ARENA_MAX=1 MALLOC_ARENA_TEST=1 does not seem to help.

I forgot one other parameter which IS the culprit in older glibc versions:

MALLOC_MMAP_THRESHOLD_=131072  # or whatever fixed value you want

So the sliding mmap threshold was the problem in glibc; but it's
fixed upstream, at least.  Just have to wait for distros to roll
it out...

[ruby-core:93203] [Ruby trunk Bug#11174] threads memory leak

[from-ruby-lang]

Issue #11174 has been updated by cvss (Kirill Vechera).


I've just tested on a manually built glibc 2.28 with ruby ruby 2.7.0dev (2019-06-17T14:25:47Z trunk 801d0d9dd7) [x86_64-linux], the problem remains. So probably glibc 2.28 has no fix yet.

----------------------------------------
Bug #11174: threads memory leak
https://bugs.ruby-lang.org/issues/11174#change-78650

* Author: cvss (Kirill Vechera)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* Target version: 
* ruby -v: 2.2.3, 2.2.0, 2.1.0, 2.0.0, 1.9.3
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------
There's strong memory growth during intensive thread using. 

Script to demonstrate the problem (on x86_64-linux):

~~~
loop {
	10.times { 1000.times.map { Thread.new { } }.each(&:join) }
	GC.start # not necessary, just to be sure
	puts File.open('/proc/self/status').grep(/VmRSS:/).first
}
~~~

Running this script shows RSS growing from 45 Mb at the start time to 700 Mb after few minutes.

~~~
$ ruby thread_memleak4.rb
VmRSS:     45036 kB
VmRSS:     66748 kB
VmRSS:     87024 kB
...
VmRSS:    678052 kB
~~~

---Files--------------------------------
thread_memleak4.rb (138 Bytes)


-- 
https://bugs.ruby-lang.org/