* [ruby-dev:50421] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
@ 2018-01-24 0:29 ` koshigoeb
2018-01-25 23:02 ` [ruby-dev:50433] " s.wanabe
` (19 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: koshigoeb @ 2018-01-24 0:29 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been reported by koshigoe (Masataka SUZUKI).
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread* [ruby-dev:50433] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
2018-01-24 0:29 ` [ruby-dev:50421] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる koshigoeb
@ 2018-01-25 23:02 ` s.wanabe
2018-01-26 2:54 ` [ruby-dev:50435] " daniel.ltw
` (18 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: s.wanabe @ 2018-01-25 23:02 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by wanabe (_ wanabe).
koshigoe (Masataka SUZUKI) wrote:
> Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
> これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
>
> Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
> https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
`git bisect` で確認したところ、 r59630 以降に今の挙動になっているようでした。
r59630 は [Bug #13412] に関連したコミットで、差分を見ると `stack_check(th)` というところから、スレットのスタックをチェックする処理が追加されていることがわかります。
ここから推測すると、おっしゃっている通り Alpine Linux で、あるいは musl で特有に見られる問題なのではないかと思われます。
意図的な挙動か、チェック内容を緩めるべきかどうか、pthread_attr_setstacksize やその他の方法で回避できるかまたそうするべきか、といったあたりはわかりませんでした。
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-69840
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread* [ruby-dev:50435] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
2018-01-24 0:29 ` [ruby-dev:50421] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる koshigoeb
2018-01-25 23:02 ` [ruby-dev:50433] " s.wanabe
@ 2018-01-26 2:54 ` daniel.ltw
2018-01-26 3:38 ` [ruby-dev:50436] " mame
` (17 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: daniel.ltw @ 2018-01-26 2:54 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by scardon (Daniel Leong).
https://qiita.com/koshigoe/items/7acebbab7b44fa2b35bc
This is the post that spin off this issue ticket.
There are more information in there.
Hope we could get some form of resolution for Alpine users.
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-69850
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread* [ruby-dev:50436] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
` (2 preceding siblings ...)
2018-01-26 2:54 ` [ruby-dev:50435] " daniel.ltw
@ 2018-01-26 3:38 ` mame
2018-01-29 9:28 ` [ruby-dev:50444] " craig
` (16 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: mame @ 2018-01-26 3:38 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by mame (Yusuke Endoh).
環境変数 `RUBY_THREAD_MACHINE_STACK_SIZE` に 1048576 とか大きい値を設定してみたら動きますか？
これで pthread_attr_setstacksize に渡すスタックサイズを調整できます。
なお、この設定は RubyVM::DEFAULT_PARAMS で読み出すことができます。
```
$ ruby -ve 'p RubyVM::DEFAULT_PARAMS[:thread_machine_stack_size]'
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux]
1048576
$ RUBY_THREAD_MACHINE_STACK_SIZE=80000 ruby -ve 'p RubyVM::DEFAULT_PARAMS[:thread_machine_stack_size]'
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux]
131072
```
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-69851
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread* [ruby-dev:50444] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
` (3 preceding siblings ...)
2018-01-26 3:38 ` [ruby-dev:50436] " mame
@ 2018-01-29 9:28 ` craig
2018-01-29 19:31 ` [ruby-dev:50451] " daniel.ltw
` (15 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: craig @ 2018-01-29 9:28 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by craigjbass (Craig Bass).
wanabe (_ wanabe) wrote:
> koshigoe (Masataka SUZUKI) wrote:
> > Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
> > これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
> >
> > Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
> > https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
>
> `git bisect` で確認したところ、 r59630 以降に今の挙動になっているようでした。
> r59630 は [Bug #13412] に関連したコミットで、差分を見ると `stack_check(th)` というところから、スレットのスタックをチェックする処理が追加されていることがわかります。
>
> ここから推測すると、おっしゃっている通り Alpine Linux で、あるいは musl で特有に見られる問題なのではないかと思われます。
> 意図的な挙動か、チェック内容を緩めるべきかどうか、pthread_attr_setstacksize やその他の方法で回避できるかまたそうするべきか、といったあたりはわかりませんでした。
I've also come across this issue: https://github.com/rspec/rspec-support/pull/343
mame (Yusuke Endoh) wrote:
> 環境変数 `RUBY_THREAD_MACHINE_STACK_SIZE` に 1048576 とか大きい値を設定してみたら動きますか？
> これで pthread_attr_setstacksize に渡すスタックサイズを調整できます。
>
> なお、この設定は RubyVM::DEFAULT_PARAMS で読み出すことができます。
>
> ```
> $ ruby -ve 'p RubyVM::DEFAULT_PARAMS[:thread_machine_stack_size]'
> ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux]
> 1048576
>
> $ RUBY_THREAD_MACHINE_STACK_SIZE=80000 ruby -ve 'p RubyVM::DEFAULT_PARAMS[:thread_machine_stack_size]'
> ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux]
> 131072
> ```
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-69961
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread* [ruby-dev:50451] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
` (4 preceding siblings ...)
2018-01-29 9:28 ` [ruby-dev:50444] " craig
@ 2018-01-29 19:31 ` daniel.ltw
2018-01-30 3:42 ` [ruby-dev:50452] " s.wanabe
` (14 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: daniel.ltw @ 2018-01-29 19:31 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by scardon (Daniel Leong).
~~~ text
/app # ruby -ve 'p RubyVM::DEFAULT_PARAMS[:thread_machine_stack_size]'
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
1048576
/app # RUBY_THREAD_MACHINE_STACK_SIZE=100000 ruby -ve 'p RubyVM::DEFAULT_PARAMS[:thread_machine_stack_size]'
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
131072
/app # RUBY_THREAD_MACHINE_STACK_SIZE=500000 ruby -ve 'p RubyVM::DEFAULT_PARAMS[:thread_machine_stack_size]'
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
503808
/app # RUBY_THREAD_MACHINE_STACK_SIZE=100000 rubycritic -f html --no-browser app lib
RubyCritic can provide more feedback if you use a Git, Mercurial or Perforce repository. Churn will not be calculated.
Traceback (most recent call last):
165: from /usr/local/bundle/bin/rubycritic:23:in `<main>'
164: from /usr/local/bundle/bin/rubycritic:23:in `load'
163: from /usr/local/bundle/gems/rubycritic-3.3.0/bin/rubycritic:10:in `<top (required)>'
162: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/cli/application.rb:20:in `execute'
161: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/commands/default.rb:19:in `execute'
160: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/commands/default.rb:24:in `critique'
159: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/analysers_runner.rb:27:in `run'
158: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/analysers_runner.rb:27:in `each'
... 153 levels...
4: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `each'
3: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `block in mass'
2: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `mass'
1: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `inject'
/usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `each': stack level too deep (SystemStackError)
/app # RUBY_THREAD_MACHINE_STACK_SIZE=500000 rubycritic -f html --no-browser app lib
RubyCritic can provide more feedback if you use a Git, Mercurial or Perforce repository. Churn will not be calculated.
Traceback (most recent call last):
165: from /usr/local/bundle/bin/rubycritic:23:in `<main>'
164: from /usr/local/bundle/bin/rubycritic:23:in `load'
163: from /usr/local/bundle/gems/rubycritic-3.3.0/bin/rubycritic:10:in `<top (required)>'
162: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/cli/application.rb:20:in `execute'
161: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/commands/default.rb:19:in `execute'
160: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/commands/default.rb:24:in `critique'
159: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/analysers_runner.rb:27:in `run'
158: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/analysers_runner.rb:27:in `each'
... 153 levels...
4: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `each'
3: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `block in mass'
2: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `mass'
1: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `inject'
/usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `each': stack level too deep (SystemStackError)
/app # rubycritic -f html --no-browser app lib
RubyCritic can provide more feedback if you use a Git, Mercurial or Perforce repository. Churn will not be calculated.
Traceback (most recent call last):
164: from /usr/local/bundle/bin/rubycritic:23:in `<main>'
163: from /usr/local/bundle/bin/rubycritic:23:in `load'
162: from /usr/local/bundle/gems/rubycritic-3.3.0/bin/rubycritic:10:in `<top (required)>'
161: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/cli/application.rb:20:in `execute'
160: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/commands/default.rb:19:in `execute'
159: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/commands/default.rb:24:in `critique'
158: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/analysers_runner.rb:27:in `run'
157: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/analysers_runner.rb:27:in `each'
... 152 levels...
4: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `inject'
3: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `each'
2: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `block in mass'
1: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `mass'
/usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `inject': stack level too deep (SystemStackError)
~~~
This is some output with the rubycritic gem trying to parse a deeply nested AST.
Below is some output with the brakeman gem trying to parse the same deeply nested AST.
~~~ text
/app # ruby -ve 'p RubyVM::DEFAULT_PARAMS[:thread_machine_stack_size]'
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
1048576
/app # RUBY_THREAD_MACHINE_STACK_SIZE=100000 ruby -ve 'p RubyVM::DEFAULT_PARAMS[:thread_machine_stack_size]'
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
131072
/app # RUBY_THREAD_MACHINE_STACK_SIZE=500000 ruby -ve 'p RubyVM::DEFAULT_PARAMS[:thread_machine_stack_size]'
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
503808
/app # brakeman -o brakeman-output.json --no-progress --separate-models --no-branching
Loading scanner...
Processing application in /app
Processing gems...
[Notice] Detected Rails 5 application
Processing configuration...
[Notice] Escaping HTML by default
Parsing files...
Processing initializers...
Processing libs...
Traceback (most recent call last):
254: from /usr/local/bundle/bin/brakeman:23:in `<main>'
253: from /usr/local/bundle/bin/brakeman:23:in `load'
252: from /usr/local/bundle/gems/brakeman-4.1.1/bin/brakeman:8:in `<top (required)>'
251: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:20:in `start'
250: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:35:in `run'
249: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:142:in `run_report'
248: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:118:in `regular_report'
247: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:133:in `run_brakeman'
... 242 levels...
4: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
3: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
2: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
1: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
/usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash': stack level too deep (SystemStackError)
/app # RUBY_THREAD_MACHINE_STACK_SIZE=100000 brakeman -o brakeman-output.json --no-progress --separate-models --no-branching
Loading scanner...
Processing application in /app
Processing gems...
[Notice] Detected Rails 5 application
Processing configuration...
[Notice] Escaping HTML by default
Parsing files...
Processing initializers...
Processing libs...
Traceback (most recent call last):
256: from /usr/local/bundle/bin/brakeman:23:in `<main>'
255: from /usr/local/bundle/bin/brakeman:23:in `load'
254: from /usr/local/bundle/gems/brakeman-4.1.1/bin/brakeman:8:in `<top (required)>'
253: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:20:in `start'
252: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:35:in `run'
251: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:142:in `run_report'
250: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:118:in `regular_report'
249: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:133:in `run_brakeman'
... 244 levels...
4: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
3: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
2: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
1: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
/usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash': stack level too deep (SystemStackError)
/app # RUBY_THREAD_MACHINE_STACK_SIZE=500000 brakeman -o brakeman-output.json --no-progress --separate-models --no-branching
Loading scanner...
Processing application in /app
Processing gems...
[Notice] Detected Rails 5 application
Processing configuration...
[Notice] Escaping HTML by default
Parsing files...
Processing initializers...
Processing libs...
Traceback (most recent call last):
256: from /usr/local/bundle/bin/brakeman:23:in `<main>'
255: from /usr/local/bundle/bin/brakeman:23:in `load'
254: from /usr/local/bundle/gems/brakeman-4.1.1/bin/brakeman:8:in `<top (required)>'
253: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:20:in `start'
252: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:35:in `run'
251: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:142:in `run_report'
250: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:118:in `regular_report'
249: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:133:in `run_brakeman'
... 244 levels...
4: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
3: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
2: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
1: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
/usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash': stack level too deep (SystemStackError)
~~~
Both brakeman and rubycritic never would have such error when we were using ruby 2.4
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-69985
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread* [ruby-dev:50452] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
` (5 preceding siblings ...)
2018-01-29 19:31 ` [ruby-dev:50451] " daniel.ltw
@ 2018-01-30 3:42 ` s.wanabe
2018-02-03 23:07 ` [ruby-dev:50463] " james
` (13 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: s.wanabe @ 2018-01-30 3:42 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by wanabe (_ wanabe).
mame (Yusuke Endoh) wrote:
> 環境変数 `RUBY_THREAD_MACHINE_STACK_SIZE` に 1048576 とか大きい値を設定してみたら動きますか？
> これで pthread_attr_setstacksize に渡すスタックサイズを調整できます。
スレッドといってもメインスレッドしか存在せず、native_thread_create() が呼び出されるわけではないので、残念ながら動作は変化していないようです。
```
/work # ./miniruby test.rb 2>&1|grep level
test.rb:8:in `each_value': stack level too deep (SystemStackError)
... 194 levels...
/work # RUBY_THREAD_MACHINE_STACK_SIZE=1048576 ./miniruby test.rb 2>&1|grep level
test.rb:8:in `each_value': stack level too deep (SystemStackError)
... 188 levels...
```
スレッドで包んでやると期待通りの挙動が得られました。
```
/work # RUBY_THREAD_MACHINE_STACK_SIZE=1048576 ./miniruby -e 'Thread.new{ load "test.rb" }.join' 2>&1|grep level
test.rb:8:in `each_value': stack level too deep (SystemStackError)
... 1720 levels...
test.rb:8:in `each_value': stack level too deep (SystemStackError)
... 1720 levels...
```
以下のようなテストプログラムを動かすと、Alpine musl 環境では rlim_cur の値と pthread_attr_getstacksize の値が大きく異なることがわかりました。
このあたりが関係しているのではないでしょうか。
```
#include <stdio.h>
#include <sys/resource.h>
#include <pthread.h>
int main (void) {
struct rlimit limit;
pthread_attr_t attr;
size_t stacksize;
getrlimit(RLIMIT_STACK, &limit);
pthread_attr_init(&attr);
pthread_attr_getstacksize(&attr, &stacksize);
pthread_attr_destroy(&attr);
printf("rlim_cur: %ld\nrlim_max: %ld\nstacksize: %ld\n", (size_t)limit.rlim_cur, (size_t)limit.rlim_max, stacksize);
return 0;
}
```
```
/work # gcc -pthread test.c && ./a.out
rlim_cur: 8388608
rlim_max: -1
stacksize: 81920
```
なお、同じプログラムを ubuntu 18.04 上で動かした場合には両者は一致しました。
```
$ gcc -pthread test.c && ./a.out
rlim_cur: 8388608
rlim_max: -1
stacksize: 8388608
```
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-69996
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread* [ruby-dev:50463] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
` (6 preceding siblings ...)
2018-01-30 3:42 ` [ruby-dev:50452] " s.wanabe
@ 2018-02-03 23:07 ` james
2018-03-08 13:32 ` [ruby-dev:50494] " ncopa
` (12 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: james @ 2018-02-03 23:07 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by jhealy (James Healy).
There's a very similar sounding issue being discussed on the python bug tracker: https://bugs.python.org/issue32307
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-70147
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread* [ruby-dev:50494] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
` (7 preceding siblings ...)
2018-02-03 23:07 ` [ruby-dev:50463] " james
@ 2018-03-08 13:32 ` ncopa
2018-03-13 17:41 ` [ruby-dev:50497] " ncopa
` (11 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: ncopa @ 2018-03-08 13:32 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by ncopa (Natanael Copa).
jhealy (James Healy) wrote:
> There's a very similar sounding issue being discussed on the python bug tracker: https://bugs.python.org/issue32307
I first thought that this is same/similar issue as the python, but it is not. (I think ruby may have same problem as python, but that is a separate issue)
What happens here is that ruby tries to calculate the stack size using `pthread_getattr_np(pthread_self())`. This does not work when current thread (pthread_self()) is main thread. Doing stack size calculation for main thread is non-trivial, if possible at all. musl developers has chosen the safe approach here and reports only the amount of stack size that kernel can guarantee.
This testcase illustrates the problem:
~~~
#define _GNU_SOURCE
#include <stdio.h>
#include <pthread.h>
int main()
{
pthread_attr_t attr;
size_t size;
if (pthread_getattr_np(pthread_self(), &attr) == -1)
return 1;
if (pthread_attr_getstacksize(&attr, &size) == -1)
return 1;
pthread_attr_destroy(&attr);
printf("stacksize pthread_self(): %zu\n", size);
return 0;
}
~~~
When running it on alpine it returns:
stacksize pthread_self(): 126976
Which is way lower than the 8MB set by ulimit.
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-70914
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread* [ruby-dev:50497] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
` (8 preceding siblings ...)
2018-03-08 13:32 ` [ruby-dev:50494] " ncopa
@ 2018-03-13 17:41 ` ncopa
2018-03-16 12:57 ` [ruby-dev:50499] " ncopa
` (10 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: ncopa @ 2018-03-13 17:41 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by ncopa (Natanael Copa).
Possible fix or workaround:
~~~
diff --git a/thread_pthread.c b/thread_pthread.c
index 951885ffa0..e2d662143b 100644
--- a/thread_pthread.c
+++ b/thread_pthread.c
@@ -721,7 +721,7 @@ ruby_init_stack(volatile VALUE *addr
native_main_thread.register_stack_start = (VALUE*)bsp;
}
#endif
-#if MAINSTACKADDR_AVAILABLE
+#if MAINSTACKADDR_AVAILABLE && !(defined(__linux__) && !defined(__GLIBC__))
if (native_main_thread.stack_maxsize) return;
{
void* stackaddr;
@@ -1680,7 +1680,7 @@ ruby_stack_overflowed_p(const rb_thread_t *th, const void *addr)
#ifdef STACKADDR_AVAILABLE
if (get_stack(&base, &size) == 0) {
-# ifdef __APPLE__
+# if defined(__APPLE__) || (defined(__linux__) && !defined(__GLIBC__))
if (pthread_equal(th->thread_id, native_main_thread.id)) {
struct rlimit rlim;
if (getrlimit(RLIMIT_STACK, &rlim) == 0 && rlim.rlim_cur > size) {
~~~
I wonder why that second hunk which fall backs to getrlimit(RLIMIT_STACK) is needed when current thread is main thread on `__APPLE__`. It looks like the exact issue we have with musl so I would not be surprised if the same issue exists on apple too.
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-70971
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
--
https://bugs.ruby-lang.org/
^ permalink raw reply related [flat|nested] 21+ messages in thread* [ruby-dev:50499] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
` (9 preceding siblings ...)
2018-03-13 17:41 ` [ruby-dev:50497] " ncopa
@ 2018-03-16 12:57 ` ncopa
2018-04-05 17:16 ` [ruby-dev:50517] " j
` (9 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: ncopa @ 2018-03-16 12:57 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by ncopa (Natanael Copa).
this is somewhat related to this comment: https://bugs.ruby-lang.org/issues/9454#note-6
There is a problem with pthread_getattr_np() when running main thread since we get different results depending on implementation. The `_np` means non-portable so we cannot assume any specific behavior from the implementation.
The proper fix would be to test for known platform (eg `__GLIBC__`) before relying on `pthread_getattr_np` and then fall back to a portable logic for unknown platforms.
The most reliable way to calculate the stack size for main thread for Linux is to parse /proc/self/maps and this is what glibc `pthread_getattr_np` implementation does.
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-71036
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: DONTNEED, 2.4: DONTNEED, 2.5: REQUIRED
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread* [ruby-dev:50517] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
` (10 preceding siblings ...)
2018-03-16 12:57 ` [ruby-dev:50499] " ncopa
@ 2018-04-05 17:16 ` j
2018-04-06 3:14 ` [ruby-dev:50519] " naruse
` (8 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: j @ 2018-04-05 17:16 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by jottr (Jonas T).
I'd like to inquire on the progress of this issue. What is missing to get this resolved?
It is affecting downstream users who deploy their ruby applications by means of alpine linux containers. The alpine linux ruby containers are currently unusable because of this.
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-71395
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: DONTNEED, 2.4: DONTNEED, 2.5: REQUIRED
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
---Files--------------------------------
0001-thread_pthread.c-make-get_main_stack-portable-on-lin.patch (2.61 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread* [ruby-dev:50519] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
` (11 preceding siblings ...)
2018-04-05 17:16 ` [ruby-dev:50517] " j
@ 2018-04-06 3:14 ` naruse
2018-05-20 16:17 ` [ruby-dev:50549] " jnardone
` (7 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: naruse @ 2018-04-06 3:14 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by naruse (Yui NARUSE).
jottr (jottr -) wrote:
> I'd like to inquire on the progress of this issue. What is missing to get this resolved?
>
> It is affecting downstream users who deploy their ruby applications by means of alpine linux containers. The alpine linux ruby containers are currently unusable because of this.
get_stack and get_main_stack needs platform dependent implementation.
In this context "platform" means libc (or libthread or something on some OSes).
Therefore ncopa's patch looks good for me in general.
The blocker seems
* ncopa's patch breaks non linux environment because the patch removes `define get_main_stack` but not defined for other thant Linux non glibc
* minor one: it should add comment about !defined(__GLIBC__) intends musl libc and uClibc defines __GLIBC__.
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-71402
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: DONTNEED, 2.4: DONTNEED, 2.5: REQUIRED
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
---Files--------------------------------
0001-thread_pthread.c-make-get_main_stack-portable-on-lin.patch (2.61 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread* [ruby-dev:50549] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
` (12 preceding siblings ...)
2018-04-06 3:14 ` [ruby-dev:50519] " naruse
@ 2018-05-20 16:17 ` jnardone
2018-05-23 0:32 ` [ruby-dev:50550] " s.wanabe
` (6 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: jnardone @ 2018-05-20 16:17 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by jnardone (joe nardone).
it's frustrating that this is still open after four months. alpine-ruby-2.5 is borderline unusable with this still in Ruby. do we know if @ncopa is still working on this issue or not?
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-72187
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: DONTNEED, 2.4: DONTNEED, 2.5: REQUIRED
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
---Files--------------------------------
0001-thread_pthread.c-make-get_main_stack-portable-on-lin.patch (2.61 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread* [ruby-dev:50550] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
` (13 preceding siblings ...)
2018-05-20 16:17 ` [ruby-dev:50549] " jnardone
@ 2018-05-23 0:32 ` s.wanabe
2018-06-02 1:23 ` [ruby-dev:50556] " naruse
` (5 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: s.wanabe @ 2018-05-23 0:32 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by wanabe (_ wanabe).
It seems to be reasonable not to rely `pthread_getattr_np()` on `defined(__linux__) && !defined(__GLIBC__)` environment because the function has suffix ["_np"](https://bugs.ruby-lang.org/issues/14387#note-12).
I guess it would be ideal if ruby relies `pthread_getattr_np()` on only tested environments like as `defined(__linux__) && defined(__GLIBC__)`, but it is too much pain to follow / test other environments.
[ncopa's 2nd patch](https://bugs.ruby-lang.org/issues/14387#note-13) has a side effect as [commented](https://bugs.ruby-lang.org/issues/14387#note-16).
So I think [1st patch](https://bugs.ruby-lang.org/issues/14387#note-10) is more pragmatic.
Naruse-san (or other?):
How about https://bugs.ruby-lang.org/issues/14387#note-10 patch?
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-72218
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: DONTNEED, 2.4: DONTNEED, 2.5: REQUIRED
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
---Files--------------------------------
0001-thread_pthread.c-make-get_main_stack-portable-on-lin.patch (2.61 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread* [ruby-dev:50556] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
` (14 preceding siblings ...)
2018-05-23 0:32 ` [ruby-dev:50550] " s.wanabe
@ 2018-06-02 1:23 ` naruse
2018-06-10 10:49 ` [ruby-dev:50564] " s.wanabe
` (4 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: naruse @ 2018-06-02 1:23 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by naruse (Yui NARUSE).
jnardone (joe nardone) wrote:
> it's frustrating that this is still open after four months. alpine-ruby-2.5 is borderline unusable with this still in Ruby. do we know if @ncopa is still working on this issue or not?
I wrote 0001-thread_pthread.c-make-get_main_stack-portable-on-lin.patch is not acceptable.
wanabe (_ wanabe) wrote:
> It seems to be reasonable not to rely `pthread_getattr_np()` on `defined(__linux__) && !defined(__GLIBC__)` environment because the function has suffix ["_np"](https://bugs.ruby-lang.org/issues/14387#note-12).
> I guess it would be ideal if ruby relies `pthread_getattr_np()` on only tested environments like as `defined(__linux__) && defined(__GLIBC__)`, but it is too much pain to follow / test other environments.
Use `pthread_getattr_np()` only on musl libc environment.
This stack related code is highly platform dependent.
You can use non portable functions for non portable codes.
You can see many `ifdef`s in get_stack.
Don't try to write portable code without `ifdef` in this area; "portable" means write `ifdef`s for all tested environments.
> [ncopa's 2nd patch](https://bugs.ruby-lang.org/issues/14387#note-13) has a side effect as [commented](https://bugs.ruby-lang.org/issues/14387#note-16).
> So I think [1st patch](https://bugs.ruby-lang.org/issues/14387#note-10) is more pragmatic.
>
> Naruse-san (or other?):
> How about https://bugs.ruby-lang.org/issues/14387#note-10 patch?
It's acceptable because it doesn't break other tested environment, but `getrlimit` really works on musl libc environment?
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-72344
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: DONTNEED, 2.4: DONTNEED, 2.5: REQUIRED
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
---Files--------------------------------
0001-thread_pthread.c-make-get_main_stack-portable-on-lin.patch (2.61 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread* [ruby-dev:50564] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
` (15 preceding siblings ...)
2018-06-02 1:23 ` [ruby-dev:50556] " naruse
@ 2018-06-10 10:49 ` s.wanabe
2018-06-11 12:22 ` [ruby-dev:50565] " ncopa
` (3 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: s.wanabe @ 2018-06-10 10:49 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by wanabe (_ wanabe).
naruse (Yui NARUSE) wrote:
> It's acceptable because it doesn't break other tested environment, but `getrlimit` really works on musl libc environment?
Thank you for your comment.
Okay, The patch needs one or more proofs of its behaviour, like that:
* Original issue [ruby-dev:50421] has gone away.
* Standard test codes run well.
* test-all
* ruby/spec
* `getrlimit` works on some situations like:
* on single thread
* with multiple threads
* with `RLIMIT_STACK` environment variable
* `getrlimit` code of musl is implemented correctly as expected.
* (But It's doubtful whether it can be. I guess that a proof of code soundness is very difficult.)
* Some "real world" applications can work.
* I think it is better example that that application(s) can't work without the patch.
I can't prove because I am not musl / Alpine guy.
Anyone can help?
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-72459
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: DONTNEED, 2.4: DONTNEED, 2.5: REQUIRED
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
---Files--------------------------------
0001-thread_pthread.c-make-get_main_stack-portable-on-lin.patch (2.61 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread* [ruby-dev:50565] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
` (16 preceding siblings ...)
2018-06-10 10:49 ` [ruby-dev:50564] " s.wanabe
@ 2018-06-11 12:22 ` ncopa
2018-06-11 12:43 ` [ruby-dev:50566] " ncopa
` (2 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: ncopa @ 2018-06-11 12:22 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by ncopa (Natanael Copa).
naruse (Yui NARUSE) wrote:
> jottr (jottr -) wrote:
> > I'd like to inquire on the progress of this issue. What is missing to get this resolved?
> >
> > It is affecting downstream users who deploy their ruby applications by means of alpine linux containers. The alpine linux ruby containers are currently unusable because of this.
>
> get_stack and get_main_stack needs platform dependent implementation.
> In this context "platform" means libc (or libthread or something on some OSes).
> Therefore ncopa's patch looks good for me in general.
>
> The blocker seems
> * ncopa's patch breaks non linux environment because the patch removes `define get_main_stack` but not defined for other thant Linux non glibc
No. `get_main_stack` gets defined in the #else block for everything that is not Linux or not glibc:
````
#if defined(__linux__) && !defined(__GLIBC__) && defined(HAVE_GETRLIMIT)
...
#else
# define get_main_stack(addr, size) get_stack(addr, size)
#endif
````
I can turn it around the other for better readability if you want:
````
#if !defined(__linux__) || defined(__GLIBC__) || !defined(HAVE_GETRLIMIT)
# define get_main_stack(addr, size) get_stack(addr, size)
#else
# ...
#endif
````
> * minor one: it should add comment about !defined(__GLIBC__) intends musl libc and uClibc defines __GLIBC__.
Will add comment about musl libc. I think uClibc behavior is irrelevant here.
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-72465
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: DONTNEED, 2.4: DONTNEED, 2.5: REQUIRED
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
---Files--------------------------------
0001-thread_pthread.c-make-get_main_stack-portable-on-lin.patch (2.61 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread* [ruby-dev:50566] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
` (17 preceding siblings ...)
2018-06-11 12:22 ` [ruby-dev:50565] " ncopa
@ 2018-06-11 12:43 ` ncopa
2018-06-11 13:44 ` [ruby-dev:50567] " ncopa
2019-12-15 3:21 ` [ruby-dev:50886] [Ruby master " h.shimoyama
20 siblings, 0 replies; 21+ messages in thread
From: ncopa @ 2018-06-11 12:43 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by ncopa (Natanael Copa).
wanabe (_ wanabe) wrote:
> It's acceptable because it doesn't break other tested environment, but `getrlimit` really works on musl libc environment?
`getrlimit` works on musl libc, as defined by POSIX. This is a syscall so there is no reason why it should not work.
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-72466
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: DONTNEED, 2.4: DONTNEED, 2.5: REQUIRED
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
---Files--------------------------------
0001-thread_pthread.c-make-get_main_stack-portable-on-lin.patch (2.61 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread* [ruby-dev:50567] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
` (18 preceding siblings ...)
2018-06-11 12:43 ` [ruby-dev:50566] " ncopa
@ 2018-06-11 13:44 ` ncopa
2019-12-15 3:21 ` [ruby-dev:50886] [Ruby master " h.shimoyama
20 siblings, 0 replies; 21+ messages in thread
From: ncopa @ 2018-06-11 13:44 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by ncopa (Natanael Copa).
wanabe (_ wanabe) wrote:
> It seems to be reasonable not to rely `pthread_getattr_np()` on `defined(__linux__) && !defined(__GLIBC__)` environment because the function has suffix ["_np"](https://bugs.ruby-lang.org/issues/14387#note-12).
> I guess it would be ideal if ruby relies `pthread_getattr_np()` on only tested environments like as `defined(__linux__) && defined(__GLIBC__)`, but it is too much pain to follow / test other environments.
I agree that `defined(__linux__) && defined(__GLIBC__)` is better. Note that there are some non-linux glibc variants too but I would expect the `pthread_getattr_np()` call work as documented in the GNU libc manual, regarless if kernel is linux, hurd or anything else, so it should be enough with `defined(__GLIBC__)`.
> [ncopa's 2nd patch](https://bugs.ruby-lang.org/issues/14387#note-13) has a side effect as [commented](https://bugs.ruby-lang.org/issues/14387#note-16).
> So I think [1st patch](https://bugs.ruby-lang.org/issues/14387#note-10) is more pragmatic.
>
> Naruse-san (or other?):
> How about https://bugs.ruby-lang.org/issues/14387#note-10 patch?
That patch is a workaround only and not a fix. It will cause the code to skip the `reserve_stack` call so behavior will differ from Linux systems with glibc. The second patch fixes the issue properly, even if it can be disputed if the `reserve_stack` is a good idea in the first place.
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-72467
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: DONTNEED, 2.4: DONTNEED, 2.5: REQUIRED
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
---Files--------------------------------
0001-thread_pthread.c-make-get_main_stack-portable-on-lin.patch (2.61 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread* [ruby-dev:50886] [Ruby master Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
[not found] <redmine.issue-14387.20180124002916@ruby-lang.org>
` (19 preceding siblings ...)
2018-06-11 13:44 ` [ruby-dev:50567] " ncopa
@ 2019-12-15 3:21 ` h.shimoyama
20 siblings, 0 replies; 21+ messages in thread
From: h.shimoyama @ 2019-12-15 3:21 UTC (permalink / raw)
To: ruby-dev
Issue #14387 has been updated by hshimoyama (Hiroyasu Shimoyama).
I tried to reproduce this issue, but it was not reproduced today.
Is this issu fixed?
```
$ docker-compose build; docker-compose up
Building web
Step 1/2 : FROM ruby:2.5-alpine
---> 5ccd5d2d442a
Step 2/2 : COPY ./test.rb .
---> Using cache
---> d83d0e042967
Successfully built d83d0e042967
Successfully tagged stack_test:latest
ruby_alpine_stack_web_1 is up-to-date
Attaching to ruby_alpine_stack_web_1
web_1 | test.rb:8:in `each_value': stack level too deep (SystemStackError)
web_1 | from test.rb:8:in `f'
web_1 | from test.rb:8:in `block in f'
web_1 | from test.rb:8:in `each_value'
web_1 | from test.rb:8:in `f'
web_1 | from test.rb:8:in `block in f'
web_1 | from test.rb:8:in `each_value'
web_1 | from test.rb:8:in `f'
web_1 | from test.rb:8:in `block in f'
web_1 | ... 10184 levels...
web_1 | from test.rb:8:in `block in f'
web_1 | from test.rb:8:in `each_value'
web_1 | from test.rb:8:in `f'
web_1 | from test.rb:11:in `<main>'
web_1 | test.rb:8:in `each_value': stack level too deep (SystemStackError)
```
ruby:2.6-alpine
```
$ docker-compose build; docker-compose up
Building web
Step 1/2 : FROM ruby:2.6-alpine
---> 3304101ccbe9
Step 2/2 : COPY ./test.rb .
---> Using cache
---> 3c476c28a73e
Successfully built 3c476c28a73e
Successfully tagged stack_test:latest
Recreating ruby_alpine_stack_web_1 ... done
Attaching to ruby_alpine_stack_web_1
web_1 | test.rb:8:in `block in f': stack level too deep (SystemStackError)
web_1 | from test.rb:8:in `each_value'
web_1 | from test.rb:8:in `f'
web_1 | from test.rb:8:in `block in f'
web_1 | from test.rb:8:in `each_value'
web_1 | from test.rb:8:in `f'
web_1 | from test.rb:8:in `block in f'
web_1 | from test.rb:8:in `each_value'
web_1 | from test.rb:8:in `f'
web_1 | ... 11550 levels...
web_1 | from test.rb:8:in `block in f'
web_1 | from test.rb:8:in `each_value'
web_1 | from test.rb:8:in `f'
web_1 | from test.rb:11:in `<main>'
```
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-83132
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: DONTNEED, 2.4: DONTNEED, 2.5: REQUIRED
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
---Files--------------------------------
0001-thread_pthread.c-make-get_main_stack-portable-on-lin.patch (2.61 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 21+ messages in thread