ruby-core@ruby-lang.org archive (unofficial mirror)
 help / color / mirror / Atom feed
* [ruby-core:101598] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor
@ 2020-12-21 18:31 eregontp
  2020-12-27 17:30 ` [ruby-core:101752] " marcandre-ruby-core
                   ` (8 more replies)
  0 siblings, 9 replies; 10+ messages in thread
From: eregontp @ 2020-12-21 18:31 UTC (permalink / raw)
  To: ruby-core

Issue #17420 has been reported by Eregon (Benoit Daloze).

----------------------------------------
Bug #17420: Unsafe mutation of $" when doing non-RubyGems require in Ractor
https://bugs.ruby-lang.org/issues/17420

* Author: Eregon (Benoit Daloze)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* ruby -v: ruby 3.0.0dev (2020-12-16T10:12:48Z master a9a7f4d8b8) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
With an empty file `a.rb`:
```
$ ruby --disable-gems -e 'Ractor.new { puts $" }.take'
-e:1:in `block in <main>': can not access global variables $" from non-main Ractors (RuntimeError)```
That is expected, given the rules for global variables.

```
ruby --disable-gems -e 'Ractor.new { require "./a.rb"; }.take; p $"'
[... , "/home/eregon/a.rb"]
```
Is it OK that the Ractor can do `require`, which does modify `$"`?

I think it's not, and it might lead to segfaults if e.g. the main Ractor mutates `$"` in parallel to some other Ractor doing `require`.

Probably `require` needs to be forbidden in non-main Ractors (it does mutate `$"`, so it's logical), or there needs to be always VM-global synchronization on any access to `$"` (otherwise, segfaults are possible).
The latter doesn't seem reasonable, especially when considering the user might do `$".each { ... }`.

---

Note that RubyGems' `require` does not work on non-main Ractors (pretty much expected given it depends on a lot of global state):
```
$ ruby -e 'Ractor.new { require "./a.rb"; }.take'
<internal:/home/eregon/prefix/ruby-master/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:37:in `require': can not access non-shareable objects in constant Kernel::RUBYGEMS_ACTIVATION_MONITOR by non-main ractor. (NameError)
```

This probably also has consequences for `autoload`.
Maybe the `zeitwerk` gem can help with the mode to resolve all autoload at once.



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [ruby-core:101752] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor
  2020-12-21 18:31 [ruby-core:101598] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor eregontp
@ 2020-12-27 17:30 ` marcandre-ruby-core
  2020-12-28  8:43 ` [ruby-core:101770] " duerst
                   ` (7 subsequent siblings)
  8 siblings, 0 replies; 10+ messages in thread
From: marcandre-ruby-core @ 2020-12-27 17:30 UTC (permalink / raw)
  To: ruby-core

Issue #17420 has been updated by marcandre (Marc-Andre Lafortune).


Maybe a solution would be to do all `require` in the main Ractor?

Something like whenever a new Ractor is created, the main thread could spin a private thread to do the needed `require`. When a `require` call happens, or a constant needs to be autoloaded, from within a Ractor other than main, inter-Ractor communication could be used (special tag to insure proper communication) to do the loading in the main thread.

----------------------------------------
Bug #17420: Unsafe mutation of $" when doing non-RubyGems require in Ractor
https://bugs.ruby-lang.org/issues/17420#change-89579

* Author: Eregon (Benoit Daloze)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* ruby -v: ruby 3.0.0dev (2020-12-16T10:12:48Z master a9a7f4d8b8) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
With an empty file `a.rb`:
```
$ ruby --disable-gems -e 'Ractor.new { puts $" }.take'
-e:1:in `block in <main>': can not access global variables $" from non-main Ractors (RuntimeError)
```
That is expected, given the rules for global variables.

```
ruby --disable-gems -e 'Ractor.new { require "./a.rb"; }.take; p $"'
[... , "/home/eregon/a.rb"]
```
Is it OK that the Ractor can do `require`, which does modify `$"`?

I think it's not, and it might lead to segfaults if e.g. the main Ractor mutates `$"` in parallel to some other Ractor doing `require`.

Probably `require` needs to be forbidden in non-main Ractors (it does mutate `$"`, so it's logical), or there needs to be always VM-global synchronization on any access to `$"` (otherwise, segfaults are possible).
The latter doesn't seem reasonable, especially when considering the user might do `$".each { ... }`.

---

Note that RubyGems' `require` does not work on non-main Ractors (pretty much expected given it depends on a lot of global state):
```
$ ruby -e 'Ractor.new { require "./a.rb"; }.take'
<internal:/home/eregon/prefix/ruby-master/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:37:in `require': can not access non-shareable objects in constant Kernel::RUBYGEMS_ACTIVATION_MONITOR by non-main ractor. (NameError)
```

This probably also has consequences for `autoload`.
Maybe the `zeitwerk` gem can help with the mode to resolve all autoload at once.



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [ruby-core:101770] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor
  2020-12-21 18:31 [ruby-core:101598] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor eregontp
  2020-12-27 17:30 ` [ruby-core:101752] " marcandre-ruby-core
@ 2020-12-28  8:43 ` duerst
  2020-12-28 11:58 ` [ruby-core:101775] " eregontp
                   ` (6 subsequent siblings)
  8 siblings, 0 replies; 10+ messages in thread
From: duerst @ 2020-12-28  8:43 UTC (permalink / raw)
  To: ruby-core

Issue #17420 has been updated by duerst (Martin Dürst).


I agree that it would be best to make require work everywhere, but always be executed in the main Reactor. That would just be part of the semantics of require (comment moved from #17477).

----------------------------------------
Bug #17420: Unsafe mutation of $" when doing non-RubyGems require in Ractor
https://bugs.ruby-lang.org/issues/17420#change-89598

* Author: Eregon (Benoit Daloze)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* ruby -v: ruby 3.0.0dev (2020-12-16T10:12:48Z master a9a7f4d8b8) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
With an empty file `a.rb`:
```
$ ruby --disable-gems -e 'Ractor.new { puts $" }.take'
-e:1:in `block in <main>': can not access global variables $" from non-main Ractors (RuntimeError)
```
That is expected, given the rules for global variables.

```
ruby --disable-gems -e 'Ractor.new { require "./a.rb"; }.take; p $"'
[... , "/home/eregon/a.rb"]
```
Is it OK that the Ractor can do `require`, which does modify `$"`?

I think it's not, and it might lead to segfaults if e.g. the main Ractor mutates `$"` in parallel to some other Ractor doing `require`.

Probably `require` needs to be forbidden in non-main Ractors (it does mutate `$"`, so it's logical), or there needs to be always VM-global synchronization on any access to `$"` (otherwise, segfaults are possible).
The latter doesn't seem reasonable, especially when considering the user might do `$".each { ... }`.

---

Note that RubyGems' `require` does not work on non-main Ractors (pretty much expected given it depends on a lot of global state):
```
$ ruby -e 'Ractor.new { require "./a.rb"; }.take'
<internal:/home/eregon/prefix/ruby-master/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:37:in `require': can not access non-shareable objects in constant Kernel::RUBYGEMS_ACTIVATION_MONITOR by non-main ractor. (NameError)
```

This probably also has consequences for `autoload`.
Maybe the `zeitwerk` gem can help with the mode to resolve all autoload at once.



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [ruby-core:101775] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor
  2020-12-21 18:31 [ruby-core:101598] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor eregontp
  2020-12-27 17:30 ` [ruby-core:101752] " marcandre-ruby-core
  2020-12-28  8:43 ` [ruby-core:101770] " duerst
@ 2020-12-28 11:58 ` eregontp
  2020-12-28 21:27 ` [ruby-core:101798] " shatrov
                   ` (5 subsequent siblings)
  8 siblings, 0 replies; 10+ messages in thread
From: eregontp @ 2020-12-28 11:58 UTC (permalink / raw)
  To: ruby-core

Issue #17420 has been updated by Eregon (Benoit Daloze).


That sounds like a good way to fix it.

marcandre (Marc-Andre Lafortune) wrote in #note-3:
> the main thread could spin a private thread to do the needed `require`.

Whatever code is loaded by require can check `Thread.current`, so it should be a regular Ruby Thread, except the VM or the Ractor impl would create it.


----------------------------------------
Bug #17420: Unsafe mutation of $" when doing non-RubyGems require in Ractor
https://bugs.ruby-lang.org/issues/17420#change-89603

* Author: Eregon (Benoit Daloze)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* ruby -v: ruby 3.0.0dev (2020-12-16T10:12:48Z master a9a7f4d8b8) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
With an empty file `a.rb`:
```
$ ruby --disable-gems -e 'Ractor.new { puts $" }.take'
-e:1:in `block in <main>': can not access global variables $" from non-main Ractors (RuntimeError)
```
That is expected, given the rules for global variables.

```
ruby --disable-gems -e 'Ractor.new { require "./a.rb"; }.take; p $"'
[... , "/home/eregon/a.rb"]
```
Is it OK that the Ractor can do `require`, which does modify `$"`?

I think it's not, and it might lead to segfaults if e.g. the main Ractor mutates `$"` in parallel to some other Ractor doing `require`.

Probably `require` needs to be forbidden in non-main Ractors (it does mutate `$"`, so it's logical), or there needs to be always VM-global synchronization on any access to `$"` (otherwise, segfaults are possible).
The latter doesn't seem reasonable, especially when considering the user might do `$".each { ... }`.

---

Note that RubyGems' `require` does not work on non-main Ractors (pretty much expected given it depends on a lot of global state):
```
$ ruby -e 'Ractor.new { require "./a.rb"; }.take'
<internal:/home/eregon/prefix/ruby-master/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:37:in `require': can not access non-shareable objects in constant Kernel::RUBYGEMS_ACTIVATION_MONITOR by non-main ractor. (NameError)
```

This probably also has consequences for `autoload`.
Maybe the `zeitwerk` gem can help with the mode to resolve all autoload at once.



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [ruby-core:101798] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor
  2020-12-21 18:31 [ruby-core:101598] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor eregontp
                   ` (2 preceding siblings ...)
  2020-12-28 11:58 ` [ruby-core:101775] " eregontp
@ 2020-12-28 21:27 ` shatrov
  2021-01-05  1:39 ` [ruby-core:101920] " ko1
                   ` (4 subsequent siblings)
  8 siblings, 0 replies; 10+ messages in thread
From: shatrov @ 2020-12-28 21:27 UTC (permalink / raw)
  To: ruby-core

Issue #17420 has been updated by kirs (Kir Shatrov).


(coming to this bug from https://bugs.ruby-lang.org/issues/17477)

I think it's fine to force `require` to be called only from the main thread/ractor, at least for now. I can imagine that would simplify a lot of things.

I'd like to make that error very clear to developers, and not `can not access non-shareable objects in constant Kernel::RUBYGEMS_ACTIVATION_MONITOR by non-main ractor` that they see right now.

----------------------------------------
Bug #17420: Unsafe mutation of $" when doing non-RubyGems require in Ractor
https://bugs.ruby-lang.org/issues/17420#change-89623

* Author: Eregon (Benoit Daloze)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* ruby -v: ruby 3.0.0dev (2020-12-16T10:12:48Z master a9a7f4d8b8) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
With an empty file `a.rb`:
```
$ ruby --disable-gems -e 'Ractor.new { puts $" }.take'
-e:1:in `block in <main>': can not access global variables $" from non-main Ractors (RuntimeError)
```
That is expected, given the rules for global variables.

```
ruby --disable-gems -e 'Ractor.new { require "./a.rb"; }.take; p $"'
[... , "/home/eregon/a.rb"]
```
Is it OK that the Ractor can do `require`, which does modify `$"`?

I think it's not, and it might lead to segfaults if e.g. the main Ractor mutates `$"` in parallel to some other Ractor doing `require`.

Probably `require` needs to be forbidden in non-main Ractors (it does mutate `$"`, so it's logical), or there needs to be always VM-global synchronization on any access to `$"` (otherwise, segfaults are possible).
The latter doesn't seem reasonable, especially when considering the user might do `$".each { ... }`.

---

Note that RubyGems' `require` does not work on non-main Ractors (pretty much expected given it depends on a lot of global state):
```
$ ruby -e 'Ractor.new { require "./a.rb"; }.take'
<internal:/home/eregon/prefix/ruby-master/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:37:in `require': can not access non-shareable objects in constant Kernel::RUBYGEMS_ACTIVATION_MONITOR by non-main ractor. (NameError)
```

This probably also has consequences for `autoload`.
Maybe the `zeitwerk` gem can help with the mode to resolve all autoload at once.



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [ruby-core:101920] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor
  2020-12-21 18:31 [ruby-core:101598] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor eregontp
                   ` (3 preceding siblings ...)
  2020-12-28 21:27 ` [ruby-core:101798] " shatrov
@ 2021-01-05  1:39 ` ko1
  2021-01-05  2:16 ` [ruby-core:101922] " marcandre-ruby-core
                   ` (3 subsequent siblings)
  8 siblings, 0 replies; 10+ messages in thread
From: ko1 @ 2021-01-05  1:39 UTC (permalink / raw)
  To: ruby-core

Issue #17420 has been updated by ko1 (Koichi Sasada).


Current `require` behavior with ractors are not well considered and we need to consider it.

> Maybe a solution would be to do all require in the main Ractor?

Yes, This is one good option. But not sure it is only one solution...

If we allow to require from non-main ractors, the only problem is `$LOADED_FEATURES`?

----------------------------------------
Bug #17420: Unsafe mutation of $" when doing non-RubyGems require in Ractor
https://bugs.ruby-lang.org/issues/17420#change-89766

* Author: Eregon (Benoit Daloze)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* ruby -v: ruby 3.0.0dev (2020-12-16T10:12:48Z master a9a7f4d8b8) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
With an empty file `a.rb`:
```
$ ruby --disable-gems -e 'Ractor.new { puts $" }.take'
-e:1:in `block in <main>': can not access global variables $" from non-main Ractors (RuntimeError)
```
That is expected, given the rules for global variables.

```
ruby --disable-gems -e 'Ractor.new { require "./a.rb"; }.take; p $"'
[... , "/home/eregon/a.rb"]
```
Is it OK that the Ractor can do `require`, which does modify `$"`?

I think it's not, and it might lead to segfaults if e.g. the main Ractor mutates `$"` in parallel to some other Ractor doing `require`.

Probably `require` needs to be forbidden in non-main Ractors (it does mutate `$"`, so it's logical), or there needs to be always VM-global synchronization on any access to `$"` (otherwise, segfaults are possible).
The latter doesn't seem reasonable, especially when considering the user might do `$".each { ... }`.

---

Note that RubyGems' `require` does not work on non-main Ractors (pretty much expected given it depends on a lot of global state):
```
$ ruby -e 'Ractor.new { require "./a.rb"; }.take'
<internal:/home/eregon/prefix/ruby-master/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:37:in `require': can not access non-shareable objects in constant Kernel::RUBYGEMS_ACTIVATION_MONITOR by non-main ractor. (NameError)
```

This probably also has consequences for `autoload`.
Maybe the `zeitwerk` gem can help with the mode to resolve all autoload at once.



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [ruby-core:101922] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor
  2020-12-21 18:31 [ruby-core:101598] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor eregontp
                   ` (4 preceding siblings ...)
  2021-01-05  1:39 ` [ruby-core:101920] " ko1
@ 2021-01-05  2:16 ` marcandre-ruby-core
  2021-01-07 13:22 ` [ruby-core:101972] " eregontp
                   ` (2 subsequent siblings)
  8 siblings, 0 replies; 10+ messages in thread
From: marcandre-ruby-core @ 2021-01-05  2:16 UTC (permalink / raw)
  To: ruby-core

Issue #17420 has been updated by marcandre (Marc-Andre Lafortune).


ko1 (Koichi Sasada) wrote in #note-7:
> If we allow to require from non-main ractors, the only problem is `$LOADED_FEATURES`?

There are some limitations in non-main ractors, in particular no non-shareable constants, so if a gem is loaded from non-main ractor, something as trivial as `VERSION = '0.0.1'` in its code (without frozen string magic comment) could make the loading fail. In particular with `autoload`, the requiring might happen at difficult to predict times, so this behavior might happen only intermitently...


----------------------------------------
Bug #17420: Unsafe mutation of $" when doing non-RubyGems require in Ractor
https://bugs.ruby-lang.org/issues/17420#change-89768

* Author: Eregon (Benoit Daloze)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* ruby -v: ruby 3.0.0dev (2020-12-16T10:12:48Z master a9a7f4d8b8) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
With an empty file `a.rb`:
```
$ ruby --disable-gems -e 'Ractor.new { puts $" }.take'
-e:1:in `block in <main>': can not access global variables $" from non-main Ractors (RuntimeError)
```
That is expected, given the rules for global variables.

```
ruby --disable-gems -e 'Ractor.new { require "./a.rb"; }.take; p $"'
[... , "/home/eregon/a.rb"]
```
Is it OK that the Ractor can do `require`, which does modify `$"`?

I think it's not, and it might lead to segfaults if e.g. the main Ractor mutates `$"` in parallel to some other Ractor doing `require`.

Probably `require` needs to be forbidden in non-main Ractors (it does mutate `$"`, so it's logical), or there needs to be always VM-global synchronization on any access to `$"` (otherwise, segfaults are possible).
The latter doesn't seem reasonable, especially when considering the user might do `$".each { ... }`.

---

Note that RubyGems' `require` does not work on non-main Ractors (pretty much expected given it depends on a lot of global state):
```
$ ruby -e 'Ractor.new { require "./a.rb"; }.take'
<internal:/home/eregon/prefix/ruby-master/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:37:in `require': can not access non-shareable objects in constant Kernel::RUBYGEMS_ACTIVATION_MONITOR by non-main ractor. (NameError)
```

This probably also has consequences for `autoload`.
Maybe the `zeitwerk` gem can help with the mode to resolve all autoload at once.



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [ruby-core:101972] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor
  2020-12-21 18:31 [ruby-core:101598] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor eregontp
                   ` (5 preceding siblings ...)
  2021-01-05  2:16 ` [ruby-core:101922] " marcandre-ruby-core
@ 2021-01-07 13:22 ` eregontp
  2021-01-07 13:23 ` [ruby-core:101973] " eregontp
  2023-01-16 13:46 ` [ruby-core:111834] " Eregon (Benoit Daloze) via ruby-core
  8 siblings, 0 replies; 10+ messages in thread
From: eregontp @ 2021-01-07 13:22 UTC (permalink / raw)
  To: ruby-core

Issue #17420 has been updated by Eregon (Benoit Daloze).


ko1 (Koichi Sasada) wrote in #note-7:
> If we allow to require from non-main ractors, the only problem is `$LOADED_FEATURES`?

Not only, as we see above RubyGems uses a Monitor, and that doesn't work with Ractor.
Making the entire logic for RubyGems' #require safe is probably not easy.

And as Marc-André says, any constant assignment with something not Ractor-shareable would fail, so basically almost every gem would fail to load in a Ractor.

So I think loading code in the main Ractor is the only realistic solution here.
Either by having a clear error if `require` is used on non-main Ractor, or delegate the require to the main Ractor main Thread.
One issue though is such a "delegation" really means interrupting the main Thread potentially at a random place (via interrupt checks I guess), which can be expensive (it might cancel no GVL code, which might need to redo a lot of work), and arbitrarily delayed (e.g., if the main Thread is in native code).

----------------------------------------
Bug #17420: Unsafe mutation of $" when doing non-RubyGems require in Ractor
https://bugs.ruby-lang.org/issues/17420#change-89824

* Author: Eregon (Benoit Daloze)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* ruby -v: ruby 3.0.0dev (2020-12-16T10:12:48Z master a9a7f4d8b8) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
With an empty file `a.rb`:
```
$ ruby --disable-gems -e 'Ractor.new { puts $" }.take'
-e:1:in `block in <main>': can not access global variables $" from non-main Ractors (RuntimeError)
```
That is expected, given the rules for global variables.

```
ruby --disable-gems -e 'Ractor.new { require "./a.rb"; }.take; p $"'
[... , "/home/eregon/a.rb"]
```
Is it OK that the Ractor can do `require`, which does modify `$"`?

I think it's not, and it might lead to segfaults if e.g. the main Ractor mutates `$"` in parallel to some other Ractor doing `require`.

Probably `require` needs to be forbidden in non-main Ractors (it does mutate `$"`, so it's logical), or there needs to be always VM-global synchronization on any access to `$"` (otherwise, segfaults are possible).
The latter doesn't seem reasonable, especially when considering the user might do `$".each { ... }`.

---

Note that RubyGems' `require` does not work on non-main Ractors (pretty much expected given it depends on a lot of global state):
```
$ ruby -e 'Ractor.new { require "./a.rb"; }.take'
<internal:/home/eregon/prefix/ruby-master/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:37:in `require': can not access non-shareable objects in constant Kernel::RUBYGEMS_ACTIVATION_MONITOR by non-main ractor. (NameError)
```

This probably also has consequences for `autoload`.
Maybe the `zeitwerk` gem can help with the mode to resolve all autoload at once.



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [ruby-core:101973] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor
  2020-12-21 18:31 [ruby-core:101598] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor eregontp
                   ` (6 preceding siblings ...)
  2021-01-07 13:22 ` [ruby-core:101972] " eregontp
@ 2021-01-07 13:23 ` eregontp
  2023-01-16 13:46 ` [ruby-core:111834] " Eregon (Benoit Daloze) via ruby-core
  8 siblings, 0 replies; 10+ messages in thread
From: eregontp @ 2021-01-07 13:23 UTC (permalink / raw)
  To: ruby-core

Issue #17420 has been updated by Eregon (Benoit Daloze).


Actually having an extra Thread in the main Ractor seems a much nicer solution, as proposed above (I forgot about it).

----------------------------------------
Bug #17420: Unsafe mutation of $" when doing non-RubyGems require in Ractor
https://bugs.ruby-lang.org/issues/17420#change-89825

* Author: Eregon (Benoit Daloze)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* ruby -v: ruby 3.0.0dev (2020-12-16T10:12:48Z master a9a7f4d8b8) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
With an empty file `a.rb`:
```
$ ruby --disable-gems -e 'Ractor.new { puts $" }.take'
-e:1:in `block in <main>': can not access global variables $" from non-main Ractors (RuntimeError)
```
That is expected, given the rules for global variables.

```
ruby --disable-gems -e 'Ractor.new { require "./a.rb"; }.take; p $"'
[... , "/home/eregon/a.rb"]
```
Is it OK that the Ractor can do `require`, which does modify `$"`?

I think it's not, and it might lead to segfaults if e.g. the main Ractor mutates `$"` in parallel to some other Ractor doing `require`.

Probably `require` needs to be forbidden in non-main Ractors (it does mutate `$"`, so it's logical), or there needs to be always VM-global synchronization on any access to `$"` (otherwise, segfaults are possible).
The latter doesn't seem reasonable, especially when considering the user might do `$".each { ... }`.

---

Note that RubyGems' `require` does not work on non-main Ractors (pretty much expected given it depends on a lot of global state):
```
$ ruby -e 'Ractor.new { require "./a.rb"; }.take'
<internal:/home/eregon/prefix/ruby-master/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:37:in `require': can not access non-shareable objects in constant Kernel::RUBYGEMS_ACTIVATION_MONITOR by non-main ractor. (NameError)
```

This probably also has consequences for `autoload`.
Maybe the `zeitwerk` gem can help with the mode to resolve all autoload at once.



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [ruby-core:111834] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor
  2020-12-21 18:31 [ruby-core:101598] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor eregontp
                   ` (7 preceding siblings ...)
  2021-01-07 13:23 ` [ruby-core:101973] " eregontp
@ 2023-01-16 13:46 ` Eregon (Benoit Daloze) via ruby-core
  8 siblings, 0 replies; 10+ messages in thread
From: Eregon (Benoit Daloze) via ruby-core @ 2023-01-16 13:46 UTC (permalink / raw)
  To: ruby-core; +Cc: Eregon (Benoit Daloze)

Issue #17420 has been updated by Eregon (Benoit Daloze).


@ko1 I believe this needs to be an exception until there is a better solution.
It's likely possible to create a segfault with concurrent mutations of `$"` currently.
And anyway `require` does not work in non-main Ractors, not just `$"` but many other problems (RubyGems' Monitor, any non-shareable constant accessed during loading in that Ractor, etc).

----------------------------------------
Bug #17420: Unsafe mutation of $" when doing non-RubyGems require in Ractor
https://bugs.ruby-lang.org/issues/17420#change-101245

* Author: Eregon (Benoit Daloze)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* ruby -v: ruby 3.0.0dev (2020-12-16T10:12:48Z master a9a7f4d8b8) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
With an empty file `a.rb`:
```
$ ruby --disable-gems -e 'Ractor.new { puts $" }.take'
-e:1:in `block in <main>': can not access global variables $" from non-main Ractors (RuntimeError)
```
That is expected, given the rules for global variables.

```
ruby --disable-gems -e 'Ractor.new { require "./a.rb"; }.take; p $"'
[... , "/home/eregon/a.rb"]
```
Is it OK that the Ractor can do `require`, which does modify `$"`?

I think it's not, and it might lead to segfaults if e.g. the main Ractor mutates `$"` in parallel to some other Ractor doing `require`.

Probably `require` needs to be forbidden in non-main Ractors (it does mutate `$"`, so it's logical), or there needs to be always VM-global synchronization on any access to `$"` (otherwise, segfaults are possible).
The latter doesn't seem reasonable, especially when considering the user might do `$".each { ... }`.

---

Note that RubyGems' `require` does not work on non-main Ractors (pretty much expected given it depends on a lot of global state):
```
$ ruby -e 'Ractor.new { require "./a.rb"; }.take'
<internal:/home/eregon/prefix/ruby-master/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:37:in `require': can not access non-shareable objects in constant Kernel::RUBYGEMS_ACTIVATION_MONITOR by non-main ractor. (NameError)
```

This probably also has consequences for `autoload`.
Maybe the `zeitwerk` gem can help with the mode to resolve all autoload at once.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2023-01-16 13:46 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-12-21 18:31 [ruby-core:101598] [Ruby master Bug#17420] Unsafe mutation of $" when doing non-RubyGems require in Ractor eregontp
2020-12-27 17:30 ` [ruby-core:101752] " marcandre-ruby-core
2020-12-28  8:43 ` [ruby-core:101770] " duerst
2020-12-28 11:58 ` [ruby-core:101775] " eregontp
2020-12-28 21:27 ` [ruby-core:101798] " shatrov
2021-01-05  1:39 ` [ruby-core:101920] " ko1
2021-01-05  2:16 ` [ruby-core:101922] " marcandre-ruby-core
2021-01-07 13:22 ` [ruby-core:101972] " eregontp
2021-01-07 13:23 ` [ruby-core:101973] " eregontp
2023-01-16 13:46 ` [ruby-core:111834] " Eregon (Benoit Daloze) via ruby-core

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).