ruby-core@ruby-lang.org archive (unofficial mirror)
 help / color / mirror / Atom feed
* [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
@ 2021-01-29 15:29 marcandre-ruby-core
  2021-01-29 15:31 ` [ruby-core:102306] " marcandre-ruby-core
                   ` (23 more replies)
  0 siblings, 24 replies; 25+ messages in thread
From: marcandre-ruby-core @ 2021-01-29 15:29 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been reported by marcandre (Marc-Andre Lafortune).

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102306] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
@ 2021-01-29 15:31 ` marcandre-ruby-core
  2021-01-29 15:41 ` [ruby-core:102307] " eregontp
                   ` (22 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: marcandre-ruby-core @ 2021-01-29 15:31 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by marcandre (Marc-Andre Lafortune).


Forgot to mention an example use-case: the URI global register for schemes see https://github.com/ruby/uri/pull/15

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90163

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102307] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
  2021-01-29 15:31 ` [ruby-core:102306] " marcandre-ruby-core
@ 2021-01-29 15:41 ` eregontp
  2021-01-29 17:03 ` [ruby-core:102308] " marcandre-ruby-core
                   ` (21 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: eregontp @ 2021-01-29 15:41 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by Eregon (Benoit Daloze).


I think 1) or 2) is much better than 0).
And also this change will make it significantly easier to run existing code on Ractor, or to change the code to make it run with Ractor.

I think read-write would also be fine. But read-only is definitely a good step.
Concurrent `@counter += 1` would already be an issue with threads, so it doesn't seem a new issue, callers need to care if they use the previous value.
For the frequent case of assigning a new shareable object, there is no problematic race there.

I suspect internally, reads and writes for module ivars will need synchronization anyway, so I guess implementation-wise it's easy to allow writes.

Something mentioned not so explicitly above is, of course, accessing `@ivars` of a Module in a non-main Ractor should only be allowed if the value is shareable.
If the value is not shareable, then it must be `IsolationError`.

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90164

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102308] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
  2021-01-29 15:31 ` [ruby-core:102306] " marcandre-ruby-core
  2021-01-29 15:41 ` [ruby-core:102307] " eregontp
@ 2021-01-29 17:03 ` marcandre-ruby-core
  2021-01-30 21:59 ` [ruby-core:102327] " joeydwong
                   ` (20 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: marcandre-ruby-core @ 2021-01-29 17:03 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by marcandre (Marc-Andre Lafortune).


From a discussion with @ko1, config could be (or should be?) using `TVar`.

We need a good solution that is builtin.

If `TVar` becomes builtin, then that is a possible solution. It seems like overkill for a mostly constant config (e.g. URI scheme list) and is not backwards compatible though.

Allowing `reading` from instance variables has the advantage of being simple and backward compatible. Just to be clear: `TVar` seems like interesting addition, but not particularly for config-style global state.

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90165

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102327] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (2 preceding siblings ...)
  2021-01-29 17:03 ` [ruby-core:102308] " marcandre-ruby-core
@ 2021-01-30 21:59 ` joeydwong
  2021-01-30 22:56 ` [ruby-core:102328] " marcandre-ruby-core
                   ` (19 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: joeydwong @ 2021-01-30 21:59 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by blowfishpro (Joseph Wong).


Would it be reasonable to force a class/module to be frozen (and its instance variables deep frozen) before being able to access class instance variables from a non-main Ractor?

There would probably need to be a new interface to do this as `Ractor.shareable?` already returns true for a module and `Ractor.make_shareable` does nothing to it.

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90185

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102328] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (3 preceding siblings ...)
  2021-01-30 21:59 ` [ruby-core:102327] " joeydwong
@ 2021-01-30 22:56 ` marcandre-ruby-core
  2021-02-01 16:58 ` [ruby-core:102363] " daniel
                   ` (18 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: marcandre-ruby-core @ 2021-01-30 22:56 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by marcandre (Marc-Andre Lafortune).


blowfishpro (Joseph Wong) wrote in #note-4:
> Would it be reasonable to force a class/module to be frozen (and its instance variables deep frozen) before being able to access class instance variables from a non-main Ractor?

Not really, no. Not only is it quite restrictive, but this gains nothing over status quo; you might as well use a constant, and deep-freeze it before you access it from non-main Ractor.


----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90186

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102363] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (4 preceding siblings ...)
  2021-01-30 22:56 ` [ruby-core:102328] " marcandre-ruby-core
@ 2021-02-01 16:58 ` daniel
  2021-02-01 17:13 ` [ruby-core:102365] " eregontp
                   ` (17 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: daniel @ 2021-02-01 16:58 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by Dan0042 (Daniel DeLorme).


If a class instance variable refers to a shareable object it would make sense to be able to read it in ractors. But what about reassigning the instance variable? How does the proposal work then?

```ruby
## in main ractor:
@a = Ractor.make_shareable(data)
# @a is readable in ractors
@a = data
# @a is no longer readable in ractors? assignment disallowed?

## in non-main ractor:
@a = Ractor.make_shareable(data)
# ???
```

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90224

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102365] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (5 preceding siblings ...)
  2021-02-01 16:58 ` [ruby-core:102363] " daniel
@ 2021-02-01 17:13 ` eregontp
  2021-02-01 17:17 ` [ruby-core:102366] " marcandre-ruby-core
                   ` (16 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: eregontp @ 2021-02-01 17:13 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by Eregon (Benoit Daloze).


For simplicity, I think it's probably best to disallow reassigning in non-main Ractors.
It could be allowed though, as long as the value is shareable, then all other Ractors would notice the new value.

Reassigning in main Ractor should be allowed, and then raise when trying to read a non-shareable values from other Ractors.
i.e., the semantics of the main Ractor should not change whether there is only the main Ractor or multiple Ractors.

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90226

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102366] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (6 preceding siblings ...)
  2021-02-01 17:13 ` [ruby-core:102365] " eregontp
@ 2021-02-01 17:17 ` marcandre-ruby-core
  2021-02-01 17:43 ` [ruby-core:102367] " daniel
                   ` (15 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: marcandre-ruby-core @ 2021-02-01 17:17 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by marcandre (Marc-Andre Lafortune).


Right. My proposal disallows reassigning from non-main ractors.

Reassigning from main Ractor to a non-shareable `data` is no problem, but future attempts to read it from non-main Ractor will result in `IsolationError` (as they do currently).

```ruby
# Main ractor:
String.singleton_class.attr_accessor :a
String.a = [].freeze
Ractor.new { p Ractor.a }.take # => []
Ractor.new { Ractor.a = nil }.take # => IsolationError (can't reassign from non-main Ractor)
String.a = []
Ractor.new { p Ractor.a } # => IsolationError (can't read non-shareable from non-main Ractor)
```


----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90227

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102367] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (7 preceding siblings ...)
  2021-02-01 17:17 ` [ruby-core:102366] " marcandre-ruby-core
@ 2021-02-01 17:43 ` daniel
  2021-02-01 18:11 ` [ruby-core:102368] " marcandre-ruby-core
                   ` (14 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: daniel @ 2021-02-01 17:43 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by Dan0042 (Daniel DeLorme).


That makes sense, but I'm curious about the implementation. I believe this requires synchronization of every access to a class instance variable?

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90228

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102368] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (8 preceding siblings ...)
  2021-02-01 17:43 ` [ruby-core:102367] " daniel
@ 2021-02-01 18:11 ` marcandre-ruby-core
  2021-02-01 19:07 ` [ruby-core:102370] " eregontp
                   ` (13 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: marcandre-ruby-core @ 2021-02-01 18:11 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by marcandre (Marc-Andre Lafortune).


Dan0042 (Daniel DeLorme) wrote in #note-9:
> That makes sense, but I'm curious about the implementation. I believe this requires synchronization of every access to a class instance variable?

I'm not sure, but I think not. From non-main Ractor, grab the current value of the class instance variable. I think that can be made safe, e.g. if the ivar storage data uses immutable data structures.

Before returning the value, verify that it is shareable.

Objects have a "shareable" bit, but that might not be set for some shareable objects, so testing if an object is shareable might "modify" it by setting this "shareable" bit (see `rb_ractor_shareable_p_continue`) but I believe that this can be done simultaneously by different Ractors without issue as it is idempotent and happens only on immutable objects. Of course, ko1 will know if I'm missing something 😅

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90229

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102370] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (9 preceding siblings ...)
  2021-02-01 18:11 ` [ruby-core:102368] " marcandre-ruby-core
@ 2021-02-01 19:07 ` eregontp
  2021-02-01 19:27 ` [ruby-core:102371] " eregontp
                   ` (12 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: eregontp @ 2021-02-01 19:07 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by Eregon (Benoit Daloze).


Dan0042 (Daniel DeLorme) wrote in #note-9:
> That makes sense, but I'm curious about the implementation. I believe this requires synchronization of every access to a class instance variable?

I think essentially all accesses to mutable data on a module must have some form of synchronization, because they can be mutated from the main thread in parallel.
So I'd think it's already synchronized.

This is BTW one thing I dislike about Ractor: @ivar accesses depend on the receiver, if it's a module it's completely different than for all other objects, and requires synchronization.
But, that's already there, so might as well allow reading from other Ractors than being both inconsistent and inconvenient.


----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90230

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102371] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (10 preceding siblings ...)
  2021-02-01 19:07 ` [ruby-core:102370] " eregontp
@ 2021-02-01 19:27 ` eregontp
  2021-02-15  9:35 ` [ruby-core:102498] " ko1
                   ` (11 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: eregontp @ 2021-02-01 19:27 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by Eregon (Benoit Daloze).


To be more precise, and looking at the source, currently there is no synchronization for module ivars, only a check that only the main Ractor can access them:
https://github.com/ruby/ruby/blob/5803ac1c734568837d2010bd38f122ba24cbae2b/variable.c#L906-L913

The fast path for the "ivar set" bytecode already doesn't handle T_MODULE/T_CLASS:
https://github.com/ruby/ruby/blob/5803ac1c734568837d2010bd38f122ba24cbae2b/vm_insnhelper.c#L1250-L1253

And it ends up here which has the Module-specific logic:
https://github.com/ruby/ruby/blob/5803ac1c734568837d2010bd38f122ba24cbae2b/variable.c#L1480-L1484

Other state like constants and methods already have synchronization, e.g.,
https://github.com/ruby/ruby/blob/5803ac1c734568837d2010bd38f122ba24cbae2b/variable.c#L3621-L3625

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90231

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102498] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (11 preceding siblings ...)
  2021-02-01 19:27 ` [ruby-core:102371] " eregontp
@ 2021-02-15  9:35 ` ko1
  2021-02-15  9:42 ` [ruby-core:102499] " ko1
                   ` (10 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: ko1 @ 2021-02-15  9:35 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by ko1 (Koichi Sasada).


This is a summary for tomorrow's dev-meeting by my understanding.

----

* ko1: This ticket proposal: To maintain global configuration (mutable information), class/module instance variables should be readable from other ractors.
 
```ruby=
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

# Current
Ractor.new{ p Foo.config } # => IsolationError

# Proposal
Ractor.new do
  p Foo.config #=> {example: 42}
  p Foo.config = {example: 43}.freeze
    #=> IsolationError, beacuse it is read-only from non-main ractors
end
    
Foo.config = {example: 44}.freeze # allowed updating from the main ractor
```

* ko1: There are two concerns: (1) atomicity concern and (2) performance concern.

(1) Atomicity concern

If two or more ivars (named `@a` and `@b`) should be update atomic, but threre is no way to synchronize them.

```ruby=
class C
  @a = @b = 0
  def self.update
    # assertion: @a, @b should be equal
    @a += 1
    @b += 1
  end
  def self.vars
    [@a, @b]
  end
end
```

Main ractor can calls `C.update` and update ivars. A ractor can call `C.vars` and it can returns inconsist values (`@a != @b`).

The danger of this concern is relatively low because this example is very artificial. Maybe most of usecase is initialization at loading time and no other ractors read while mutating. Also there is no coupled variables (like `@a, @b`), there is no problem. For example, there is no problem with only one `@config` ivar which manages all configrations. In other words, two or more configurations `@configA`, `@configB`, ... can have an atomicity issue.

The following code is also artifitial example.

```ruby=
class Fib
  @a = @b = 1
  # @a and @b are successive parts of the Fibonacci sequence.
  def self.next
    @a, @b = @b, @a + @b
  end

  def values
    # it should return successive parts of the Fib seq.
    # == "Fib seq constraint"
    [@a, @b]
  end

  def self.eventloop
    loop{
      Ractor.receive; Fib.next
    }
  end
end

gen = Ractor.new(Ractor.current){|main| loop{ m << true } }
con = Ractor.new{ 
  p Fib.value #=> return values can violate "Fib seq constraint"
}
```

If a user misused as an above example (using class/module ivars for the mutable state repository and updating them with multiple ractors (via the main-ractor)), it is danger.

For this concern, we have several options.

* (a) there is no problem to introduce this feature because it is almost safe.
* (b) Ractor is designed to avoid such consistency issues even if it can be avoided by careful programming. So this feature should not be introduced.

(b) is my position, but I agree it is very conservative.

This proposal has advantage for compatibility because many existing code can use ivars for sharing the global configurations and there is no need to rewrite them (if they only refer to sharable objects).

For example, `pp` library has one global configuration: `sharing_detection` which is stored in a class instance variable.

https://github.com/ruby/ruby/blob/master/lib/pp.rb#L109

For Ractor, it was rewrote by using Ractor-local configuration, but it was not ideal modification but ad-hoc modification with existing tools.

If this proposal is introduced, we can revert the ad-hoc modification.

(2) Performance concern

To allow accessing ivars from other ractors, every ivar accesses to class/module should be synchronized. Current implementation doesn't need to synchronize this accesses.

For constants and method search, we implemented const cache and method cache mechanisms. Such cache mechanisms are reasonable because they are not frequently rewriting. On the other hands, ivars can be mutated more frequently and not sure the it is reasonable to have a cache mechanism for it.

----

ko1: Alternative proposal is using TVar.

Rewriting configuration example with TVar:

```ruby=
module Foo
  Config = Ractor::TVar.new{ {example: 42}.freeze }
end

Ractor.new do
  Ractor.atomically do
    p Foo::Config.value #=> {example: 42}
  end
  ...
  Ractor.atomically do
    Foo::Config.value = {example: 43}.freeze
    # modification is allowed within atomically block
  end
end

Ractor.atomically do
  Foo::Config.value = {example: 44}.freeze
end
```

The advantage of this example is it is clear that manipulating sharable state between Ractors because `Ractor.atomically` method is needed. Another way of saying this is that we can become more aware of creating a shared state. With instance variables, it is hard to figure out which instance variables are shared with multiple Ractors.

Disadvantages:

* Writing `Ractor.atomically` is long to type.
* Incompatible with older versions.

Configuration should not be changed frequently, so the performance should not be a problem.

```ruby=
module Foo
  Config = Ractor::TVar.new{ {example: 42}.freeze }
end

Ractor.new do
  p Foo::Config.value #=> {example: 42}
end
    
Ractor.atomically do
  Foo::Config.value = {example: 44}.freeze
end
```

Rewriting other examples with TVar.

```ruby=
class C
  A = Ractor::TVar.new 0
  B = Ractor::TVar.new 0

  def self.update
    # assertion: A.value and B.value should be equal
    Ractor.atomically do
      A.value += 1
      B.value += 1
    end
  end
  def self.vars
    Ractor.atomically do
      [A.value, B.value]
    end
  end
end
```

```ruby=
class Fib
  A = Ractor::TVar.new 1
  B = Ractor::TVar.new 1

  def self.next
    Ractor.atomically do
      A.value, B.value = B.value, A.value + B.value
    end
  end

  def values
    # it should return successive parts of the Fib seq.
    # == "Fib seq constraint"
    Ractor.atomically do
      [A.value, B.value]
    end
  end

  def self.eventloop
    loop{
      Ractor.receive; Fib.next
    }
  end
end

gen = Ractor.new(Ractor.current){|main| loop{ m << true } }
con = Ractor.new{ 
  p Fib.value #=> TVar allows us to keep constraint
}
Fib.eventloop

# NOTE: Using TVars, there is no reason to maintain states
# by a main-ractor, so gen ractor can access Fib.next directly.

gen = Ractor.new{loop{ Fib.next } }
con = Ractor.new{ 
  p Fib.value #=> TVar allows us to keep constraint
}
```


----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90393

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102499] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (12 preceding siblings ...)
  2021-02-15  9:35 ` [ruby-core:102498] " ko1
@ 2021-02-15  9:42 ` ko1
  2021-02-15 16:49 ` [ruby-core:102507] " eregontp
                   ` (9 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: ko1 @ 2021-02-15  9:42 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by ko1 (Koichi Sasada).


Eregon (Benoit Daloze) wrote in #note-11:
> This is BTW one thing I dislike about Ractor: @ivar accesses depend on the receiver, if it's a module it's completely different than for all other objects, and requires synchronization.
> But, that's already there, so might as well allow reading from other Ractors than being both inconsistent and inconvenient.

Sorry, I don't understand this point.
There is no special for class/module as a receiver objects (implementation is special, but no Ruby-level difference).
Do I miss something?

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90394

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102507] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (13 preceding siblings ...)
  2021-02-15  9:42 ` [ruby-core:102499] " ko1
@ 2021-02-15 16:49 ` eregontp
  2021-02-16  2:14 ` [ruby-core:102512] " ko1
                   ` (8 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: eregontp @ 2021-02-15 16:49 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by Eregon (Benoit Daloze).


ko1 (Koichi Sasada) wrote in #note-14:
> Sorry, I couldn't understand this point.
> There is no special for class/module as a receiver objects (implementation is special, but no Ruby-level difference).
> Do I miss something?

There is a behavior difference, before Ractor `@foo` would always work and never raise.
And `@foo =` would only raise if the receiver is frozen.

Inside a Ractor, currently `@foo` works unless `self.is_a?(Module)`, same for `@foo =` (the inconsistency I mention: the same syntax has widely different semantics based on the receiver).
If we accept this proposal, then at least `@foo` on a Module works if the value can be safely read (= the value is shareable).

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90401

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102512] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (14 preceding siblings ...)
  2021-02-15 16:49 ` [ruby-core:102507] " eregontp
@ 2021-02-16  2:14 ` ko1
  2021-02-16 12:45 ` [ruby-core:102537] " eregontp
                   ` (7 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: ko1 @ 2021-02-16  2:14 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by ko1 (Koichi Sasada).


Eregon (Benoit Daloze) wrote in #note-15:
> There is a behavior difference, before Ractor `@foo` would always work and never raise.
> And `@foo =` would only raise if the receiver is frozen.

OK, you meant sharable or not, but not about class/module (class/module are typical cases, though).


----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90406

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102537] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (15 preceding siblings ...)
  2021-02-16  2:14 ` [ruby-core:102512] " ko1
@ 2021-02-16 12:45 ` eregontp
  2021-02-16 20:29 ` [ruby-core:102546] " marcandre-ruby-core
                   ` (6 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: eregontp @ 2021-02-16 12:45 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by Eregon (Benoit Daloze).


ko1 (Koichi Sasada) wrote in #note-16:
> OK, you meant sharable or not, but not about class/module (class/module are typical cases, though).

Isn't it actually only about class/module though?
For a `Fixnum` or a `Ractor.make_shareable(Object.new)`, it's allowed to read @ivars. And it's a regular FrozenError to try to write an @ivar since they are frozen.

But for class/module, and I think only for those, it's currently not allowed to even read @ivars, even though class/module are considered shareable.

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90437

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102546] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (16 preceding siblings ...)
  2021-02-16 12:45 ` [ruby-core:102537] " eregontp
@ 2021-02-16 20:29 ` marcandre-ruby-core
  2021-02-16 23:26 ` [ruby-core:102547] " eregontp
                   ` (5 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: marcandre-ruby-core @ 2021-02-16 20:29 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by marcandre (Marc-Andre Lafortune).


ko1 (Koichi Sasada) wrote in #note-13:
> (2) Performance concern
> 
> To allow accessing ivars from other ractors, every ivar accesses to class/module should be synchronized.

I am surprised this is the case. Would it not be possible to have overwriting an existing instance variable be a single memory write operation? Adding a new instance variable would similarly involve creating a new ivar table and setting it with a single memory write operation?

> Current implementation doesn't need to synchronize this accesses.

That is not completely accurate. Current implementation does not but it is not 100% safe. The following completely artificial code crashes (sometimes):

```ruby
class String
  def self.test
    4000.times.map { |i| eval(<<~RUBY) }.each(&:join)
      Thread.new do
        10.times do
          Thread.pass
          @x#{i} = 42
        end
      end
    RUBY
  end
end

String.test
p String.instance_variables.size # => 4000
```

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90447

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102547] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (17 preceding siblings ...)
  2021-02-16 20:29 ` [ruby-core:102546] " marcandre-ruby-core
@ 2021-02-16 23:26 ` eregontp
  2021-02-16 23:33 ` [ruby-core:102548] " eregontp
                   ` (4 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: eregontp @ 2021-02-16 23:26 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by Eregon (Benoit Daloze).


marcandre (Marc-Andre Lafortune) wrote in #note-18:
> I am surprised this is the case. Would it not be possible to have overwriting an existing instance variable be a single memory write operation? Adding a new instance variable would similarly involve creating a new ivar table and setting it with a single memory write operation?

That would lose writes if there are concurrent writes for both a new variable and an existing one ([illustration](https://speakerdeck.com/eregon/efficient-and-thread-safe-objects-for-dynamically-typed-languages?slide=18)).
It always needs some form of synchronization, or more indirections (e.g., chaining).

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90449

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102548] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (18 preceding siblings ...)
  2021-02-16 23:26 ` [ruby-core:102547] " eregontp
@ 2021-02-16 23:33 ` eregontp
  2021-02-17  0:20 ` [ruby-core:102549] " marcandre-ruby-core
                   ` (3 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: eregontp @ 2021-02-16 23:33 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by Eregon (Benoit Daloze).


I should add, instance variable writes on modules on TruffleRuby are already synchronized (for modules assigned to a constant, and effectively reachable by multiple threads), and that doesn't seem to be a performance issue at all.
Reading can be done without synchronization for existing variables (just need to check if the ivar storage is large enough if it's only growing).

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90450

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102549] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (19 preceding siblings ...)
  2021-02-16 23:33 ` [ruby-core:102548] " eregontp
@ 2021-02-17  0:20 ` marcandre-ruby-core
  2021-02-17  0:24 ` [ruby-core:102550] " marcandre-ruby-core
                   ` (2 subsequent siblings)
  23 siblings, 0 replies; 25+ messages in thread
From: marcandre-ruby-core @ 2021-02-17  0:20 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by marcandre (Marc-Andre Lafortune).


Sorry, I should have been more clear. Synchronization for writing: yes (as it should probably already be the case, see my crashing example). I meant that the access for reading wouldn't have to be synchronized I believe.

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90451

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102550] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (20 preceding siblings ...)
  2021-02-17  0:20 ` [ruby-core:102549] " marcandre-ruby-core
@ 2021-02-17  0:24 ` marcandre-ruby-core
  2021-02-24 17:12 ` [ruby-core:102594] " marcandre-ruby-core
  2021-10-22 16:33 ` [ruby-core:105753] " ko1 (Koichi Sasada)
  23 siblings, 0 replies; 25+ messages in thread
From: marcandre-ruby-core @ 2021-02-17  0:24 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by marcandre (Marc-Andre Lafortune).


Additionally, my suggestion would make synchronization not necessary across threads in the sense that multiple writes from different threads would simply produce "bad" results but wouldn't crash. It would be up to developpers to use `Mutex` in those cases.

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90452

* Author: marcandre (Marc-Andre Lafortune)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:102594] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (21 preceding siblings ...)
  2021-02-17  0:24 ` [ruby-core:102550] " marcandre-ruby-core
@ 2021-02-24 17:12 ` marcandre-ruby-core
  2021-10-22 16:33 ` [ruby-core:105753] " ko1 (Koichi Sasada)
  23 siblings, 0 replies; 25+ messages in thread
From: marcandre-ruby-core @ 2021-02-24 17:12 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by marcandre (Marc-Andre Lafortune).

Status changed from Open to Assigned

This has been accepted 🎉

Follow-up question: could we introduce this change in 3.0.1 line? Otherwise we have to wait until 3.1.0 and many developpers might resort to the `const_set` hack in the meantime...

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-90579

* Author: marcandre (Marc-Andre Lafortune)
* Status: Assigned
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [ruby-core:105753] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables
  2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
                   ` (22 preceding siblings ...)
  2021-02-24 17:12 ` [ruby-core:102594] " marcandre-ruby-core
@ 2021-10-22 16:33 ` ko1 (Koichi Sasada)
  23 siblings, 0 replies; 25+ messages in thread
From: ko1 (Koichi Sasada) @ 2021-10-22 16:33 UTC (permalink / raw)
  To: ruby-core

Issue #17592 has been updated by ko1 (Koichi Sasada).

Status changed from Assigned to Closed

https://github.com/ruby/ruby/pull/5006
merged.

----------------------------------------
Feature #17592: Ractor should allowing reading shareable class instance variables
https://bugs.ruby-lang.org/issues/17592#change-94260

* Author: marcandre (Marc-Andre Lafortune)
* Status: Closed
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
It would be very helpful if Ractor was allowing reading class instance variables from non-main Ractor.


Currently is raises an IsolationError:

```ruby
module Foo
  singleton_class.attr_accessor :config
  Foo.config = {example: 42}.freeze
end

Ractor.new { p Foo.config } # => IsolationError
```

This limitation makes it challenging to have an efficient way to store general configs, i.e. global data that mutated a few times when resources get loaded but it immutable afterwards, and needs to be read all the time.

Currently the only way to do this is to use a constant and use `remove_const` + `const_set` (which can not be made atomic easily).

I think that allowing reading only may be the best solution to avoid any race condition, e.g. two different Ractors that call `@counter += 1`.

The only 3 scenarios I see here are:
0) declare the constant hack the official way to store config-style data
1) allow reading of instance variables for shareable objects (as long as the data is shareable)
2) allow read-write

I prefer 1)



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 25+ messages in thread

end of thread, other threads:[~2021-10-22 16:36 UTC | newest]

Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-29 15:29 [ruby-core:102305] [Ruby master Feature#17592] Ractor should allowing reading shareable class instance variables marcandre-ruby-core
2021-01-29 15:31 ` [ruby-core:102306] " marcandre-ruby-core
2021-01-29 15:41 ` [ruby-core:102307] " eregontp
2021-01-29 17:03 ` [ruby-core:102308] " marcandre-ruby-core
2021-01-30 21:59 ` [ruby-core:102327] " joeydwong
2021-01-30 22:56 ` [ruby-core:102328] " marcandre-ruby-core
2021-02-01 16:58 ` [ruby-core:102363] " daniel
2021-02-01 17:13 ` [ruby-core:102365] " eregontp
2021-02-01 17:17 ` [ruby-core:102366] " marcandre-ruby-core
2021-02-01 17:43 ` [ruby-core:102367] " daniel
2021-02-01 18:11 ` [ruby-core:102368] " marcandre-ruby-core
2021-02-01 19:07 ` [ruby-core:102370] " eregontp
2021-02-01 19:27 ` [ruby-core:102371] " eregontp
2021-02-15  9:35 ` [ruby-core:102498] " ko1
2021-02-15  9:42 ` [ruby-core:102499] " ko1
2021-02-15 16:49 ` [ruby-core:102507] " eregontp
2021-02-16  2:14 ` [ruby-core:102512] " ko1
2021-02-16 12:45 ` [ruby-core:102537] " eregontp
2021-02-16 20:29 ` [ruby-core:102546] " marcandre-ruby-core
2021-02-16 23:26 ` [ruby-core:102547] " eregontp
2021-02-16 23:33 ` [ruby-core:102548] " eregontp
2021-02-17  0:20 ` [ruby-core:102549] " marcandre-ruby-core
2021-02-17  0:24 ` [ruby-core:102550] " marcandre-ruby-core
2021-02-24 17:12 ` [ruby-core:102594] " marcandre-ruby-core
2021-10-22 16:33 ` [ruby-core:105753] " ko1 (Koichi Sasada)

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).