[ruby-core:105504] [Ruby master Bug#18232] Ractor.make_shareable is broken in code loaded with RubyVM::InstructionSequence.load_from_binary

[ruby-core:105504] [Ruby master Bug#18232] Ractor.make_shareable is broken in code loaded with RubyVM::InstructionSequence.load_from_binary

[byroot (Jean Boussier)]

Issue #18232 has been reported by byroot (Jean Boussier).

----------------------------------------
Bug #18232: Ractor.make_shareable is broken in code loaded with RubyVM::InstructionSequence.load_from_binary
https://bugs.ruby-lang.org/issues/18232

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.0.2p107, ruby 3.1.0dev
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------
This was first reported as a Bootsnap bug: https://github.com/Shopify/bootsnap/issues/376

But I reduced it to:

```ruby
source = <<~RUBY
  x = :foo
  pr = Proc.new { p x }
  Ractor.make_shareable(pr)
  pr.call
RUBY
RubyVM::InstructionSequence.compile(source).eval # :foo
RubyVM::InstructionSequence.load_from_binary(RubyVM::InstructionSequence.compile(source).to_binary).eval # false
```

The bug is present on both 3.0.2 and 3.1.0-dev. I have no idea what might cause this.



-- 
https://bugs.ruby-lang.org/

[ruby-core:105567] [Ruby master Bug#18232] Ractor.make_shareable is broken in code loaded with RubyVM::InstructionSequence.load_from_binary

[byroot (Jean Boussier)]

Issue #18232 has been updated by byroot (Jean Boussier).


Because [OpenStruct was recently changed to use `make_shareable`](https://github.com/ruby/ostruct/pull/29), this bug suddenly became much more problematic. We actually hit it in production.

----------------------------------------
Bug #18232: Ractor.make_shareable is broken in code loaded with RubyVM::InstructionSequence.load_from_binary
https://bugs.ruby-lang.org/issues/18232#change-94024

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.0.2p107, ruby 3.1.0dev
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------
This was first reported as a Bootsnap bug: https://github.com/Shopify/bootsnap/issues/376

But I reduced it to:

```ruby
source = <<~RUBY
  x = :foo
  pr = Proc.new { p x }
  Ractor.make_shareable(pr)
  pr.call
RUBY
RubyVM::InstructionSequence.compile(source).eval # :foo
RubyVM::InstructionSequence.load_from_binary(RubyVM::InstructionSequence.compile(source).to_binary).eval # false
```

The bug is present on both 3.0.2 and 3.1.0-dev. I have no idea what might cause this.



-- 
https://bugs.ruby-lang.org/

[ruby-core:105574] [Ruby master Bug#18232] Ractor.make_shareable is broken in code loaded with RubyVM::InstructionSequence.load_from_binary

[Eregon (Benoit Daloze)]

Issue #18232 has been updated by Eregon (Benoit Daloze).


FYI the way it currently works for OStruct seems buggy on the Ractor side: #18243

----------------------------------------
Bug #18232: Ractor.make_shareable is broken in code loaded with RubyVM::InstructionSequence.load_from_binary
https://bugs.ruby-lang.org/issues/18232#change-94030

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.0.2p107, ruby 3.1.0dev
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------
This was first reported as a Bootsnap bug: https://github.com/Shopify/bootsnap/issues/376

But I reduced it to:

```ruby
source = <<~RUBY
  x = :foo
  pr = Proc.new { p x }
  Ractor.make_shareable(pr)
  pr.call
RUBY
RubyVM::InstructionSequence.compile(source).eval # :foo
RubyVM::InstructionSequence.load_from_binary(RubyVM::InstructionSequence.compile(source).to_binary).eval # false
```

The bug is present on both 3.0.2 and 3.1.0-dev. I have no idea what might cause this.



-- 
https://bugs.ruby-lang.org/

[ruby-core:105575] [Ruby master Bug#18232] Ractor.make_shareable is broken in code loaded with RubyVM::InstructionSequence.load_from_binary

[Eregon (Benoit Daloze)]

Issue #18232 has been updated by Eregon (Benoit Daloze).


Smaller repro:
```
ruby -e 'Object.new.instance_eval { p object_id; proc = Ractor.make_shareable -> { self }; Ractor.new(proc) { |c| p c.call.object_id }.take }'
$ 60
<internal:ractor>:267: warning: Ractor is experimental, and the behavior may change in future versions of Ruby! Also there are many implementation issues.
60
```

In contrast to:
```
ruby -e 'a=Object.new; Ractor.make_shareable -> { a }'
<internal:ractor>:816:in `make_shareable': can not make shareable Proc because it can refer unshareable object #<Object:0x00007fd8bdd8f688> from variable `a' (Ractor::IsolationError)
	from -e:1:in `<main>'
```

It seems there is no check for `self` but there should be, it's like a capture local variable.

Either it raises like for local vars or it `Ractor.make_shareable` the `self`.

If it raises ostruct.rb will break but that's fixable by `Ractor.make_shareable self` first which is probably best for clarity anyway, cc @marcandre.

----------------------------------------
Bug #18232: Ractor.make_shareable is broken in code loaded with RubyVM::InstructionSequence.load_from_binary
https://bugs.ruby-lang.org/issues/18232#change-94033

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.0.2p107, ruby 3.1.0dev
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------
This was first reported as a Bootsnap bug: https://github.com/Shopify/bootsnap/issues/376

But I reduced it to:

```ruby
source = <<~RUBY
  x = :foo
  pr = Proc.new { p x }
  Ractor.make_shareable(pr)
  pr.call
RUBY
RubyVM::InstructionSequence.compile(source).eval # :foo
RubyVM::InstructionSequence.load_from_binary(RubyVM::InstructionSequence.compile(source).to_binary).eval # false
```

The bug is present on both 3.0.2 and 3.1.0-dev. I have no idea what might cause this.



-- 
https://bugs.ruby-lang.org/

[ruby-core:105577] [Ruby master Bug#18232] Ractor.make_shareable is broken in code loaded with RubyVM::InstructionSequence.load_from_binary

[tenderlovemaking (Aaron Patterson)]

Issue #18232 has been updated by tenderlovemaking (Aaron Patterson).


It looks like the [outer variables list](https://github.com/ruby/ruby/blob/e4d85d3a2d0c7d4a1abf981fb1ba557e21cad968/vm.c#L1124) is being lost when we dump the iseq to binary.  AFAICT, that list is only created at compile time.  I'll try to make a patch that reconstructs the list on iseq load as well.

----------------------------------------
Bug #18232: Ractor.make_shareable is broken in code loaded with RubyVM::InstructionSequence.load_from_binary
https://bugs.ruby-lang.org/issues/18232#change-94035

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.0.2p107, ruby 3.1.0dev
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------
This was first reported as a Bootsnap bug: https://github.com/Shopify/bootsnap/issues/376

But I reduced it to:

```ruby
source = <<~RUBY
  x = :foo
  pr = Proc.new { p x }
  Ractor.make_shareable(pr)
  pr.call
RUBY
RubyVM::InstructionSequence.compile(source).eval # :foo
RubyVM::InstructionSequence.load_from_binary(RubyVM::InstructionSequence.compile(source).to_binary).eval # false
```

The bug is present on both 3.0.2 and 3.1.0-dev. I have no idea what might cause this.



-- 
https://bugs.ruby-lang.org/

[ruby-core:105582] [Ruby master Bug#18232] Ractor.make_shareable is broken in code loaded with RubyVM::InstructionSequence.load_from_binary

[tenderlovemaking (Aaron Patterson)]

Issue #18232 has been updated by tenderlovemaking (Aaron Patterson).

File 0001-Dump-outer-variables-tables-when-dumping-an-iseq-to-.patch added

I've attached a patch that fixes this (for me) along with a test.  I had to change the information that we dump when dumping an ISeq, so maybe we need to change the binary format version number or something?  I'm not sure.  Anyway there is also a PR [here](https://github.com/ruby/ruby/pull/4942)

----------------------------------------
Bug #18232: Ractor.make_shareable is broken in code loaded with RubyVM::InstructionSequence.load_from_binary
https://bugs.ruby-lang.org/issues/18232#change-94052

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.0.2p107, ruby 3.1.0dev
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------
This was first reported as a Bootsnap bug: https://github.com/Shopify/bootsnap/issues/376

But I reduced it to:

```ruby
source = <<~RUBY
  x = :foo
  pr = Proc.new { p x }
  Ractor.make_shareable(pr)
  pr.call
RUBY
RubyVM::InstructionSequence.compile(source).eval # :foo
RubyVM::InstructionSequence.load_from_binary(RubyVM::InstructionSequence.compile(source).to_binary).eval # false
```

The bug is present on both 3.0.2 and 3.1.0-dev. I have no idea what might cause this.

---Files--------------------------------
0001-Dump-outer-variables-tables-when-dumping-an-iseq-to-.patch (5.55 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:105652] [Ruby master Bug#18232] Ractor.make_shareable is broken in code loaded with RubyVM::InstructionSequence.load_from_binary

[nagachika (Tomoyuki Chikanaga)]

Issue #18232 has been updated by nagachika (Tomoyuki Chikanaga).

Backport changed from 3.0: REQUIRED to 3.0: DONE

ruby_3_0 a2fe4b75e4b236ad15778c59ace63006ace53889 merged revision(s) 217df51f0e5d9824ed712a4d175f555d932e44d8.

----------------------------------------
Bug #18232: Ractor.make_shareable is broken in code loaded with RubyVM::InstructionSequence.load_from_binary
https://bugs.ruby-lang.org/issues/18232#change-94154

* Author: byroot (Jean Boussier)
* Status: Closed
* Priority: Normal
* ruby -v: ruby 3.0.2p107, ruby 3.1.0dev
* Backport: 3.0: DONE
----------------------------------------
This was first reported as a Bootsnap bug: https://github.com/Shopify/bootsnap/issues/376

But I reduced it to:

```ruby
source = <<~RUBY
  x = :foo
  pr = Proc.new { p x }
  Ractor.make_shareable(pr)
  pr.call
RUBY
RubyVM::InstructionSequence.compile(source).eval # :foo
RubyVM::InstructionSequence.load_from_binary(RubyVM::InstructionSequence.compile(source).to_binary).eval # false
```

The bug is present on both 3.0.2 and 3.1.0-dev. I have no idea what might cause this.

---Files--------------------------------
0001-Dump-outer-variables-tables-when-dumping-an-iseq-to-.patch (5.55 KB)


-- 
https://bugs.ruby-lang.org/