[ruby-core:103159] [Ruby master Bug#12689] Thread isolation of $~ and $_

[ruby-core:103159] [Ruby master Bug#12689] Thread isolation of $~ and $_

[headius]

Issue #12689 has been updated by headius (Charles Nutter).


Waking this up a bit...

The original issue that prompted this bug report has now been FIXED in JRuby 9.2.17.0 by making String#split never read backref from the frame-local storage:

https://github.com/jruby/jruby/pull/6644

Further improvements will come in 9.3 with the following PR, which eliminates ALL core method reads of backref (none of them used its contents anyway, and only read it to reuse it):

https://github.com/jruby/jruby/pull/6647

With these changes, all concurrency issues surrounding $~ *within core methods* are resolved. Users that opt into using `$~` via the variable or methods like `last_match` will still have to take care that the value is not being updated across threads, but such updates will not interfere with any `$~`-related methods in JRuby 9.3.

----------------------------------------
Bug #12689: Thread isolation of $~ and $_
https://bugs.ruby-lang.org/issues/12689#change-91231

* Author: headius (Charles Nutter)
* Status: Open
* Priority: Normal
* Backport: 2.1: UNKNOWN, 2.2: UNKNOWN, 2.3: UNKNOWN
----------------------------------------
We are debating what is correct behavior now, and what should be correct behavior in the future, for the thread-visibility of the special variables `%~` and `$_`

We have several examples from https://github.com/jruby/jruby/issues/3031 that seem to exhibit conflicting behavior...or at least the behavior is unexpected in many cases.

```
$ ruby23 -e 'p = proc { p $~; "foo" =~ /foo/ }; Thread.new {p.call}.join; Thread.new{p.call}.join'
nil
nil

$ ruby23 -e 'def foo; proc { p $~; "foo" =~ /foo/ }; end; p = foo; Thread.new {p.call}.join; Thread.new{p.call}.join'
nil
#<MatchData "foo">

$ ruby23 -e 'p = proc { p $~; "foo" =~ /foo/ }; def foo(p); Thread.new {p.call}.join; Thread.new{p.call}.join; end; foo(p)'
nil
#<MatchData "foo">

$ ruby23 -e 'class Foo; P = proc { p $~; "foo" =~ /foo/ }; def foo; Thread.new {P.call}.join; Thread.new{P.call}.join; end; end; Foo.new.foo'
nil
#<MatchData "foo">

$ ruby23 -e 'def foo; p = proc { p $~; "foo" =~ /foo/ }; Thread.new {p.call}.join; Thread.new{p.call}.join; end; foo'
nil
nil

$ ruby23 -e 'def foo; p = proc { p $~; "foo" =~ /foo/ }; bar(p); end; def bar(p); Thread.new {p.call}.join; Thread.new{p.call}.join; end; foo'
nil
#<MatchData "foo">
```

These cases exhibit some oddities in whether $~ (and presumably $_) are shared across threads.

The immediate thought is that they should be both frame and thread-local...but ko1 points out that such a change would break cases like this:

```
def foo
  /foo/ =~ 'foo'
  Proc.new{
    p $~
  }
end

Thread.new{
  foo.call
}.join
```

So there's a clear conflict here. Users sometimes expect the $~ value to be shared across threads (at least for read, as in ko1's example) and sometimes do not want it shared at all (as in the case of https://github.com/jruby/jruby/issues/3031

Now we discuss.



-- 
https://bugs.ruby-lang.org/

[ruby-core:103160] [Ruby master Bug#12689] Thread isolation of $~ and $_

[headius]

Issue #12689 has been updated by headius (Charles Nutter).


Also note this experimental PR that eliminates the update of `$~` from String#split, since no specs and no tests check that behavior and it seems unexpected and unpredictable (it updates to the last match during the split loop).

https://github.com/jruby/jruby/pull/6646

And a bug I just filed to eliminate backref updating from `start_with?` which should be a fast boolean check and not create a MatchData or update backref:

https://bugs.ruby-lang.org/issues/17771

----------------------------------------
Bug #12689: Thread isolation of $~ and $_
https://bugs.ruby-lang.org/issues/12689#change-91232

* Author: headius (Charles Nutter)
* Status: Open
* Priority: Normal
* Backport: 2.1: UNKNOWN, 2.2: UNKNOWN, 2.3: UNKNOWN
----------------------------------------
We are debating what is correct behavior now, and what should be correct behavior in the future, for the thread-visibility of the special variables `%~` and `$_`

We have several examples from https://github.com/jruby/jruby/issues/3031 that seem to exhibit conflicting behavior...or at least the behavior is unexpected in many cases.

```
$ ruby23 -e 'p = proc { p $~; "foo" =~ /foo/ }; Thread.new {p.call}.join; Thread.new{p.call}.join'
nil
nil

$ ruby23 -e 'def foo; proc { p $~; "foo" =~ /foo/ }; end; p = foo; Thread.new {p.call}.join; Thread.new{p.call}.join'
nil
#<MatchData "foo">

$ ruby23 -e 'p = proc { p $~; "foo" =~ /foo/ }; def foo(p); Thread.new {p.call}.join; Thread.new{p.call}.join; end; foo(p)'
nil
#<MatchData "foo">

$ ruby23 -e 'class Foo; P = proc { p $~; "foo" =~ /foo/ }; def foo; Thread.new {P.call}.join; Thread.new{P.call}.join; end; end; Foo.new.foo'
nil
#<MatchData "foo">

$ ruby23 -e 'def foo; p = proc { p $~; "foo" =~ /foo/ }; Thread.new {p.call}.join; Thread.new{p.call}.join; end; foo'
nil
nil

$ ruby23 -e 'def foo; p = proc { p $~; "foo" =~ /foo/ }; bar(p); end; def bar(p); Thread.new {p.call}.join; Thread.new{p.call}.join; end; foo'
nil
#<MatchData "foo">
```

These cases exhibit some oddities in whether $~ (and presumably $_) are shared across threads.

The immediate thought is that they should be both frame and thread-local...but ko1 points out that such a change would break cases like this:

```
def foo
  /foo/ =~ 'foo'
  Proc.new{
    p $~
  }
end

Thread.new{
  foo.call
}.join
```

So there's a clear conflict here. Users sometimes expect the $~ value to be shared across threads (at least for read, as in ko1's example) and sometimes do not want it shared at all (as in the case of https://github.com/jruby/jruby/issues/3031

Now we discuss.



-- 
https://bugs.ruby-lang.org/