[ruby-core:107892] [Ruby master Bug#18631] Range check breaks multiplex backreferences in regular expressions

[ruby-core:107892] [Ruby master Bug#18631] Range check breaks multiplex backreferences in regular expressions

[jirkamarsik (Jirka Marsik)]

Issue #18631 has been reported by jirkamarsik (Jirka Marsik).

----------------------------------------
Bug #18631: Range check breaks multiplex backreferences in regular expressions
https://bugs.ruby-lang.org/issues/18631

* Author: jirkamarsik (Jirka Marsik)
* Status: Open
* Priority: Normal
* ruby -v: ruby 2.7.5p203 (2021-11-24) [x86_64-linux]
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
The regular expression engine can sometimes produce wrong results when using multiplex backreferences near the end of the input string.

``` ruby
irb(main):001:0> /(?<x>a)(?<x>aa)\k<x>/.match("aaaaa")
=> #<MatchData "aaaaa" x:"a" x:"aa">
irb(main):002:0> /(?<x>a)(?<x>aa)\k<x>/.match("aaaa")
=> nil
irb(main):003:0> /(?<x>a)(?<x>aa)\k<x>/.match("aaaab")
=> #<MatchData "aaaa" x:"a" x:"aa">
```

The second and third calls to `match` should produce the same result.

The cause is the `DATA_ENSURE(n)` macro in the `OP_BACKREF_MULTI` case in `regexec.c` (https://github.com/ruby/ruby/blob/master/regexec.c#L2646). Instead of using `continue` to try to match the other referents for the backref (as all the other branches do), the `DATA_ENSURE` macro uses `goto fail` and so skips the other referents of the multiplex backref. This means that after failing the range check, no other referent can match. By extending the input string in the third example above, we have avoided the bug and got the correct result.



-- 
https://bugs.ruby-lang.org/

[ruby-core:108057] [Ruby master Bug#18631] Range check breaks multiplex backreferences in regular expressions

[jeremyevans0 (Jeremy Evans)]

Issue #18631 has been updated by jeremyevans0 (Jeremy Evans).


I don't know much about this code, but the approach described does fix the issue.  I submitted a pull request for it: https://github.com/ruby/ruby/pull/5710

----------------------------------------
Bug #18631: Range check breaks multiplex backreferences in regular expressions
https://bugs.ruby-lang.org/issues/18631#change-97019

* Author: jirkamarsik (Jirka Marsik)
* Status: Open
* Priority: Normal
* ruby -v: ruby 2.7.5p203 (2021-11-24) [x86_64-linux]
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
The regular expression engine can sometimes produce wrong results when using multiplex backreferences near the end of the input string.

``` ruby
irb(main):001:0> /(?<x>a)(?<x>aa)\k<x>/.match("aaaaa")
=> #<MatchData "aaaaa" x:"a" x:"aa">
irb(main):002:0> /(?<x>a)(?<x>aa)\k<x>/.match("aaaa")
=> nil
irb(main):003:0> /(?<x>a)(?<x>aa)\k<x>/.match("aaaab")
=> #<MatchData "aaaa" x:"a" x:"aa">
```

The second and third calls to `match` should produce the same result.

The cause is the `DATA_ENSURE(n)` macro in the `OP_BACKREF_MULTI` case in `regexec.c` (https://github.com/ruby/ruby/blob/master/regexec.c#L2646). Instead of using `continue` to try to match the other referents for the backref (as all the other branches do), the `DATA_ENSURE` macro uses `goto fail` and so skips the other referents of the multiplex backref. This means that after failing the range check, no other referent can match. By extending the input string in the third example above, we have avoided the bug and got the correct result.



-- 
https://bugs.ruby-lang.org/

[ruby-core:109837] [Ruby master Bug#18631] Range check breaks multiplex backreferences in regular expressions

[nagachika (Tomoyuki Chikanaga)]

Issue #18631 has been updated by nagachika (Tomoyuki Chikanaga).

Backport changed from 2.7: REQUIRED, 3.0: REQUIRED, 3.1: REQUIRED to 2.7: REQUIRED, 3.0: REQUIRED, 3.1: DONE

ruby_3_1 6b8bf6ba5d1893b3c933c2de3ada1eb59a94b644 merged revision(s) 6d3f447aecfb56f7d3edbdf9cc68e748e150d7d8.

----------------------------------------
Bug #18631: Range check breaks multiplex backreferences in regular expressions
https://bugs.ruby-lang.org/issues/18631#change-99074

* Author: jirkamarsik (Jirka Marsik)
* Status: Closed
* Priority: Normal
* ruby -v: ruby 2.7.5p203 (2021-11-24) [x86_64-linux]
* Backport: 2.7: REQUIRED, 3.0: REQUIRED, 3.1: DONE
----------------------------------------
The regular expression engine can sometimes produce wrong results when using multiplex backreferences near the end of the input string.

``` ruby
irb(main):001:0> /(?<x>a)(?<x>aa)\k<x>/.match("aaaaa")
=> #<MatchData "aaaaa" x:"a" x:"aa">
irb(main):002:0> /(?<x>a)(?<x>aa)\k<x>/.match("aaaa")
=> nil
irb(main):003:0> /(?<x>a)(?<x>aa)\k<x>/.match("aaaab")
=> #<MatchData "aaaa" x:"a" x:"aa">
```

The second and third calls to `match` should produce the same result.

The cause is the `DATA_ENSURE(n)` macro in the `OP_BACKREF_MULTI` case in `regexec.c` (https://github.com/ruby/ruby/blob/master/regexec.c#L2646). Instead of using `continue` to try to match the other referents for the backref (as all the other branches do), the `DATA_ENSURE` macro uses `goto fail` and so skips the other referents of the multiplex backref. This means that after failing the range check, no other referent can match. By extending the input string in the third example above, we have avoided the bug and got the correct result.



-- 
https://bugs.ruby-lang.org/