[ruby-core:68675] [Ruby trunk - Bug #11014] [Open] String#partition doesn't return correct result on zero-width match

[ruby-core:68675] [Ruby trunk - Bug #11014] [Open] String#partition doesn't return correct result on zero-width match

[janko.marohnic]

Issue #11014 has been reported by Janko Marohnić.

----------------------------------------
Bug #11014: String#partition doesn't return correct result on zero-width match
https://bugs.ruby-lang.org/issues/11014

* Author: Janko Marohnić
* Status: Open
* Priority: Normal
* Assignee: 
* ruby -v: ruby 2.2.1p85 (2015-02-26 revision 49769) [x86_64-darwin14]
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------
First, to see how String#match works on my example:

~~~
match = "foo".match(/^=*/)
match.pre_match  #=> ""
match[0]         #=> ""
match.post_match #=> "foo"
~~~

Now, if I used String#partition instead of match, I'd expect to get `["", "", "foo"]` (pre_match, match, post_match). However

~~~
"foo".partition(/^=*/) #=> ["foo", "", ""]
~~~

String#rpartition returns the correct result (with the same regex).



-- 
https://bugs.ruby-lang.org/

[ruby-core:68927] [Ruby trunk - Bug #11014] [Assigned] String#partition doesn't return correct result on zero-width match

[nobu]

Issue #11014 has been updated by Nobuyoshi Nakada.

Description updated
Status changed from Open to Assigned
Assignee set to Yukihiro Matsumoto

These methods have been taken from Python, and seems same in Python.
I'm not sure what's the rationale of this behavior.

----------------------------------------
Bug #11014: String#partition doesn't return correct result on zero-width match
https://bugs.ruby-lang.org/issues/11014#change-52193

* Author: Janko Marohnić
* Status: Assigned
* Priority: Normal
* Assignee: Yukihiro Matsumoto
* ruby -v: ruby 2.2.1p85 (2015-02-26 revision 49769) [x86_64-darwin14]
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------
First, to see how `String#match` works on my example:

~~~ruby
match = "foo".match(/^=*/)
match.pre_match  #=> ""
match[0]         #=> ""
match.post_match #=> "foo"
~~~

Now, if I used `String#partition` instead of `match`, I'd expect to get `["", "", "foo"]` (pre_match, match, post_match). However

~~~ruby
"foo".partition(/^=*/) #=> ["foo", "", ""]
~~~

`String#rpartition` returns the correct result (with the same regex).



-- 
https://bugs.ruby-lang.org/

[ruby-core:96701] [Ruby master Bug#11014] String#partition doesn't return correct result on zero-width match

[sawadatsuyoshi]

Issue #11014 has been updated by sawa (Tsuyoshi Sawada).


The problem is not just for `partition`, but is also involves `split` and `scan`.

I think your regex `/^=*/` is unnecessarily complex. Your point can be shown by `/\A/`, which is simpler.

I tried with four regex patterns `/\A/`, `/\A.*/`, `/\z/`, `/.*\z/`, and compared methods `split`, `partition`, `scan`. The result of the first example in each group below matches the second and the third, and the fourth one matches the middle element. So far, so good.

```ruby
"foo".match(/\z/).then{[_1.pre_match, _1[0], _1.post_match]} # => ["foo", "", ""]
"foo".split(/(\z)/, -1) # => ["foo", "", ""]
"foo".partition(/\z/) # => ["foo", "", ""]
"foo".scan(/\z/) # => [""]
```

```ruby
"foo".match(/\A.*/).then{[_1.pre_match, _1[0], _1.post_match]} # => ["", "foo", ""]
"foo".split(/(\A.*)/, -1) # => ["", "foo", ""]
"foo".partition(/\A.*/) # => ["", "foo", ""]
"foo".scan(/\A.*/) # => ["foo"]
```

In the following, we see problems:

```ruby
"foo".match(/\A/).then{[_1.pre_match, _1[0], _1.post_match]} # => ["", "", "foo"]
"foo".split(/(\A)/, -1) # => ["foo"]
"foo".partition(/\A/) # => ["foo", "", ""]
"foo".scan(/\A/) # => [""]
```

```ruby
"foo".match(/.*\z/).then{[_1.pre_match, _1[0], _1.post_match]} # => ["", "foo", ""]
"foo".split(/(.*\z)/, -1) # => ["", "foo", ""]
"foo".partition(/.*\z/) # => ["", "foo", ""]
"foo".scan(/.*\z/) # => ["foo", ""]
```

The problematic cases and their expected values (in terms of consistency) are:


```ruby
"foo".split(/(\A)/, -1) # => ["foo"], expected [ "", "", "foo"]
"foo".partition(/\A/) # => ["foo", "", ""], expected ["", "", "foo"]
"foo".scan(/.*\z/) # => ["foo", ""], expected ["foo"]
```

The case described in the issue is the second case above.

----------------------------------------
Bug #11014: String#partition doesn't return correct result on zero-width match
https://bugs.ruby-lang.org/issues/11014#change-83687

* Author: janko (Janko Marohnić)
* Status: Assigned
* Priority: Normal
* Assignee: matz (Yukihiro Matsumoto)
* Target version: 
* ruby -v: ruby 2.2.1p85 (2015-02-26 revision 49769) [x86_64-darwin14]
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------
First, to see how `String#match` works on my example:

~~~ruby
match = "foo".match(/^=*/)
match.pre_match  #=> ""
match[0]         #=> ""
match.post_match #=> "foo"
~~~

Now, if I used `String#partition` instead of `match`, I'd expect to get `["", "", "foo"]` (pre_match, match, post_match). However

~~~ruby
"foo".partition(/^=*/) #=> ["foo", "", ""]
~~~

`String#rpartition` returns the correct result (with the same regex).



-- 
https://bugs.ruby-lang.org/

[ruby-core:96702] [Ruby master Bug#11014] String#partition doesn't return correct result on zero-width match

[daniel]

Issue #11014 has been updated by Dan0042 (Daniel DeLorme).


IIRC this has to do with zero-length matches being ignored in certain conditions, in particular having to do with repeating/multiple matches.

if `"foo".split(/\A/)` was `["","foo"]`
then `"foo".split(//)` would have to be `["","f","o","o"]`
and `"foo".split(/\G/)` could result in infinite loop matching `["","","","","",..."foo"]`

But I don't understand why `partition` doesn't behave like `match`.

Note that gsub has different behavior:
`"foo".gsub(/\G/,'_') #=> "_f_o_o_"`
`"foo".gsub(//,'_') #=> "_f_o_o_"`

explained better than I ever could:
https://www.regular-expressions.info/zerolength.html

----------------------------------------
Bug #11014: String#partition doesn't return correct result on zero-width match
https://bugs.ruby-lang.org/issues/11014#change-83688

* Author: janko (Janko Marohnić)
* Status: Assigned
* Priority: Normal
* Assignee: matz (Yukihiro Matsumoto)
* Target version: 
* ruby -v: ruby 2.2.1p85 (2015-02-26 revision 49769) [x86_64-darwin14]
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------
First, to see how `String#match` works on my example:

~~~ruby
match = "foo".match(/^=*/)
match.pre_match  #=> ""
match[0]         #=> ""
match.post_match #=> "foo"
~~~

Now, if I used `String#partition` instead of `match`, I'd expect to get `["", "", "foo"]` (pre_match, match, post_match). However

~~~ruby
"foo".partition(/^=*/) #=> ["foo", "", ""]
~~~

`String#rpartition` returns the correct result (with the same regex).



-- 
https://bugs.ruby-lang.org/

[ruby-core:96830] [Ruby master Bug#11014] String#partition doesn't return correct result on zero-width match

[mame]

Issue #11014 has been updated by mame (Yusuke Endoh).


We'd like to focus on String#partition in this ticket.

IMO, String#scan and #split are heavily used so they should not change just for consistency reason.  Please create another ticket if you really need to discuss.  And a patch suggestion is welcome.

----------------------------------------
Bug #11014: String#partition doesn't return correct result on zero-width match
https://bugs.ruby-lang.org/issues/11014#change-83829

* Author: janko (Janko Marohnić)
* Status: Assigned
* Priority: Normal
* Assignee: matz (Yukihiro Matsumoto)
* Target version: 
* ruby -v: ruby 2.2.1p85 (2015-02-26 revision 49769) [x86_64-darwin14]
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------
First, to see how `String#match` works on my example:

~~~ruby
match = "foo".match(/^=*/)
match.pre_match  #=> ""
match[0]         #=> ""
match.post_match #=> "foo"
~~~

Now, if I used `String#partition` instead of `match`, I'd expect to get `["", "", "foo"]` (pre_match, match, post_match). However

~~~ruby
"foo".partition(/^=*/) #=> ["foo", "", ""]
~~~

`String#rpartition` returns the correct result (with the same regex).



-- 
https://bugs.ruby-lang.org/

[ruby-core:96885] [Ruby master Bug#11014] String#partition doesn't return correct result on zero-width match

[akr]

Issue #11014 has been updated by akr (Akira Tanaka).


nobu (Nobuyoshi Nakada) wrote:
> These methods have been taken from Python, and seems same in Python.
> I'm not sure what's the rationale of this behavior.

I couldn't confirm it.

```
% python3
Python 3.7.3 (default, Apr  3 2019, 05:39:12) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> "abc".partition("")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: empty separator
>>> 
```

The empty separator causes an error in Python.

----------------------------------------
Bug #11014: String#partition doesn't return correct result on zero-width match
https://bugs.ruby-lang.org/issues/11014#change-83897

* Author: janko (Janko Marohnić)
* Status: Assigned
* Priority: Normal
* Assignee: matz (Yukihiro Matsumoto)
* Target version: 
* ruby -v: ruby 2.2.1p85 (2015-02-26 revision 49769) [x86_64-darwin14]
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------
First, to see how `String#match` works on my example:

~~~ruby
match = "foo".match(/^=*/)
match.pre_match  #=> ""
match[0]         #=> ""
match.post_match #=> "foo"
~~~

Now, if I used `String#partition` instead of `match`, I'd expect to get `["", "", "foo"]` (pre_match, match, post_match). However

~~~ruby
"foo".partition(/^=*/) #=> ["foo", "", ""]
~~~

`String#rpartition` returns the correct result (with the same regex).



-- 
https://bugs.ruby-lang.org/

[ruby-core:96887] [Ruby master Bug#11014] String#partition doesn't return correct result on zero-width match

[akr]

Issue #11014 has been updated by akr (Akira Tanaka).


I feel the current behavior is just a bug and `"abc".partition(//)` should return `["", "", "abc"]` instead `["abc", "", ""]`.

----------------------------------------
Bug #11014: String#partition doesn't return correct result on zero-width match
https://bugs.ruby-lang.org/issues/11014#change-83899

* Author: janko (Janko Marohnić)
* Status: Assigned
* Priority: Normal
* Assignee: matz (Yukihiro Matsumoto)
* Target version: 
* ruby -v: ruby 2.2.1p85 (2015-02-26 revision 49769) [x86_64-darwin14]
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------
First, to see how `String#match` works on my example:

~~~ruby
match = "foo".match(/^=*/)
match.pre_match  #=> ""
match[0]         #=> ""
match.post_match #=> "foo"
~~~

Now, if I used `String#partition` instead of `match`, I'd expect to get `["", "", "foo"]` (pre_match, match, post_match). However

~~~ruby
"foo".partition(/^=*/) #=> ["foo", "", ""]
~~~

`String#rpartition` returns the correct result (with the same regex).



-- 
https://bugs.ruby-lang.org/