[ruby-core:90031] [Ruby trunk Bug#15337] String#each_grapheme_cluster wrongly splits "\r\n"

[ruby-core:90031] [Ruby trunk Bug#15337] String#each_grapheme_cluster wrongly splits "\r\n"

[duerst]

Issue #15337 has been reported by duerst (Martin Dürst).

----------------------------------------
Bug #15337: String#each_grapheme_cluster wrongly splits "\r\n"
https://bugs.ruby-lang.org/issues/15337

* Author: duerst (Martin Dürst)
* Status: Open
* Priority: Normal
* Assignee: naruse (Yui NARUSE)
* Target version: 2.6
* ruby -v: ruby 2.6.0dev (2018-11-24 trunk 65950) [x86_64-linux]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
The method `each_grapheme_cluster` on the `String` class wrongly splits CRLF into two clusters, as follows:

```
[duerst@stazersee ruby2]$ ./ruby -e 'puts "\r\n".each_grapheme_cluster.to_a.inspect'
["\r", "\n"]
[duerst@stazersee ruby2]$ ./ruby -v
ruby 2.6.0dev (2018-11-24 trunk 65950) [x86_64-linux]
```

The expected result in the above case is:
```
["\r\n"]
```

Except for the test cases that contain (unpaired) surrogates, which don't apply for us, this is the only test case that fails when testing with the data at http://www.unicode.org/Public/10.0.0/ucd/auxiliary/GraphemeBreakTest.txt. I have a test script using all that data that I plan to commit as soon as this bug is dealt with.

I would like to make sure that this is correct for Unicode 10.0.0 before moving to Unicode 11.0.0. In both specifications, the CRLF case is listed explicitly first in the specification.

I will try to find out how to fix this by myself, but would definitely appreciate help.





-- 
https://bugs.ruby-lang.org/

[ruby-core:90035] [Ruby trunk Bug#15337] String#each_grapheme_cluster wrongly splits "\r\n"

[duerst]

Issue #15337 has been updated by duerst (Martin Dürst).


Once we have a fix for this bug, it should be backported to all the versions that implement \X and that we still support.

----------------------------------------
Bug #15337: String#each_grapheme_cluster wrongly splits "\r\n"
https://bugs.ruby-lang.org/issues/15337#change-75143

* Author: duerst (Martin Dürst)
* Status: Closed
* Priority: Normal
* Assignee: naruse (Yui NARUSE)
* Target version: 2.6
* ruby -v: ruby 2.6.0dev (2018-11-24 trunk 65950) [x86_64-linux]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
The method `each_grapheme_cluster` on the `String` class wrongly splits CRLF into two clusters, as follows:

```
[duerst@stazersee ruby2]$ ./ruby -e 'puts "\r\n".each_grapheme_cluster.to_a.inspect'
["\r", "\n"]
[duerst@stazersee ruby2]$ ./ruby -v
ruby 2.6.0dev (2018-11-24 trunk 65950) [x86_64-linux]
```

The expected result in the above case is:
```
["\r\n"]
```

Except for the test cases that contain (unpaired) surrogates, which don't apply for us, this is the only test case that fails when testing with the data at http://www.unicode.org/Public/10.0.0/ucd/auxiliary/GraphemeBreakTest.txt. I have a test script using all that data that I plan to commit as soon as this bug is dealt with.

I would like to make sure that this is correct for Unicode 10.0.0 before moving to Unicode 11.0.0. In both specifications, the CRLF case is listed explicitly first in the specification.

I will try to find out how to fix this by myself, but would definitely appreciate help.





-- 
https://bugs.ruby-lang.org/

[ruby-core:90121] [Ruby trunk Bug#15337] String#each_grapheme_cluster wrongly splits "\r\n"

[nagachika00]

Issue #15337 has been updated by nagachika (Tomoyuki Chikanaga).

Backport changed from 2.3: DONTNEED, 2.4: REQUIRED, 2.5: REQUIRED to 2.3: DONTNEED, 2.4: REQUIRED, 2.5: DONE

ruby_2_5 r66073 merged revision(s) 65954,65955,65958.

----------------------------------------
Bug #15337: String#each_grapheme_cluster wrongly splits "\r\n"
https://bugs.ruby-lang.org/issues/15337#change-75243

* Author: duerst (Martin Dürst)
* Status: Closed
* Priority: Normal
* Assignee: naruse (Yui NARUSE)
* Target version: 2.6
* ruby -v: ruby 2.6.0dev (2018-11-24 trunk 65950) [x86_64-linux]
* Backport: 2.3: DONTNEED, 2.4: REQUIRED, 2.5: DONE
----------------------------------------
The method `each_grapheme_cluster` on the `String` class wrongly splits CRLF into two clusters, as follows:

```
[duerst@stazersee ruby2]$ ./ruby -e 'puts "\r\n".each_grapheme_cluster.to_a.inspect'
["\r", "\n"]
[duerst@stazersee ruby2]$ ./ruby -v
ruby 2.6.0dev (2018-11-24 trunk 65950) [x86_64-linux]
```

The expected result in the above case is:
```
["\r\n"]
```

Except for the test cases that contain (unpaired) surrogates, which don't apply for us, this is the only test case that fails when testing with the data at http://www.unicode.org/Public/10.0.0/ucd/auxiliary/GraphemeBreakTest.txt. I have a test script using all that data that I plan to commit as soon as this bug is dealt with.

I would like to make sure that this is correct for Unicode 10.0.0 before moving to Unicode 11.0.0. In both specifications, the CRLF case is listed explicitly first in the specification.

I will try to find out how to fix this by myself, but would definitely appreciate help.





-- 
https://bugs.ruby-lang.org/