[ruby-core:90149] [Ruby trunk Bug#15343] String#each_grapheme_cluster wrongly splits some emoji (genie, zombie, wrestling)

[duerst@it.aoyama.ac.jp]

Issue #15343 has been updated by duerst (Martin Dürst).


Some data points from a discussion between @naruse and myself:

- Up to elf (U+1F9DD) is Emoji_Modifier_Base, but genie (U+1F9DE) isn't.

- Emoji_Modifier only includes skin tones (U+1F3FB-1F3FF, light skin tone..dark skin tone)

- For experts, that seems to make sense, because there are apparently light and dark elves, but all the zombies have the same half-dead skin color.

- For 'wrestling' again, it doesn't allow skin colors.

- So the error seems to appear when an emoji takes male/female specifiers, but isn't allowed to take skin tones.

- As we are going to rewrite the underlying implementation (function `node_extended_grapheme_cluster` in regparse.c), we may not care to fix this bug anymore. But if somebody finds a fix, they may want to apply it to older versions of Ruby (2.5 and 2.4).



----------------------------------------
Bug #15343: String#each_grapheme_cluster wrongly splits some emoji (genie, zombie, wrestling)
https://bugs.ruby-lang.org/issues/15343#change-75265

* Author: duerst (Martin Dürst)
* Status: Open
* Priority: Normal
* Assignee: naruse (Yui NARUSE)
* Target version: 2.6
* ruby -v: ruby 2.6.0dev (2018-11-26 trunk 65989) [x86_64-cygwin]
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
All the codepoint combinations that turn up in the various emoji files provided by Unicode (currently we use those at https://www.unicode.org/Public/emoji/5.0/) are recognized as grapheme clusters by `String#each_grapheme_cluster`, except those relating to genies, zombies, and wrestling (THIS IS NOT A JOKE!).

Taking an example from https://www.unicode.org/Public/emoji/5.0/emoji-zwj-sequences.txt, line 396:

```
$ ./ruby -e '"\u{1F9DE 200D 2640 FE0F}".each_grapheme_cluster.to_a.length.display'
2
```
The correct result is 1, not 2. The sequence of codepoints represents a woman genie.

I will commit the file test/ruby/enc/test_emoji_breaks.rb, which excludes genie, zombie, and wrestling emoji to make sure the tests pass.

I would like to make sure that this is correct for Unicode 10.0.0 before moving to Unicode 11.0.0. I will try to find out how to fix this by myself, but would definitely appreciate help.




-- 
https://bugs.ruby-lang.org/