[ruby-core:92412] [Ruby trunk Bug#15792] GC can leave strings used as hash keys in a corrupted state

[ruby-core:92412] [Ruby trunk Bug#15792] GC can leave strings used as hash keys in a corrupted state

[jean.boussier]

Issue #15792 has been reported by byroot (Jean Boussier).

----------------------------------------
Bug #15792: GC can leave strings used as hash keys in a corrupted state
https://bugs.ruby-lang.org/issues/15792

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 2.6.2
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
The following script showcase the issue:

```
#!/usr/bin/env ruby --disable-gems
a = ('a' * 24).encode(Encoding::ASCII).gsub('x', '')
b = ('b' * 24).encode(Encoding::ASCII).gsub('x', '')

hash = {}
hash[a] = true
hash[b] = true

puts "Bebore garbage_collection: a=#{a.inspect} b=#{b.inspect}"
4.times { GC.start }
puts "After garbage_collection: a=#{a.inspect} b=#{b.inspect}"
```

Expected output:

```
Bebore garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
After garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
```

Actual output:

```
Ruby: 2.6.2
Bebore garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
After garbage_collection: a="}\x0Eu\xDB\xFC\a\x00\x80\xE9\ru\xDB\xFC\a\x00\x10\x04\x00aaaaaa" b="\x00\x00\x00\x00\x00\x00\x00\xC0\x00\x00\x00\x00\x00\x00\x00\xC0\x02\x00bbbbbb"
```

We reduced the repro script as much as we could, both the `.encode(ASCII)` and the `gsub` are necessary for the bug to manifest itself.

We also used `ObjectSpace.dump()` to analyze the corrupted string.

```
b = "shared":true, "encoding":"US-ASCII", "references":["0x7faf4a01aeb8"]
0x7faf4a01aeb8 = "frozen":true, "fstring":true, "bytesize":24, "value":"bbbbbbbbbbbbbbbbbbbbbbbb", "encoding":"US-ASCII"
```

Big thanks to Édouard Chin who did most of the initial repro reduction.



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>

[ruby-core:92413] [Ruby trunk Bug#15792] GC can leave strings used as hash keys in a corrupted state

[jean.boussier]

Issue #15792 has been updated by byroot (Jean Boussier).


Actually, even simpler repro script:

```
#!/usr/bin/env ruby --disable-gems
a = ('a' * 24).encode(Encoding::ASCII).gsub('x', '')

hash = {}
hash[a] = true

puts "Before garbage_collection: a=#{a.inspect}"
4.times { GC.start }
puts "After garbage_collection: a=#{a.inspect}"
```


----------------------------------------
Bug #15792: GC can leave strings used as hash keys in a corrupted state
https://bugs.ruby-lang.org/issues/15792#change-77770

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 2.6.2
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
The following script showcase the issue:

```
#!/usr/bin/env ruby --disable-gems
a = ('a' * 24).encode(Encoding::ASCII).gsub('x', '')
b = ('b' * 24).encode(Encoding::ASCII).gsub('x', '')

hash = {}
hash[a] = true
hash[b] = true

puts "Bebore garbage_collection: a=#{a.inspect} b=#{b.inspect}"
4.times { GC.start }
puts "After garbage_collection: a=#{a.inspect} b=#{b.inspect}"
```

Expected output:

```
Bebore garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
After garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
```

Actual output:

```
Ruby: 2.6.2
Bebore garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
After garbage_collection: a="}\x0Eu\xDB\xFC\a\x00\x80\xE9\ru\xDB\xFC\a\x00\x10\x04\x00aaaaaa" b="\x00\x00\x00\x00\x00\x00\x00\xC0\x00\x00\x00\x00\x00\x00\x00\xC0\x02\x00bbbbbb"
```

We reduced the repro script as much as we could, both the `.encode(ASCII)` and the `gsub` are necessary for the bug to manifest itself.

We also used `ObjectSpace.dump()` to analyze the corrupted string.

```
b = "shared":true, "encoding":"US-ASCII", "references":["0x7faf4a01aeb8"]
0x7faf4a01aeb8 = "frozen":true, "fstring":true, "bytesize":24, "value":"bbbbbbbbbbbbbbbbbbbbbbbb", "encoding":"US-ASCII"
```

Big thanks to Édouard Chin who did most of the initial repro reduction.



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>

[ruby-core:92421] [Ruby trunk Bug#15792] GC can leave strings used as hash keys in a corrupted state

[jean.boussier]

Issue #15792 has been updated by byroot (Jean Boussier).


Édouad Chin bisected the script against the MRI repository, and found https://bugs.ruby-lang.org/issues/15251 the first commit to trigger the problem.

That being said it's unlikely to really be the source of the bug. It's much more likely to be in the `fstring` function.



----------------------------------------
Bug #15792: GC can leave strings used as hash keys in a corrupted state
https://bugs.ruby-lang.org/issues/15792#change-77778

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 2.6.2
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
The following script showcase the issue:

```
#!/usr/bin/env ruby --disable-gems
a = ('a' * 24).encode(Encoding::ASCII).gsub('x', '')
b = ('b' * 24).encode(Encoding::ASCII).gsub('x', '')

hash = {}
hash[a] = true
hash[b] = true

puts "Bebore garbage_collection: a=#{a.inspect} b=#{b.inspect}"
4.times { GC.start }
puts "After garbage_collection: a=#{a.inspect} b=#{b.inspect}"
```

Expected output:

```
Bebore garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
After garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
```

Actual output:

```
Ruby: 2.6.2
Bebore garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
After garbage_collection: a="}\x0Eu\xDB\xFC\a\x00\x80\xE9\ru\xDB\xFC\a\x00\x10\x04\x00aaaaaa" b="\x00\x00\x00\x00\x00\x00\x00\xC0\x00\x00\x00\x00\x00\x00\x00\xC0\x02\x00bbbbbb"
```

We reduced the repro script as much as we could, both the `.encode(ASCII)` and the `gsub` are necessary for the bug to manifest itself.

We also used `ObjectSpace.dump()` to analyze the corrupted string.

```
b = "shared":true, "encoding":"US-ASCII", "references":["0x7faf4a01aeb8"]
0x7faf4a01aeb8 = "frozen":true, "fstring":true, "bytesize":24, "value":"bbbbbbbbbbbbbbbbbbbbbbbb", "encoding":"US-ASCII"
```

Big thanks to Édouard Chin who did most of the initial repro reduction.



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>

[ruby-core:92424] [Ruby trunk Bug#15792] GC can leave strings used as hash keys in a corrupted state

[s.wanabe]

Issue #15792 has been updated by wanabe (_ wanabe).


I guess it is caused by the combination of `str_duplicate` and `rb_fstring` from r52074.

```
$ cat bug15792.rb 
a = ('a' * 24).b.strip

->{ eval "", binding, a, 1 }.call

puts "Before garbage_collection: a=#{a.inspect}"
4.times { GC.start }
puts "After garbage_collection: a=#{a.inspect}"

$ ./miniruby -v bug15792.rb 
ruby 2.3.0dev (2015-10-07 trunk 52074) [x86_64-linux]
Before garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa"
After garbage_collection: a="\x00B\xFB\xCCmU\x00\x00\x10\x10\xE7\xCCmU\x00\x00aaaaaaaa"
```

----------------------------------------
Bug #15792: GC can leave strings used as hash keys in a corrupted state
https://bugs.ruby-lang.org/issues/15792#change-77780

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 2.6.2
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
The following script showcase the issue:

```
#!/usr/bin/env ruby --disable-gems
a = ('a' * 24).encode(Encoding::ASCII).gsub('x', '')
b = ('b' * 24).encode(Encoding::ASCII).gsub('x', '')

hash = {}
hash[a] = true
hash[b] = true

puts "Bebore garbage_collection: a=#{a.inspect} b=#{b.inspect}"
4.times { GC.start }
puts "After garbage_collection: a=#{a.inspect} b=#{b.inspect}"
```

Expected output:

```
Bebore garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
After garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
```

Actual output:

```
Ruby: 2.6.2
Bebore garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
After garbage_collection: a="}\x0Eu\xDB\xFC\a\x00\x80\xE9\ru\xDB\xFC\a\x00\x10\x04\x00aaaaaa" b="\x00\x00\x00\x00\x00\x00\x00\xC0\x00\x00\x00\x00\x00\x00\x00\xC0\x02\x00bbbbbb"
```

We reduced the repro script as much as we could, both the `.encode(ASCII)` and the `gsub` are necessary for the bug to manifest itself.

We also used `ObjectSpace.dump()` to analyze the corrupted string.

```
b = "shared":true, "encoding":"US-ASCII", "references":["0x7faf4a01aeb8"]
0x7faf4a01aeb8 = "frozen":true, "fstring":true, "bytesize":24, "value":"bbbbbbbbbbbbbbbbbbbbbbbb", "encoding":"US-ASCII"
```

Big thanks to Édouard Chin who did most of the initial repro reduction.



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>

[ruby-core:92434] [Ruby trunk Bug#15792] GC can leave strings used as hash keys in a corrupted state

[nobu]

Issue #15792 has been updated by nobu (Nobuyoshi Nakada).

Status changed from Open to Closed

Thank you, fixed at https://git.ruby-lang.org/ruby.git/commit/?id=3f9562015e651735bfc2fdd14e8f6963b673e22a.

----------------------------------------
Bug #15792: GC can leave strings used as hash keys in a corrupted state
https://bugs.ruby-lang.org/issues/15792#change-77789

* Author: byroot (Jean Boussier)
* Status: Closed
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 2.6.2
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
The following script showcase the issue:

```
#!/usr/bin/env ruby --disable-gems
a = ('a' * 24).encode(Encoding::ASCII).gsub('x', '')
b = ('b' * 24).encode(Encoding::ASCII).gsub('x', '')

hash = {}
hash[a] = true
hash[b] = true

puts "Bebore garbage_collection: a=#{a.inspect} b=#{b.inspect}"
4.times { GC.start }
puts "After garbage_collection: a=#{a.inspect} b=#{b.inspect}"
```

Expected output:

```
Bebore garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
After garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
```

Actual output:

```
Ruby: 2.6.2
Bebore garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
After garbage_collection: a="}\x0Eu\xDB\xFC\a\x00\x80\xE9\ru\xDB\xFC\a\x00\x10\x04\x00aaaaaa" b="\x00\x00\x00\x00\x00\x00\x00\xC0\x00\x00\x00\x00\x00\x00\x00\xC0\x02\x00bbbbbb"
```

We reduced the repro script as much as we could, both the `.encode(ASCII)` and the `gsub` are necessary for the bug to manifest itself.

We also used `ObjectSpace.dump()` to analyze the corrupted string.

```
b = "shared":true, "encoding":"US-ASCII", "references":["0x7faf4a01aeb8"]
0x7faf4a01aeb8 = "frozen":true, "fstring":true, "bytesize":24, "value":"bbbbbbbbbbbbbbbbbbbbbbbb", "encoding":"US-ASCII"
```

Big thanks to Édouard Chin who did most of the initial repro reduction.



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>

[ruby-core:94173] [Ruby master Bug#15792] GC can leave strings used as hash keys in a corrupted state

[nagachika00]

Issue #15792 has been updated by nagachika (Tomoyuki Chikanaga).

Backport changed from 2.4: REQUIRED, 2.5: REQUIRED, 2.6: REQUIRED to 2.4: REQUIRED, 2.5: REQUIRED, 2.6: DONE

ruby_2_6 r67731 merged revision(s) 3f9562015e651735bfc2fdd14e8f6963b673e22a,c06ddfee878524168e4af07443217ed2f8d0954b,3b3b4a44e57dfe03ce3913009d69a33d6f6100be.

----------------------------------------
Bug #15792: GC can leave strings used as hash keys in a corrupted state
https://bugs.ruby-lang.org/issues/15792#change-80430

* Author: byroot (Jean Boussier)
* Status: Closed
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 2.6.2
* Backport: 2.4: REQUIRED, 2.5: REQUIRED, 2.6: DONE
----------------------------------------
The following script showcase the issue:

```
#!/usr/bin/env ruby --disable-gems
a = ('a' * 24).encode(Encoding::ASCII).gsub('x', '')
b = ('b' * 24).encode(Encoding::ASCII).gsub('x', '')

hash = {}
hash[a] = true
hash[b] = true

puts "Bebore garbage_collection: a=#{a.inspect} b=#{b.inspect}"
4.times { GC.start }
puts "After garbage_collection: a=#{a.inspect} b=#{b.inspect}"
```

Expected output:

```
Bebore garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
After garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
```

Actual output:

```
Ruby: 2.6.2
Bebore garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
After garbage_collection: a="}\x0Eu\xDB\xFC\a\x00\x80\xE9\ru\xDB\xFC\a\x00\x10\x04\x00aaaaaa" b="\x00\x00\x00\x00\x00\x00\x00\xC0\x00\x00\x00\x00\x00\x00\x00\xC0\x02\x00bbbbbb"
```

We reduced the repro script as much as we could, both the `.encode(ASCII)` and the `gsub` are necessary for the bug to manifest itself.

We also used `ObjectSpace.dump()` to analyze the corrupted string.

```
b = "shared":true, "encoding":"US-ASCII", "references":["0x7faf4a01aeb8"]
0x7faf4a01aeb8 = "frozen":true, "fstring":true, "bytesize":24, "value":"bbbbbbbbbbbbbbbbbbbbbbbb", "encoding":"US-ASCII"
```

Big thanks to Édouard Chin who did most of the initial repro reduction.



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>

[ruby-core:94575] [Ruby master Bug#15792] GC can leave strings used as hash keys in a corrupted state

[usa]

Issue #15792 has been updated by usa (Usaku NAKAMURA).

Backport changed from 2.4: REQUIRED, 2.5: REQUIRED, 2.6: DONE to 2.4: REQUIRED, 2.5: DONE, 2.6: DONE

ruby_2_5 r67766 merged revision(s) 3f9562015e651735bfc2fdd14e8f6963b673e22a,c06ddfee878524168e4af07443217ed2f8d0954b,3b3b4a44e5.

----------------------------------------
Bug #15792: GC can leave strings used as hash keys in a corrupted state
https://bugs.ruby-lang.org/issues/15792#change-81033

* Author: byroot (Jean Boussier)
* Status: Closed
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 2.6.2
* Backport: 2.4: REQUIRED, 2.5: DONE, 2.6: DONE
----------------------------------------
The following script showcase the issue:

```
#!/usr/bin/env ruby --disable-gems
a = ('a' * 24).encode(Encoding::ASCII).gsub('x', '')
b = ('b' * 24).encode(Encoding::ASCII).gsub('x', '')

hash = {}
hash[a] = true
hash[b] = true

puts "Bebore garbage_collection: a=#{a.inspect} b=#{b.inspect}"
4.times { GC.start }
puts "After garbage_collection: a=#{a.inspect} b=#{b.inspect}"
```

Expected output:

```
Bebore garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
After garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
```

Actual output:

```
Ruby: 2.6.2
Bebore garbage_collection: a="aaaaaaaaaaaaaaaaaaaaaaaa" b="bbbbbbbbbbbbbbbbbbbbbbbb"
After garbage_collection: a="}\x0Eu\xDB\xFC\a\x00\x80\xE9\ru\xDB\xFC\a\x00\x10\x04\x00aaaaaa" b="\x00\x00\x00\x00\x00\x00\x00\xC0\x00\x00\x00\x00\x00\x00\x00\xC0\x02\x00bbbbbb"
```

We reduced the repro script as much as we could, both the `.encode(ASCII)` and the `gsub` are necessary for the bug to manifest itself.

We also used `ObjectSpace.dump()` to analyze the corrupted string.

```
b = "shared":true, "encoding":"US-ASCII", "references":["0x7faf4a01aeb8"]
0x7faf4a01aeb8 = "frozen":true, "fstring":true, "bytesize":24, "value":"bbbbbbbbbbbbbbbbbbbbbbbb", "encoding":"US-ASCII"
```

Big thanks to Édouard Chin who did most of the initial repro reduction.



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>