From: mk <noreply@ruby-lang.org>
To: ruby-core@ml.ruby-lang.org
Subject: [ruby-core:111063] [Ruby master Bug#19156] ObjectSpace.dump_all segfault during string inspection
Date: Tue, 29 Nov 2022 14:37:44 +0000 (UTC) [thread overview]
Message-ID: <redmine.journal-100323.20221129143743.51782@ruby-lang.org> (raw)
In-Reply-To: redmine.issue-19156.20221128143052.51782@ruby-lang.org
Issue #19156 has been updated by mk (Matthias Käppler).
It looks like I can't do anything useful with that address:
```
(gdb) p *0x7fced23f4000
Cannot access memory at address 0x7fced23f4000
```
This is getting mysterious. I looked at process maps to see what kind of memory region this is, and it looks like it's a memory-mapped file from `prometheus-client-mmap`:
```
pmap -x 216
...
00007fced23f3000 4 4 0 rw-s- gauge_max_puma_0-0.db
00007fced23f5000 4 4 0 r---- wait.so
...
```
I also cannot dump this region, but I can print the value for the object starting at `0x...3000`:
```
(gdb) dump memory /tmp/memdmp 0x7fced23f3000 0x7fced23f5000
Cannot access memory at address 0x7fced23f4000
```
but:
```
(gdb) p *0x7fced23f3000
$1 = 104
```
That doesn't tell me anything, but I know this library uses a binary format to serialize metric samples into JSON strings from Ruby hashes using a C extension. The only Ruby strings in here are metric names and labels IIRC.
I will investigate more in this direction. It indeed does sound like it is not strictly a bug with MRI, though I wonder if it should be more resilient to these kind of rogue objects and just filter them out?
----------------------------------------
Bug #19156: ObjectSpace.dump_all segfault during string inspection
https://bugs.ruby-lang.org/issues/19156#change-100323
* Author: mk (Matthias Käppler)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.0.4p208 (2022-04-12 revision 3fa771dded) [x86_64-linux]
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
I am working on a feature that would allow our application to capture heap dumps during shutdown for later inspection.
These heap dumps are captured via `ObjectSpace.dump_all(output: io)`. While walking the object space, MRI occasionally segfaults while inspecting string objects in `search_nonascii` of `string.c`:
```
/usr/local/lib/ruby/3.0.0/objspace.rb:87: [BUG] Segmentation fault at 0x00007efee4201000
ruby 3.0.4p208 (2022-04-12 revision 3fa771dded) [x86_64-linux]
...
-- Control frame information -----------------------------------------------
c:0053 p:---- s:0312 e:000311 CFUNC :_dump_all
c:0052 p:0130 s:0305 e:000304 METHOD /usr/local/lib/ruby/3.0.0/objspace.rb:87
c:0051 p:0023 s:0295 e:000294 METHOD /home/git/gitlab/lib/gitlab/memory/reports/heap_dump.rb:26
...
-- C level backtrace information -------------------------------------------
/usr/local/lib/libruby.so.3.0(rb_print_backtrace+0x11) [0x7efee4ad0c5e] vm_dump.c:758
/usr/local/lib/libruby.so.3.0(rb_vm_bugreport) vm_dump.c:998
/usr/local/lib/libruby.so.3.0(rb_bug_for_fatal_signal+0xf8) [0x7efee48d0b08] error.c:787
/usr/local/lib/libruby.so.3.0(sigsegv+0x55) [0x7efee4a23db5] signal.c:963
/lib/x86_64-linux-gnu/libpthread.so.0(__restore_rt+0x0) [0x7efee4f12140] ../sysdeps/pthread/funlockfile.c:28
/usr/local/lib/libruby.so.3.0(search_nonascii+0x30) [0x7efee4a3ca60] string.c:552
/usr/local/lib/libruby.so.3.0(coderange_scan) string.c:585
/usr/local/lib/libruby.so.3.0(enc_coderange_scan+0x1b) [0x7efee4a3e28a] string.c:709
/usr/local/lib/libruby.so.3.0(rb_enc_str_coderange) string.c:727
/usr/local/lib/ruby/3.0.0/x86_64-linux/objspace.so(is_broken_string+0x8) [0x7efeced9c304] ../../internal/string.h:116
/usr/local/lib/ruby/3.0.0/x86_64-linux/objspace.so(dump_object) objspace_dump.c:388
/usr/local/lib/ruby/3.0.0/x86_64-linux/objspace.so(heap_i+0x39) [0x7efeced9caa9] objspace_dump.c:521
/usr/local/lib/libruby.so.3.0(objspace_each_objects_without_setup+0xaf) [0x7efee48e878f] gc.c:3232
/usr/local/lib/libruby.so.3.0(objspace_each_objects_protected+0x14) [0x7efee48e87c4] gc.c:3242
/usr/local/lib/libruby.so.3.0(rb_ensure+0x12a) [0x7efee48d96aa] eval.c:1162
/usr/local/lib/libruby.so.3.0(objspace_each_objects+0x28) [0x7efee48fb458] gc.c:3310
/usr/local/lib/libruby.so.3.0(rb_objspace_each_objects) gc.c:3298
/usr/local/lib/ruby/3.0.0/x86_64-linux/objspace.so(objspace_dump_all+0x88) [0x7efeced9b068] objspace_dump.c:616
...
```
Unfortunately I couldn't get my hands on that memory region to see which strings are causing this since this doesn't always happen.
I suspect this is also a problem with MRI master since the code looks unchanged from 3.0.4.
--
https://bugs.ruby-lang.org/
______________________________________________
ruby-core mailing list -- ruby-core@ml.ruby-lang.org
To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/
next prev parent reply other threads:[~2022-11-29 14:37 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-11-28 14:30 [ruby-core:111036] [Ruby master Bug#19156] ObjectSpace.dump_all segfault during string inspection mk
2022-11-29 8:30 ` [ruby-core:111048] " byroot (Jean Boussier)
2022-11-29 11:10 ` [ruby-core:111051] " mk
2022-11-29 11:39 ` [ruby-core:111052] " byroot (Jean Boussier)
2022-11-29 13:21 ` [ruby-core:111056] " byroot (Jean Boussier)
2022-11-29 14:00 ` [ruby-core:111057] " mk
2022-11-29 14:02 ` [ruby-core:111058] " mk
2022-11-29 14:09 ` [ruby-core:111059] " byroot (Jean Boussier)
2022-11-29 14:12 ` [ruby-core:111060] " byroot (Jean Boussier)
2022-11-29 14:18 ` [ruby-core:111061] " mk
2022-11-29 14:21 ` [ruby-core:111062] " byroot (Jean Boussier)
2022-11-29 14:37 ` mk [this message]
2022-11-29 15:00 ` [ruby-core:111065] " mk
2022-11-29 15:03 ` [ruby-core:111066] " byroot (Jean Boussier)
2022-11-29 15:06 ` [ruby-core:111067] " byroot (Jean Boussier)
2022-11-29 15:12 ` [ruby-core:111068] " byroot (Jean Boussier)
2022-11-29 15:13 ` [ruby-core:111069] " mk
2022-11-29 15:14 ` [ruby-core:111070] " byroot (Jean Boussier)
2022-11-29 15:29 ` [ruby-core:111071] " mk
2022-12-02 9:54 ` [ruby-core:111148] " mk
2022-12-02 11:13 ` [ruby-core:111149] " byroot (Jean Boussier)
2022-12-02 11:26 ` [ruby-core:111150] " byroot (Jean Boussier)
2022-12-02 11:31 ` [ruby-core:111151] " byroot (Jean Boussier)
2022-12-02 12:32 ` [ruby-core:111154] " mk
2022-12-02 12:56 ` [ruby-core:111156] " mk
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-list from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.ruby-lang.org/en/community/mailing-lists/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=redmine.journal-100323.20221129143743.51782@ruby-lang.org \
--to=ruby-core@ruby-lang.org \
--cc=ruby-core@ml.ruby-lang.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).