From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-3.3 required=3.0 tests=AWL,BAYES_00, MAILING_LIST_MULTI,RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY shortcircuit=no autolearn=no autolearn_force=no version=3.4.2 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 274CC1F5AE for ; Wed, 22 Jul 2020 17:57:49 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 862D9120B13; Thu, 23 Jul 2020 02:57:19 +0900 (JST) Received: from xtrwkhkc.outbound-mail.sendgrid.net (xtrwkhkc.outbound-mail.sendgrid.net [167.89.16.28]) by neon.ruby-lang.org (Postfix) with ESMTPS id 3DC1F120B0F for ; Thu, 23 Jul 2020 02:57:18 +0900 (JST) Received: by filterdrecv-p3las1-7754f7d4cc-h4ckv with SMTP id filterdrecv-p3las1-7754f7d4cc-h4ckv-20-5F187E18-66 2020-07-22 17:57:44.967014076 +0000 UTC m=+2331848.611305501 Received: from herokuapp.com (unknown) by ismtpd0084p1mdw1.sendgrid.net (SG) with ESMTP id jf5ihwODSgOCJgWNfjfAEA for ; Wed, 22 Jul 2020 17:57:44.822 +0000 (UTC) Date: Wed, 22 Jul 2020 17:57:45 +0000 (UTC) From: tenderlove@ruby-lang.org Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 75062 X-Redmine-Project: ruby-master X-Redmine-Issue-Tracker: Bug X-Redmine-Issue-Id: 17023 X-Redmine-Issue-Author: larskanis X-Redmine-Issue-Assignee: tenderlovemaking X-Redmine-Sender: tenderlovemaking X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: =?us-ascii?Q?lWh=2FbSnEjJFRTUHRj45oOQBme8zohdTU9K3TxrDAbWIXLjArzsp=2FN5AaQGGeyO?= =?us-ascii?Q?liGiv+Vzi8=2F+QgPKX85GP1nhBjVkcsh7t54H5pc?= =?us-ascii?Q?VpfsIVOcxTCu1BjUQ8t=2FzlpKf+X=2F1P25fdf=2Flaa?= =?us-ascii?Q?=2FpwrSuoBhJVIPND=2FR2C2r9f1WjbYa7rbam4+0YW?= =?us-ascii?Q?8xHdKMBcVKAIE07zSSf0w192GzqFD6VetPFpIN8?= =?us-ascii?Q?w0wJEo4n9Fkt7iXM4=3D?= To: ruby-core@ruby-lang.org X-ML-Name: ruby-core X-Mail-Count: 99273 Subject: [ruby-core:99273] [Ruby master Bug#17023] How to prevent String memory to be relocated in ruby-ffi X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #17023 has been updated by tenderlovemaking (Aaron Patterson). One idea I had is that we could make `RSTRING_PTR`, `StringValueCStr`, `StringValuePtr`, etc ensure that the character buffer it returns is not embedded in the object. So, if it's an embedded string, move it to `malloc`. We could introduce "unsafe" functions that don't do that, then change MRI internals to use the unsafe versions. This seems like a big change, but I'm struggling to think of something smaller. Another thought is rather than exposing `rb_gc_register_address`, just implement an object that has one reference and calls `rb_gc_mark` on the reference. Again, this would have to be a change for FFI's API, but I think it's worthwhile since anyone could just `const_set(:A, nil)` and the string would get collected and the saved char* pointer would go bad. (Obviously people probably aren't going to do that `const_set` thing, but relying on an indirect reference to keep your object alive is asking for trouble) I'm willing to do the work to introduce "unsafe" versions of the string buffer APIs if we think it would be worthwhile and if we can't find a smaller solution. ---------------------------------------- Bug #17023: How to prevent String memory to be relocated in ruby-ffi https://bugs.ruby-lang.org/issues/17023#change-86661 * Author: larskanis (Lars Kanis) * Status: Closed * Priority: Normal * Assignee: tenderlovemaking (Aaron Patterson) * ruby -v: ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-linux] * Backport: 2.5: DONTNEED, 2.6: DONTNEED, 2.7: DONE ---------------------------------------- [ruby-ffi](https://github.com/ffi/ffi) allows to pass String objects to C by using the `:string` argument type. This way the string memory returned by `RSTRING_PTR` is passed to the C function. The user has to ensure on Ruby level that the string isn't GC'ed - as long as it is used on C level. That's the contract and this worked with all past ruby versions, but ruby-2.7 introduced `GC.compact`, which can relocate strings to another memory location. This example shows the situation and that the string is relocated although it is still referenced in ruby code: ```ruby File.write "string-relocate.c", <<-EOC static char *g_str; void set(char* str) { g_str = str; } char* get() { return g_str; } EOC system "gcc -shared -fPIC string-relocate.c -o string-relocate.so" require 'ffi' class Foo extend FFI::Library ffi_lib File.expand_path('string-relocate.so') attach_function :set, [:string], :void attach_function :get, [], :string def initialize(count) proc {} # necessary to trigger relocation a = "a" * count set(a) GC.verify_compaction_references(toward: :empty, double_heap: true) puts "get(#{count}): #{get} (should be: #{a})" end end Foo.new(23) Foo.new(24) ``` The output looks like so on ruby-2.7.1: ``` get(23): (should be: aaaaaaaaaaaaaaaaaaaaaaa) get(24): aaaaaaaaaaaaaaaaaaaaaaaa (should be: aaaaaaaaaaaaaaaaaaaaaaaa) ``` So using `GC.compact` while a string parameter is in use, both on Ruby and on C level, can cause invalid memory access. How can this prevented? A C extension is expected to use `rb_gc_mark()` in order to pin the VALUE to a memory location. But I couldn't find a way to pin a `VALUE` at the time the argument is passed to the C function, which is the only point in time ruby-ffi has access to it. ---Files-------------------------------- string-relocate.rb (653 Bytes) 0001-Only-marked-objects-should-be-considered-movable.patch (1.23 KB) -- https://bugs.ruby-lang.org/