From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-4.1 required=3.0 tests=AWL,BAYES_00, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 077F31F8C8 for ; Fri, 17 Sep 2021 14:19:32 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 09D63120BE1; Fri, 17 Sep 2021 23:18:04 +0900 (JST) Received: from xtrwkhkc.outbound-mail.sendgrid.net (xtrwkhkc.outbound-mail.sendgrid.net [167.89.16.28]) by neon.ruby-lang.org (Postfix) with ESMTPS id E1990120BF7 for ; Fri, 17 Sep 2021 23:18:01 +0900 (JST) Received: by filterdrecv-55446c4d49-xldjr with SMTP id filterdrecv-55446c4d49-xldjr-1-6144A3E5-80 2021-09-17 14:19:17.705690334 +0000 UTC m=+1353561.184881653 Received: from herokuapp.com (unknown) by geopod-ismtpd-2-1 (SG) with ESMTP id 8VDNfHVURHucow4AwHOw0A for ; Fri, 17 Sep 2021 14:19:17.678 +0000 (UTC) Date: Fri, 17 Sep 2021 14:19:17 +0000 (UTC) From: "byroot (Jean Boussier)" Message-ID: References: Mime-Version: 1.0 X-Redmine-Project: ruby-master X-Redmine-Issue-Tracker: Bug X-Redmine-Issue-Id: 18141 X-Redmine-Issue-Author: byroot X-Redmine-Sender: byroot X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-Redmine-MailingListIntegration-Message-Ids: 81503 X-SG-EID: =?us-ascii?Q?Dq8GNIcVqP8cs5uR+EIAabfg3LE9XdC6dZ5KYNrMpf6VzBWG6HqArcu5raQjXL?= =?us-ascii?Q?IGTFdxKHUt7++67+KD54pNYXheirMcsUIf3sdgF?= =?us-ascii?Q?Riq06j0txXRI2f1Ps5+LgGDKByIFLtc1jvSFZoH?= =?us-ascii?Q?QbMQMxiCmr+wvP8Fwliw0M5WCnS+pGm+3SRDqGC?= =?us-ascii?Q?eqE9rmMmz97GwUNAVOggqLEFKI9u0f4D=2Fke+avS?= =?us-ascii?Q?0qp1w9boa29Ck8cWchKH4V2FscJzKrczqwroN4J?= =?us-ascii?Q?QSgoKTdQB0EDtQMqZR6JQ=3D=3D?= To: ruby-core@ruby-lang.org X-Entity-ID: b/2+PoftWZ6GuOu3b0IycA== X-ML-Name: ruby-core X-Mail-Count: 105327 Subject: [ruby-core:105327] [Ruby master Bug#18141] Marshal load with proc yield strings before they are fully initialized X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #18141 has been updated by byroot (Jean Boussier). So while working on https://bugs.ruby-lang.org/issues/18148, I discovered that many other types of objects are impacted. Just a few examples: ```ruby def round_trip(obj, proc = ->(o) { o.freeze }) Marshal.load(Marshal.dump(obj), proc) end h = {} h.instance_variable_set(:@foo, 42) # round_trip(h) rescue p $! a = [] a.instance_variable_set(:@foo, 42) # round_trip(a) rescue p $! ``` Also, probably by design, but since you can replace the oject by what the proc returns: ```ruby a = {} a.instance_variable_set(:@foo, 42) round_trip(a, proc { 24 }) rescue p $! # ``` I fixed most cases in https://github.com/ruby/ruby/pull/4859, which is my current attempt at implementing https://bugs.ruby-lang.org/issues/18148, but since I just noticed this was marked for backport, I might need to split the bug fix from the new feature. No? ---------------------------------------- Bug #18141: Marshal load with proc yield strings before they are fully initialized https://bugs.ruby-lang.org/issues/18141#change-93742 * Author: byroot (Jean Boussier) * Status: Closed * Priority: Normal * Backport: 2.6: REQUIRED, 2.7: REQUIRED, 3.0: REQUIRED ---------------------------------------- I assume this is a bug because I can't find any spec or test for this behaviour: Consider the following script: ```ruby payload = Marshal.dump("foo") Marshal.load(payload, -> (obj) { if obj.is_a?(String) p [obj, obj.encoding] end obj }) p [:final, string, string.encoding] ``` outputs: ```ruby ["foo", #] [:final, "foo", #] ``` So `Marshal` call the proc before the string get its encoding assigned, this is because the encoding is stored alongside as a `TYPE_IVAR`. I think in such cases `Marshal` should delay calling the proc until the object is fully restored. A corollary to this behaviour is that the following code: ```ruby Marshal.load(payload, :freeze.to_proc) ``` raises with `can't modify frozen String: "foo" (FrozenError)`. -- https://bugs.ruby-lang.org/