From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-3.9 required=3.0 tests=AWL,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 99B511FF9C for ; Tue, 27 Oct 2020 02:50:13 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id CAE09120AB5; Tue, 27 Oct 2020 11:49:28 +0900 (JST) Received: from xtrwkhkc.outbound-mail.sendgrid.net (xtrwkhkc.outbound-mail.sendgrid.net [167.89.16.28]) by neon.ruby-lang.org (Postfix) with ESMTPS id 1BC241209D6 for ; Tue, 27 Oct 2020 11:49:25 +0900 (JST) Received: by filterdrecv-p3iad2-64988c98cc-gzbc7 with SMTP id filterdrecv-p3iad2-64988c98cc-gzbc7-19-5F978AD6-4 2020-10-27 02:49:58.088556553 +0000 UTC m=+280506.160946269 Received: from herokuapp.com (unknown) by ismtpd0027p1iad2.sendgrid.net (SG) with ESMTP id s9hUN7S2TOmGDeGHCryP0Q for ; Tue, 27 Oct 2020 02:49:58.054 +0000 (UTC) Date: Tue, 27 Oct 2020 02:49:58 +0000 (UTC) From: marcandre-ruby-core@marc-andre.ca Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 76466 X-Redmine-Project: ruby-master X-Redmine-Issue-Tracker: Feature X-Redmine-Issue-Id: 17284 X-Redmine-Issue-Author: ko1 X-Redmine-Sender: marcandre X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: =?us-ascii?Q?6=2FIMxCQLDposcQf5wmbDAtfaKduBAO0bKyhL3BGZtMQ5q7K2TvpbN6A7JIyt9E?= =?us-ascii?Q?aOEzTvOF87JQwUkYk3qfov=2F28Dc0sE51Uns3LMH?= =?us-ascii?Q?4ST0D1OQrHMeT77KeSbEnpDETib9eiqLrroW6y1?= =?us-ascii?Q?GGgO1EuZ9rVf0WUqlrWZa26PS5X3RgNdKxcG1d0?= =?us-ascii?Q?uG0MkTm0GDiGOkP86Af6weNkvyo+Ofs7qevGG0a?= =?us-ascii?Q?68eSvfI9mlj7PuV2RWNh8XVA7GGa9fBg0EFKUq?= To: ruby-core@ruby-lang.org X-Entity-ID: b/2+PoftWZ6GuOu3b0IycA== X-ML-Name: ruby-core X-Mail-Count: 100594 Subject: [ruby-core:100594] [Ruby master Feature#17284] Shareable Proc X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #17284 has been updated by marcandre (Marc-Andre Lafortune). ko1 (Koichi Sasada) wrote in #note-12: > My understanding: > > `Proc#detach` do > > * 1. allocate a snapshot area > * 2. copy object (which can be referred from Proc's variables) references to snapshot > > `Proc#call` do > > * 1. allocate outer-lvars area for outer variables > * 2. copy snapshot refs to outer-lvars area > > `Proc#freeze` in this ticket is similar to `Proc#detach`, but does not do special at `Proc#call`. Indeed. But `freeze` must do special check for reassignments. What I dislike about `Proc#freeze` is that it does not make intuitive sense to me. A `Proc` is not mutable per say. Calling `Proc.new { ... }` does not change the Proc. Also, if after `Proc#freeze` you can reassign `a` outside the block and has no effect inside the block, then they are different local variables. It is not intuitive for me to disallow reassigning one and not the other. I will agree that in general, these variables will not be reassigned anyways so it won't matter much. I just think it is easier to understand if you are allowed to reassigning it. Do you think there would be a noticeable difference in performance either way? ---------------------------------------- Feature #17284: Shareable Proc https://bugs.ruby-lang.org/issues/17284#change-88225 * Author: ko1 (Koichi Sasada) * Status: Open * Priority: Normal ---------------------------------------- For some reasons, we need to provide a way to make sharable Proc between ractors. * (1) A block for the `Ractor.new`. * (2) Send a proc between ractors. * (3) A block for global callback methods: `define_method` ([Bug #17159]), `TracePoint`, ... For (1), we use `Proc#isolate` (`isolate` is temporary name here) which prohibit to access outer variables. ```ruby a = 1 Proc.new{ p a }.isolate # => can not isolate a Proc because it accesses outer variables (a). # error on `isolate` method call ``` There are no states to share, so it is okay. For (2), `Proc#isolate` is one option because we can send parameters with an argument `call`. But it should be a bit long. ```ruby i, j, k = nil pr = Proc.new do |i, j, k| p i, j, k end.isolate r = Ractor.new do |task, param| task.call(*param) end r.send([pr, [i, j, k]]) ``` For (3), maybe we need to make more flexible Proc which can *read* outer block parameter on that snapshot (discussed in #17159). Now, I named it with `freeze`, because it seems frozen Proc. ```ruby a = 1 # try to read, and returns old value (snapshot at `freeze`) pr = Proc.new{ p a #=> 1 } pr = pr.freeze pr.call a = 2 pr.call #=> 1 # try to write, and it is not allowed pr2 = Proc.new{ a = 1 } pr2 = pr.freeze #=> can not freeze a Proc because it accesses outer variables (a). (ArgumentError) ``` To share the "frozen" Proc between ractors, outer values should be (deep) frozen. It means readable values (in above case, `a`) should be shareable. Now we named it `Proc#shareable!` ```ruby a = [1, [2, 3]] pr = Proc.new{ p a.frozen? #=> true }.shareable! a[0] = 0 #=> frozen error ``` This ticket has three different variant of mutability and shareability for Proc. | | outer lvar | shareable | freeze/making shareable other objects |---------------|---------------|------------|------------------------------------------ |a. isolate | N/A | Yes | No |b. freeze | allow to read | No | No |c. shareable! | allow to read | Yes | Yes I want to introduce functionality of `shareable!`, but not sure the Ruby-level API. I think (b) `freeze` for this semantics is good name because it only allows to read-only local variables. However, it is not enough to make a sharable Proc because read objects from the Proc should be also sharable. Making `freeze` with (c) `shareable!` functionality is one idea, but I think `freeze` should not deep-freezing because it is very surprising that read objects become the sharable (== frozen) for usual Ruby users. Maybe `Ractor.make_sharable(pr)` makes `pr` sharable is no surprise because it is good declaration the `pr` should be shareable, even if the read objects from `pr` become shareable (== frozen). Removing (a) `isolate` and using (c) `shareable!` at `Ractor.new(&b)` is one idea, but I think it is surprising that they can access outer local variables, but the they can not access newly assigned variables as usual blocks. ``` a = 1 Ractor.new do p a # only 1 end a = 2 ``` (a) `isolate` does not have such issue because all outer lvars accesses are not allowed == easy to understand, easy to debug. In practice, accessing outer variables with multi-ractor program is very useful because we need to declare same local variables if we want to access them from different ractors. The following example is from [Feature #17261]: ```ruby tv1 = Thread::TVar.new(0) tv2 = Thread::TVar.new(0) r1 = Ractor.new tv1, tv2 do |tv1, tv2| # <-- here loop do Thread.atomically do v1, v2 = tv1.value, tv2.value raise if v1 != v2 end end end ``` With (c) `shareable!` semantics, it is easier to write: ```ruby tv1 = Thread::TVar.new(0) tv2 = Thread::TVar.new(0) r1 = Ractor.new do loop do Thread.atomically do v1, v2 = tv1.value, tv2.value raise if v1 != v2 end end end ``` Above example is also enable to make more simple: ```ruby i, j, k = nil pr = Proc.new do p i, j, k end r = Ractor.new do |task| task.call end r.send(pr) ``` However, using this semantics (`shareable!`) can freeze extra-variables in accidents: ```ruby a = [1, 2, 3] Ractor.new do do_something if a.length > 0 end a << 4 # raises FrozenError ``` It is clear that there is a syntax or method to apply `shareable!` functionality. ```ruby a = [1, 2, 3] Ractor.new &(Ractor.make_shareable(Proc.new{ a.length ... }) ``` It can be used with `define_method` which can invoke from ractors: ```ruby define_method(name, Ractor.make_shareable(Proc.new{ ... }))` ``` But it is too long. There are implementations for (a), (b) and (c), but the API is not fixed, so there is no PR now. I'm thinking to introduce (c)'s feature in `Ractor.make_sharaeble(pr)`. To use with `define_method`, maybe it should be more friendly. Ideally, new syntax is great. There is no conclusion, and your comments are welcome. Thanks, Koichi -- https://bugs.ruby-lang.org/