From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-2.8 required=3.0 tests=AWL,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS shortcircuit=no autolearn=no autolearn_force=no version=3.4.2 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 412D51F463 for ; Thu, 2 Jan 2020 12:44:46 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 5A66B120A00; Thu, 2 Jan 2020 21:44:29 +0900 (JST) Received: from o1678948x4.outbound-mail.sendgrid.net (o1678948x4.outbound-mail.sendgrid.net [167.89.48.4]) by neon.ruby-lang.org (Postfix) with ESMTPS id 31BF61209E5 for ; Thu, 2 Jan 2020 21:44:26 +0900 (JST) Received: by filterdrecv-p3mdw1-56c97568b5-x76nl with SMTP id filterdrecv-p3mdw1-56c97568b5-x76nl-20-5E0DE5AF-25 2020-01-02 12:44:31.433516702 +0000 UTC m=+1425684.503407586 Received: from herokuapp.com (unknown [3.89.196.85]) by ismtpd0052p1mdw1.sendgrid.net (SG) with ESMTP id cXILpovOR12ho5NDeRuZWw for ; Thu, 02 Jan 2020 12:44:31.337 +0000 (UTC) Date: Thu, 02 Jan 2020 12:44:31 +0000 (UTC) From: eregontp@gmail.com Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 72286 X-Redmine-Project: ruby-master X-Redmine-Issue-Id: 16461 X-Redmine-Issue-Author: shugo X-Redmine-Sender: Eregon X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: =?us-ascii?Q?KippOI8ZHtTweq7XfQzW93937kJ4QNWwSBuHnaMEcr3z4e8Eam+vV4179VDevd?= =?us-ascii?Q?WA7gndRnAoB4f6fZmQO36rN1pMaaHGRVzXuEGEf?= =?us-ascii?Q?c925n12HRPNVWp582=2F6Xq9SLIuovYq2Z4viQ1HT?= =?us-ascii?Q?1vEpqSxeWZwLXzogj9zIYYovvIQeR9n+GIRpvPL?= =?us-ascii?Q?1UXuRljZ1+5xFiLYjNaeQ6PqlSVfmFWSts0pxSc?= =?us-ascii?Q?MuvfGASIvh+EagZr4=3D?= To: ruby-core@ruby-lang.org X-ML-Name: ruby-core X-Mail-Count: 96624 Subject: [ruby-core:96624] [Ruby master Feature#16461] Proc#using X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #16461 has been updated by Eregon (Benoit Daloze). shugo (Shugo Maeda) wrote: > It doesn't mutate the Proc, but the block, and if OtherRefinement is activated before the block invocation, both refinements are activated. > If the refinements are conflicted, it doesn't work, but I don't think such a situation happens in real world use cases. So what happens if I have this: ```ruby block = Proc.new do 1 / 2 3 + 4 end Thread.new { block.using(DivRefinement) } Thread.new { block.using(AddRefinement) } block.call ``` Can I have `/` be the original one (Integer#/) and `+` be the refined one (AddRefinement#+) in the same `block.call`? > In my proposal refinements are activated per block (not per Proc), so the refinements activated in a given call site are eventually same. I start to understand what you mean by that, `block.using(IntegerDivExt)` means every Proc instance of that block will use those refinements, so: ```ruby using Proc::Refinements def my_proc Proc.new { 1 / 2 } end proc = my_proc proc.using(IntegerDivExt) proc.call # Uses IntegerDivExt my_proc.call # Uses IntegerDivExt too even though it's not the same Proc object ``` So `Proc#using` mutates the refinements on that block, globally, for all future invocations of that block. I think this will be very hard to explain and document, unfortunately. > Is it hard to invalidate cache only when new refinements are activated by `Proc#using`? Basically this slows down call sites inline caches, by having to always check the activated refinements of that block. Or, we could invalidate all method lookups inside that block whenever `Proc#using` is used with a new refinement, and then we wouldn't need to check it. How would this work with Guilds or MVM? Would it imply having a copy of the block bytecode per Guild/VM to have inline caches for call sites inside per Guild/VM? Or `Proc#using` would affect the calls in other Guilds too? That seems unacceptable for the MVM case at least. > `using IntegerDivExt` activates refinements in that entire scope, but I'd like to narrow the scope to blocks for DSLs which refine built-in classes (e.g, https://github.com/amatsuda/activerecord-refinements). I see, you want ```ruby using Proc::Refinements User.where { :age > 3 } User.where { :name == 'matz' } ``` to work with `Symbol#>` and `Symbol#==` refinements? Overriding `Symbol#==` in that block does seem pretty dangerous to me. Any `==` in that block suddenly becomes a hard-to-diagnose bug. Example: ```ruby mode = get_current_mode allowed = User.where { mode == :admin ? :name == 'matz' : :age > 3 } ``` Maybe not being able to have refinements per block encourages people to not refine Integer#/ and Symbol#==, which might be a lot safer anyway. Refining existing methods feels quite dangerous to me in general. Refining new methods seems fine and safe. I understand the intent, and not willing to have `using ActiveRecord::DSL` at the top-level as it would then override `Symbol#==` everywhere in that file. > Or it may be possible to prohibit adding *new* refinements after the first call of the block. I think that could be a good rule, all `Proc#using` must happen `before Proc#call` for a given block. That would mean the code inside a block always calls the same methods (i.e., the refinements for a given call site are fixed), which seems essential to me for sanity when reading code. I think something like this should be prevented if feasible: ```ruby b = Proc.new { :age > 3 } b.call # no refinements User.where(&b) # using refinements b.call # using refinements because Proc#using is global state! ``` It would be nice if we could kind of mark methods taking a *lexical* block (`where {}`, not `where(&b)`) and wanting to apply refinements to them: ```ruby class ActiverRecord::Model def where yield end use_refinements_for_method :where, DSLRefinements end ``` Which might make it possible to require that `where` only accepts a lexical block. I'm thinking all `where(&b)` are going to be confusing, because the block body is far from its usage which enables refinements. > Allowing `using` in blocks may be used instead, but it's too verbose for such DSLs. Isn't `using Proc::Refinements` already too verbose for Rails users? It would be useful as a hint we need to be able to invalidate all lookups in blocks affected by `using Proc::Refinements` though. Otherwise we would need the ability to invalidate every block, which means in TruffleRuby a higher memory footprint of one Assumption for every block. OTOH, `using Proc::Refinements` would likely apply to far more blocks than needed, if e.g., used at the top-level of the file. Imagine `using Proc::Refinements` at the top of https://github.com/amatsuda/activerecord-refinements/blob/master/spec/where_spec.rb for example. ---------------------------------------- Feature #16461: Proc#using https://bugs.ruby-lang.org/issues/16461#change-83595 * Author: shugo (Shugo Maeda) * Status: Open * Priority: Normal * Assignee: * Target version: 2.8 ---------------------------------------- ## Overview I propose Proc#using to support block-level refinements. ```ruby module IntegerDivExt refine Integer do def /(other) quo(other) end end end def instance_eval_with_integer_div_ext(obj, &block) block.using(IntegerDivExt) # using IntegerDivExt in the block represented by the Proc object obj.instance_eval(&block) end # necessary where blocks are defined (not where Proc#using is called) using Proc::Refinements p 1 / 2 #=> 0 instance_eval_with_integer_div_ext(1) do p self / 2 #=> (1/2) end p 1 / 2 #=> 0 ``` ## PoC implementation For CRuby: https://github.com/shugo/ruby/pull/2 For JRuby: https://github.com/shugo/jruby/pull/1 ## Background I proposed [Feature #12086: using: option for instance_eval etc.](https://bugs.ruby-lang.org/issues/12086) before, but it has problems: * Thread safety: The same block can be invoked with different refinements in multiple threads, so it's hard to implement method caching. * _exec family support: {instance,class,module}_exec cannot be supported. * Implicit use of refinements: every blocks can be used with refinements, so there was implementation difficulty in JRuby and it has usability issue in headius's opinion. ## Solutions in this proposal ### Thread safety Proc#using affects the block represented by the Proc object, neither the specific Proc object nor the specific block invocation. Method calls in a block are resolved with refinements which are used by Proc#using in the block at the time. Once all possible refinements are used in the block, there is no need to invalidate method cache anymore. See [these tests](https://github.com/shugo/ruby/pull/2/commits/1c922614ad7d1fb43b73e195348c81da7a4546ef) to understand how it works. Which refinements are used is depending on the order of Proc#using invocations until all Proc#using calls are finished, but eventually method calls in a block are resolved with the same refinements. ### * _exec family support [Feature #12086](https://bugs.ruby-lang.org/issues/12086) was an extension of _eval family, so it cannot be used with _exec family, but Proc#using is independent from _eval family, and can be used with _exec family: ```ruby def instance_exec_with_integer_div_ext(obj, *args, &block) block.using(IntegerDivExt) obj.instance_exec(*args, &block) end using Proc::Refinements p 1 / 2 #=> 0 instance_exec_with_integer_div_ext(1, 2) do |other| p self / other #=> (1/2) end p 1 / 2 #=> 0 ``` ### Implicit use of refinements Proc#using can be used only if `using Proc::Refinements` is called in the scope of the block represented by the Proc object. Otherwise, a RuntimeError is raised. There are two reasons: * JRuby creates a special CallSite for refinements at compile-time only when `using` is called at the scope. * When reading programs, it may help understanding behavior. IMHO, it may be unnecessary if libraries which uses Proc#using are well documented. `Proc::Refinements` is a dummy module, and has no actual refinements. -- https://bugs.ruby-lang.org/