From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-4.1 required=3.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id C4E6C1F466 for ; Tue, 7 Jan 2020 01:39:20 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 99FD31208F8; Tue, 7 Jan 2020 10:39:04 +0900 (JST) Received: from o1678948x4.outbound-mail.sendgrid.net (o1678948x4.outbound-mail.sendgrid.net [167.89.48.4]) by neon.ruby-lang.org (Postfix) with ESMTPS id 44B751208D0 for ; Tue, 7 Jan 2020 10:39:02 +0900 (JST) Received: by filterdrecv-p3mdw1-56c97568b5-xjbx9 with SMTP id filterdrecv-p3mdw1-56c97568b5-xjbx9-19-5E13E13B-36 2020-01-07 01:39:07.737256782 +0000 UTC m=+1817760.787158609 Received: from herokuapp.com (unknown [18.209.103.75]) by ismtpd0077p1mdw1.sendgrid.net (SG) with ESMTP id e_UEpa2kQiuHi4CSz-uXnQ for ; Tue, 07 Jan 2020 01:39:07.688 +0000 (UTC) Date: Tue, 07 Jan 2020 01:39:07 +0000 (UTC) From: shugo@ruby-lang.org Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 72359 X-Redmine-Project: ruby-master X-Redmine-Issue-Id: 16461 X-Redmine-Issue-Author: shugo X-Redmine-Sender: shugo X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: =?us-ascii?Q?A4qw=2FE7uJv6zjRy5ZA2RlSNSopH4PRxwlkqj2Qg3aeSyhwVyWyPtmLotDV2Em7?= =?us-ascii?Q?eYOG2uGJQrwdrrZ8LlZP4Y99mQUeGYeUqUddzmh?= =?us-ascii?Q?3dmggusSzAAXWZsH1BfEropvj7DbsBXpVkqznpg?= =?us-ascii?Q?4GwUkvn0YZAZfpCg5=2Ffg+Ydd0cFsN0XfHeUnWYm?= =?us-ascii?Q?mRbI0yVWOe6JImQ31nAw7u0MXKz7CsNddnB5KQ1?= =?us-ascii?Q?oH+HIgf+5z5wTKkME=3D?= To: ruby-core@ruby-lang.org X-ML-Name: ruby-core X-Mail-Count: 96697 Subject: [ruby-core:96697] [Ruby master Feature#16461] Proc#using X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #16461 has been updated by shugo (Shugo Maeda). Eregon (Benoit Daloze) wrote: > shugo (Shugo Maeda) wrote: > > It doesn't mutate the Proc, but the block, and if OtherRefinement is activated before the block invocation, both refinements are activated. > > If the refinements are conflicted, it doesn't work, but I don't think such a situation happens in real world use cases. > > So what happens if I have this: > ```ruby > using Proc::Refinements > > block = Proc.new do > 1 / 2 > 3 + 4 > end > > Thread.new { block.using(DivRefinement) } > Thread.new { block.using(AddRefinement) } > block.call > ``` > > Can I have `/` be the original one (Integer#/) and `+` be the refined one (AddRefinement#+) in the same `block.call`? No, you can't have `/` be the original one once `block.using(DivRefinement)` is called. I don't come up with use cases using conflicting refinements in the same block, so it may be better to prohibit activating new refinements after the first block call. > So `Proc#using` mutates the refinements on that block, globally, for all future invocations of that block. Yes. > I think this will be very hard to explain and document, unfortunately. I admit that the behavior is complex, but DSL users need not understand details. > > Is it hard to invalidate cache only when new refinements are activated by `Proc#using`? > > Basically this slows down call sites inline caches, by having to always check the activated refinements of that block. > Or, we could invalidate all method lookups inside that block whenever `Proc#using` is used with a new refinement, and then we wouldn't need to check it. I meant the latter, but it's not necessary if we prohibit activating new refinements after the first block call. > How would this work with Guilds or MVM? > Would it imply having a copy of the block bytecode per Guild/VM to have inline caches for call sites inside per Guild/VM? > Or `Proc#using` would affect the calls in other Guilds too? That seems unacceptable for the MVM case at least. I have no idea with Guilds, but I agree with you as for MVM. > > `using IntegerDivExt` activates refinements in that entire scope, but I'd like to narrow the scope to blocks for DSLs which refine built-in classes (e.g, https://github.com/amatsuda/activerecord-refinements). > > I see, you want > ```ruby > using Proc::Refinements > > User.where { :age > 3 } > User.where { :name == 'matz' } > ``` > to work with `Symbol#>` and `Symbol#==` refinements? Yes. > Overriding `Symbol#==` in that block does seem pretty dangerous to me. > Any `==` in that block suddenly becomes a hard-to-diagnose bug. > Example: > ```ruby > using Proc::Refinements > > mode = get_current_mode > allowed = User.where { mode == :admin ? :name == 'matz' : :age > 3 } > ``` > > Maybe not being able to have refinements per block encourages people to not refine Integer#/ and Symbol#==, which might be a lot safer anyway. > Refining existing methods feels quite dangerous to me in general. Refining new methods seems fine and safe. I agree with you in general. There is a trade off between power and safety, but I think the current specification is too conservative. > I understand the intent, and not willing to have `using ActiveRecord::DSL` at the top-level as it would then override `Symbol#==` everywhere in that file. Yes. > > Or it may be possible to prohibit adding *new* refinements after the first call of the block. > > I think that could be a good rule, all `Proc#using` must happen `before Proc#call` for a given block. > That would mean the code inside a block always calls the same methods (i.e., the refinements for a given call site are fixed), which seems essential to me for sanity when reading code. OK, I'll try to fix PoC implementation. > It would be nice if we could kind of mark methods taking a *lexical* block (`where {}`, not `where(&b)`) and wanting to apply refinements to them: > ```ruby > class ActiverRecord::Model > def where > yield > end > use_refinements_for_method :where, DSLRefinements > end > ``` > > Which might make it possible to require that `where` only accepts a lexical block. > I'm thinking all `where(&b)` are going to be confusing, because the block body is far from its usage which enables refinements. It's an interesting idea, but I heard a discussion about deprecating yield from CRuby committers before. Does anyone know the status of the discussion? And I want to provide a way to use refinements with instance_eval. > > Allowing `using` in blocks may be used instead, but it's too verbose for such DSLs. > > Isn't `using Proc::Refinements` already too verbose for Rails users? Maybe. I proposed it mainly for JRuby implementation. > It would be useful as a hint we need to be able to invalidate all lookups in blocks affected by `using Proc::Refinements` though. > Otherwise we would need the ability to invalidate every block, which means in TruffleRuby a higher memory footprint of one Assumption for every block. > OTOH, `using Proc::Refinements` would likely apply to far more blocks than needed, if e.g., used at the top-level of the file. > Imagine `using Proc::Refinements` at the top of https://github.com/amatsuda/activerecord-refinements/blob/master/spec/where_spec.rb for example. I understand your concern. Magic comments like `block-level-refinements: true` may be used instead. ---------------------------------------- Feature #16461: Proc#using https://bugs.ruby-lang.org/issues/16461#change-83682 * Author: shugo (Shugo Maeda) * Status: Open * Priority: Normal * Assignee: * Target version: 2.8 ---------------------------------------- ## Overview I propose Proc#using to support block-level refinements. ```ruby module IntegerDivExt refine Integer do def /(other) quo(other) end end end def instance_eval_with_integer_div_ext(obj, &block) block.using(IntegerDivExt) # using IntegerDivExt in the block represented by the Proc object obj.instance_eval(&block) end # necessary where blocks are defined (not where Proc#using is called) using Proc::Refinements p 1 / 2 #=> 0 instance_eval_with_integer_div_ext(1) do p self / 2 #=> (1/2) end p 1 / 2 #=> 0 ``` ## PoC implementation For CRuby: https://github.com/shugo/ruby/pull/2 For JRuby: https://github.com/shugo/jruby/pull/1 ## Background I proposed [Feature #12086: using: option for instance_eval etc.](https://bugs.ruby-lang.org/issues/12086) before, but it has problems: * Thread safety: The same block can be invoked with different refinements in multiple threads, so it's hard to implement method caching. * _exec family support: {instance,class,module}_exec cannot be supported. * Implicit use of refinements: every blocks can be used with refinements, so there was implementation difficulty in JRuby and it has usability issue in headius's opinion. ## Solutions in this proposal ### Thread safety Proc#using affects the block represented by the Proc object, neither the specific Proc object nor the specific block invocation. Method calls in a block are resolved with refinements which are used by Proc#using in the block at the time. Once all possible refinements are used in the block, there is no need to invalidate method cache anymore. See [these tests](https://github.com/shugo/ruby/pull/2/commits/1c922614ad7d1fb43b73e195348c81da7a4546ef) to understand how it works. Which refinements are used is depending on the order of Proc#using invocations until all Proc#using calls are finished, but eventually method calls in a block are resolved with the same refinements. ### * _exec family support [Feature #12086](https://bugs.ruby-lang.org/issues/12086) was an extension of _eval family, so it cannot be used with _exec family, but Proc#using is independent from _eval family, and can be used with _exec family: ```ruby def instance_exec_with_integer_div_ext(obj, *args, &block) block.using(IntegerDivExt) obj.instance_exec(*args, &block) end using Proc::Refinements p 1 / 2 #=> 0 instance_exec_with_integer_div_ext(1, 2) do |other| p self / other #=> (1/2) end p 1 / 2 #=> 0 ``` ### Implicit use of refinements Proc#using can be used only if `using Proc::Refinements` is called in the scope of the block represented by the Proc object. Otherwise, a RuntimeError is raised. There are two reasons: * JRuby creates a special CallSite for refinements at compile-time only when `using` is called at the scope. * When reading programs, it may help understanding behavior. IMHO, it may be unnecessary if libraries which uses Proc#using are well documented. `Proc::Refinements` is a dummy module, and has no actual refinements. -- https://bugs.ruby-lang.org/