From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-3.9 required=3.0 tests=AWL,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id D42861F466 for ; Fri, 17 Jan 2020 16:21:08 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id ADDB8120AE1; Sat, 18 Jan 2020 01:20:51 +0900 (JST) Received: from xtrwkhkc.outbound-mail.sendgrid.net (xtrwkhkc.outbound-mail.sendgrid.net [167.89.16.28]) by neon.ruby-lang.org (Postfix) with ESMTPS id 12824120A3C for ; Sat, 18 Jan 2020 01:20:48 +0900 (JST) Received: by filterdrecv-p3las1-5bf99c48d-8v9zg with SMTP id filterdrecv-p3las1-5bf99c48d-8v9zg-18-5E21DEE6-86 2020-01-17 16:20:54.993557692 +0000 UTC m=+1877035.292397727 Received: from herokuapp.com (unknown [3.89.197.5]) by ismtpd0020p1iad2.sendgrid.net (SG) with ESMTP id tOKNFdonQFileex2gMXRLA for ; Fri, 17 Jan 2020 16:20:54.919 +0000 (UTC) Date: Fri, 17 Jan 2020 16:20:55 +0000 (UTC) From: daniel@dan42.com Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 72595 X-Redmine-Project: ruby-master X-Redmine-Issue-Id: 16511 X-Redmine-Issue-Author: Dan0042 X-Redmine-Sender: Dan0042 X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: =?us-ascii?Q?8sy4RigFvRTdBfCVJrT9zb2J88PC92TMQwdNgaWYaq69Tl5vWSkckshbwrhTQv?= =?us-ascii?Q?wj=2FoZEPI5pWRqJd4JfgzET6YttqRfUk1szF+wt7?= =?us-ascii?Q?yy4bFdnJce2KhYT+tZ15Zm6iTzFcIIO1U2c0k7f?= =?us-ascii?Q?JeBkyTQd=2FrxV0aOfJgdSyY+Uqf8TzUqouIG6kZ3?= =?us-ascii?Q?YFD8ceJvDx+BY2of3n9kh9CV9TymjA4T4SyqTtx?= =?us-ascii?Q?C7ZCUARQkpF54c7j0=3D?= To: ruby-core@ruby-lang.org X-ML-Name: ruby-core X-Mail-Count: 96930 Subject: [ruby-core:96930] [Ruby master Feature#16511] Subclass of Hash for keyword arguments X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #16511 has been updated by Dan0042 (Daniel DeLorme). jeremyevans0 (Jeremy Evans) wrote: > In this case the minimum arity is met, but you would still end up with ArgumentError because it would treat the `debug_log(opts)` call as passing keywords instead of a positional hash, and `debug_log` doesn't support a `quux` keyword. But most importantly it's compatible with 2.6. In other words since it breaks in 2.6 this case "doesn't exist" in real production code, and there's no "real" compatibility issue. It's also rational; since opts is a KwHash it conceptually makes sense to interpret it as keyword arguments. Maybe not in your static paradigm, but dynamically yes. So we just keep this backward compatibility with warnings for a while longer until 2.6 (or 2.7?) is EOL. Keyword arguments have worked _mostly_ fine since 2.0, there's no reason to be in such a rush to break things right now now now. If you really want the new "fixed" but backward incompatible behavior, you have to _opt in_ to it with `debug_log({**opts})` or such. Unlike 2.6 this would finally be possible. It's far "better" to opt in to the incompatible behavior for a few cases like this than to force people to opt out of the incompatible behavior for a much larger number of cases. And by "better" here I mean more respectful of people's time and trouble. ---------------------------------------- Feature #16511: Subclass of Hash for keyword arguments https://bugs.ruby-lang.org/issues/16511#change-83957 * Author: Dan0042 (Daniel DeLorme) * Status: Open * Priority: Normal * Assignee: * Target version: ---------------------------------------- As an alternative to #16463 and #16494 I'd like to propose this approach, which I believe allows a **much** more flexible path for migration of keyword arguments. The idea is to have a subclass of Hash (let's name it "KwHash") which provides a clean, object-oriented design with various benefits. I'll try to describe the idea by breaking it down into figurative steps. Imagine starting with ruby 2.6 and then: ### Step 1 When a double-splat or a brace-less hash is used, instead of a Hash it creates a KwHash. ```ruby def foo(x) x end foo(k:1).class #=> KwHash foo(**hash).class #=> KwHash [k:1].last.class #=> KwHash [**hash].last.class #=> KwHash {**hash}.class #=> Hash ``` At this point we haven't introduced any real change. Everything that worked before is still working the same way, with the ONLY exception being code like `kw.class == Hash` which now returns false. But no one actually writes code like that; it's always `kw.is_a?(Hash)`, which still returns true. ### Step 2 When there is ambiguity due to optional vs keyword argument, we rely on the last argument being Hash or KwHash to disambiguate. ```ruby def foo(x=nil, **kw) [x,kw] end foo({k:1}) #=> [{k:1},{}] foo(k:1) #=> [nil,{k:1}] ``` This is the _minimum_ amount of incompatibility required to solve ALL bugs previously reported with keyword arguments. (#8040, #8316, #9898, #10856, #11236, #11967, #12104, #12717, #12821, #13336, #13647, #14130, etc.) ### Step 3 Introduce additional incompatibility to improve clarity of design. Here we deprecate the automatic conversion of Hash to keyword argument; only KwHash is accepted. With a deprecation/warning phase, of course. The "automatic" promotion of a KwHash to a keyword argument follows the same rules as a Hash in 2.6; since the KwHash is conceptually intended to represent keyword arguments, this conversion makes sense in a way that a normal data Hash doesn't. We've taken the "last positional hash" concept and split it into "conceptually a hash" and "conceptually keyword arguments". _Most importantly_, all the changes required to silence these warnings are _compatible with 2.6_. ```ruby def foo(x, **kw); end foo(k:1) # ArgumentError because x not specified foo(1, {k:1}) # ArgumentError because too many arguments; Hash cannot be converted to KwHashs opts = [k:1].first foo(opts) # opts is a KwHash therefore used as keyword argument; ArgumentError because x not specified foo(1, opts) # opts is a KwHash therefore used as keyword argument ``` At this point we have achieved _almost-full_ **dynamic** keyword separation, as opposed to the current _almost-full_ **static** approach. I want to make the point here that, yes, keyword arguments **are** separated, it's just a different paradigm. With static separation, a keyword argument is defined lexically by a double-splat. With dynamic separation, a keyword argument is when the last argument is a KwHash. {{Note: I'm saying "almost-full" because KwHash is not promoted to keywords in `def foo(a,**kw);end;foo(x:1)` and because static keywords are auto-demoted to positional in `def foo(a);end;foo(x:1)`] Any form of delegation works with no change required. This preserves the behavior of 2.6 but only for KwHash objects. This is similar to having 2.7 with `ruby2_keywords` enabled by default. But also different in some ways; most notably it allows the case shown in #16494 to work by default: ```ruby array = [x:1] array.push(x:2) array.map{ |x:| x } #=> [1,2] ``` The current approach does not allow this to work at all. The solution proposed in #16494 has all the same flaws as Hash-based keyword arguments; what happens to `each{ |x=nil,**kw| }` ? The subclass-based solution allows a KwHash to be converted to... keywords. Very unsurprising. Given that ruby is a dynamically-typed language I feel that dynamic typing of keywords if a more natural fit than static typing. But I realize that many disagree with that, which is why we continue to... ### Step 4 Introduce additional incompatibility to reach static/lexical separation of keyword arguments. Here we require that even a KwHash should be passed with a double-splat in order to qualify as a keyword argument. ```ruby def bar(**kw) end def foo(**kw) bar(kw) #=> error; KwHash passed without ** bar(**kw) #=> ok end ``` At this point we've reached the same behavior as 2.7. Delegation needs to be fixed, but as we know the changes required to silence these warnings are **not** compatible with 2.6. So here we introduce a way to _silence **only** these "Step 4" warnings_, for people who need to remain compatible with 2.6. And we keep them as warnings instead of errors until ruby 2.6 is EOL. So instead of having to update a bunch of places with `ruby2_keywords` right now, it's a single flag like `Warning[:ruby3_keywords]`. Once ruby 2.6 is EOL these become controlled by `Warning[:deprecated]` which tells people they **have** to fix their code. Which is just like the eventual deprecation of `ruby2_keywords`, just without the busy work of adding `ruby2_keywords` statements in the first place. The question remains of how to handle #16494 here. Either disallow it entirely, but I think that would be a shame. Or just like #16494 suggests, allow hash unpacking in non-lambda Proc. Except that now it can be a KwHash instead of a Hash, which at least preserves dynamic keyword separation. ## Putting it all together The idea is _not_ to reimplement keyword argument separation; all that is needed is to implement the things above that are not in 2.7: * Create a KwHash object when a double-splat is used. * If a warning is due to a KwHash instead of a Hash, make it a different kind of warning that can be toggled off separately from the Hash warnings (and that will stay as warnings until 2.6 is EOL) I think that's all, really... ### Pros * Cleaner way to solve #16494 * Better compatibility (at least until 2.6 is EOL) * delegation * storing an argument list that ends with a KwHash * destructuring iteration (#16494) * We can avoid the "unfortunate corner case" as described in the [release notes](https://www.ruby-lang.org/en/news/2019/12/12/separation-of-positional-and-keyword-arguments-in-ruby-3-0/) * in 2.7 only do not output "Step 4" warnings, leave delegation like it was * in 2.8 the "Step 3" warnings have been fixed and a Hash will not be converted to keyword arguments * delegation can now safely be fixed to use the `**` syntax * ruby2_keywords is not required, which is desirable because * it's a hidden flag _hack_ * it requires to change the code now, and change it _again_ when ruby2_keywords is deprecated; twice the work; twice the gem upgrades * it was supposed to be used only for people who need to support 2.6 or below, but it's being misunderstood as an acceptable way to fix delegation in general * there's the non-zero risk that ruby2_keywords will never be removed, leaving us with a permanent "hack mode" * dynamic keywords are by far preferable to supporting ruby2_keywords forever * Likely _better performance_, as the KwHash class can be optimized specifically for the characteristics of keyword arguments. * More flexible migration * Allow more time to upgrade the hard stuff in Step 4 * Can reach the _same_ goal as the current static approach * Larger "support zone" https://xkcd.com/2224/ * Instead of wide-ranging incompatibilities all at once, there's the _possibility_ of making it finer-grained and more gradual * rubyists can _choose_ to migrate all at once or in smaller chunks * It hedges the risks by keeping more possibilities open for now. * It allows to cop-out at Step 3 if Step 4 turns out too hard because it breaks too much stuff ### Cons * It allows to cop-out at Step 3 if Step 4 turns out too hard because it breaks too much stuff -- https://bugs.ruby-lang.org/