[ruby-core:92192] [Ruby trunk Feature#14183] "Real" keyword argument

[merch-redmine@jeremyevans.net]

Issue #14183 has been updated by jeremyevans0 (Jeremy Evans).


I have updated my GitHub branch (https://github.com/jeremyevans/ruby/commits/keyword-argument-separation) to fix the issues in mame's branch that I identified in my previous comment.

Now, my branch keeps compatibility with Ruby 2.6 in regards to treating a hash argument as keywords, issuing warnings as expected for all three cases where behavior will change in Ruby 3:

```ruby
def foo(a, *b, **c)
  [a, b, c]
end

def bar(a, b=1, **c)
  [a, b, c]
end

def baz(a, **c)
  [a, c]
end

foo(1, {a: 1})
# warning: The last argument for `foo' (defined at (irb):1) is used as the keyword parameter
# => [1, [], {:a=>1}]

bar(1, {a: 1})
# warning: The last argument for `bar' (defined at (irb):5) is used as the keyword parameter
# => [1, 1, {:a=>1}]

baz(1, {a: 1})
# warning: The last argument for `baz' (defined at (irb):9) is used as the keyword parameter
# => [1, {:a=>1}]
```

I have also found another behavior change with mame's branch that I think is undesirable, and that is how positional hash arguments with non-Symbol keys are converted to keywords.  That breaks backwards compatibility with Ruby 2.6, and there is no reason to change behavior and emit a warning in Ruby 2.7 when the Ruby 3 behavior will be same as Ruby 2.6 in this case.

Ruby 2.6 and my branch:

```ruby
def a(x=1, **h)
  [x, h]
end

a({:a=>1})
# => [{}, {:a=>1}]

a({"a"=>1})
# [{"a"=>1}, {}]
```

There is still backwards compatibility breakage in cases where the last positional hash has both Symbol keys and non-Symbol keys.  In Ruby 2.6, those hashes would be split, but I'm not sure if we want to keep backwards compatibility and warn about that case in Ruby 2.7. Doing so would probably increase complexity significantly. My branch changes the behavior so that the positional hash is always treated as a positional argument if it contains a non-Symbol key:

Ruby 2.6:

```ruby
a({"a"=>1, :a=>1})
# => [{"a"=>1}, {:a=>1}]
```

My branch:

```ruby
a({"a"=>1, :a=>1})
# => [{"a"=>1, :a=>1}, {}]
```

I have fixed all issues in `lib` that caused warnings, and no changes are required in `ext`.  Most changes were in the csv library, with minor changes in net, rdoc, rubygems, tempfile, and tmpdir.  Here is a stat for issues in `lib`:

```
 lib/csv.rb                             | 36 ++++++++++++++++++------------------
 lib/csv/core_ext/array.rb              |  2 +-
 lib/csv/core_ext/string.rb             |  2 +-
 lib/csv/row.rb                         |  2 +-
 lib/csv/table.rb                       |  4 ++--
 lib/net/ftp.rb                         |  2 +-
 lib/net/protocol.rb                    |  2 +-
 lib/rdoc/generator/darkfish.rb         | 12 ++++++------
 lib/rubygems.rb                        |  2 +-
 lib/rubygems/commands/setup_command.rb |  2 +-
 lib/rubygems/package.rb                |  2 +-
 lib/tempfile.rb                        |  4 ++--
 lib/tmpdir.rb                          |  4 ++--
```

Here's a comparison with the changes mame's branch requires in `lib` and `ext`:

```
 ext/etc/extconf.rb                                     |   2 +-
 ext/json/lib/json/common.rb                            |  28 +++++++++++++++-------------
 ext/json/lib/json/generic_object.rb                    |   4 ++--
 ext/openssl/lib/openssl/ssl.rb                         |   4 ++--
 ext/psych/lib/psych.rb                                 |   2 +-
 ext/psych/lib/psych/core_ext.rb                        |   4 ++--
 lib/bundler/dsl.rb                                     |   6 +++---
 lib/bundler/runtime.rb                                 |   2 +-
 lib/cgi/core.rb                                        |   7 ++++---
 lib/cgi/html.rb                                        |  11 ++++++-----
 lib/csv.rb                                             |  24 ++++++++++++------------
 lib/csv/core_ext/array.rb                              |   2 +-
 lib/csv/core_ext/string.rb                             |   2 +-
 lib/csv/row.rb                                         |   2 +-
 lib/csv/table.rb                                       |   4 ++--
 lib/erb.rb                                             |   3 ++-
 lib/mkmf.rb                                            |   4 ++--
 lib/net/ftp.rb                                         |  10 +++++-----
 lib/net/http.rb                                        |   5 ++---
 lib/net/http/generic_request.rb                        |   2 +-
 lib/net/imap.rb                                        |   2 +-
 lib/net/protocol.rb                                    |   2 +-
 lib/open-uri.rb                                        |  23 ++++++++++++-----------
 lib/open3.rb                                           |  96 +++++++++++++++++++++++++-----------------------------------------------------------------------
 lib/rdoc/generator/darkfish.rb                         |  18 +++++++++---------
 lib/resolv.rb                                          |  18 +++++++-----------
 lib/rexml/document.rb                                  |   6 ++----
 lib/rexml/xpath.rb                                     |   2 +-
 lib/rss/parser.rb                                      |   7 +------
 lib/rss/rss.rb                                         |   9 ++-------
 lib/rubygems.rb                                        |   6 +++---
 lib/rubygems/command.rb                                |   4 ++--
 lib/rubygems/commands/install_command.rb               |   4 ++--
 lib/rubygems/commands/pristine_command.rb              |   4 ++--
 lib/rubygems/commands/setup_command.rb                 |   2 +-
 lib/rubygems/dependency_installer.rb                   |   4 ++--
 lib/rubygems/installer.rb                              |   8 ++++----
 lib/rubygems/package.rb                                |   2 +-
 lib/rubygems/request_set.rb                            |  10 +++++-----
 lib/rubygems/request_set/gem_dependency_api.rb         |  11 +++++------
 lib/rubygems/resolver/git_specification.rb             |   4 ++--
 lib/rubygems/resolver/lock_specification.rb            |   2 +-
 lib/rubygems/resolver/specification.rb                 |   8 ++++----
 lib/rubygems/test_case.rb                              |  18 +++++++++---------
 lib/rubygems/test_utilities.rb                         |  14 +++++++-------
 lib/tempfile.rb                                        |   8 ++++----
 lib/uri/file.rb                                        |   4 ++--
 lib/uri/ftp.rb                                         |   2 +-
 lib/uri/generic.rb                                     |  22 +++++++++++-----------
 lib/uri/http.rb                                        |   6 +++---
 lib/uri/mailto.rb                                      |   2 +-
 lib/yaml/store.rb                                      |   4 ++--
```

I have fixed all issues in `test` that caused warnings.  These were also much less extensive than in mame's branch.

I have fixed a few broken tests that expected a specific format for warning messages for invalid keywords, as well as a test that assumed TypeError for a non-Symbol keyword (an ArgumentError is now used).

----------------------------------------
Feature #14183: "Real" keyword argument
https://bugs.ruby-lang.org/issues/14183#change-77517

* Author: mame (Yusuke Endoh)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: Next Major
----------------------------------------
In RubyWorld Conference 2017 and RubyConf 2017, Matz officially said that Ruby 3.0 will have "real" keyword arguments.  AFAIK there is no ticket about it, so I'm creating this (based on my understanding).

In Ruby 2, the keyword argument is a normal argument that is a Hash object (whose keys are all symbols) and is passed as the last argument.  This design is chosen because of compatibility, but it is fairly complex, and has been a source of many corner cases where the behavior is not intuitive.  (Some related tickets: #8040, #8316, #9898, #10856, #11236, #11967, #12104, #12717, #12821, #13336, #13647, #14130)

In Ruby 3, a keyword argument will be completely separated from normal arguments.  (Like a block parameter that is also completely separated from normal arguments.)
This change will break compatibility; if you want to pass or accept keyword argument, you always need to use bare `sym: val` or double-splat `**` syntax:

```
# The following calls pass keyword arguments
foo(..., key: val)
foo(..., **hsh)
foo(..., key: val, **hsh)

# The following calls pass **normal** arguments
foo(..., {key: val})
foo(..., hsh)
foo(..., {key: val, **hsh})

# The following method definitions accept keyword argument
def foo(..., key: val)
end
def foo(..., **hsh)
end

# The following method definitions accept **normal** argument
def foo(..., hsh)
end
```

In other words, the following programs WILL NOT work:

```
# This will cause an ArgumentError because the method foo does not accept keyword argument
def foo(a, b, c, hsh)
  p hsh[:key]
end
foo(1, 2, 3, key: 42)

# The following will work; you need to use keyword rest operator explicitly
def foo(a, b, c, **hsh)
  p hsh[:key]
end
foo(1, 2, 3, key: 42)

# This will cause an ArgumentError because the method call does not pass keyword argument
def foo(a, b, c, key: 1)
end
h = {key: 42}
foo(1, 2, 3, h)

# The following will work; you need to use keyword rest operator explicitly
def foo(a, b, c, key: 1)
end
h = {key: 42}
foo(1, 2, 3, **h)
```

I think here is a transition path:

* Ruby 2.6 (or 2.7?) will output a warning when a normal argument is interpreted as keyword argument, or vice versa.
* Ruby 3.0 will use the new semantics.

---Files--------------------------------
vm_args.diff (4.19 KB)
vm_args_v2.diff (4.18 KB)


-- 
https://bugs.ruby-lang.org/