[ruby-core:95928] [Ruby master Misc#16188] What are the performance implications of the new keyword arguments in 2.7 and 3.0?

[merch-redmine@jeremyevans.net]

Issue #16188 has been updated by jeremyevans0 (Jeremy Evans).


Eregon (Benoit Daloze) wrote:
> jeremyevans0 (Jeremy Evans) wrote:
> > On CRuby master branch, in the worst possible case I could design, the difference was about 1%.
> 
> Why is `def a(x) end; a(*arr)` the worst case? Would it not be more expensive to call a method accepting keyword arguments, since then further checks might be needed?

`def a(x) end; a(*arr) end` is the worst case performance decrease, assuming that what we are measuring is the effect of supporting `ruby2_keywords` when not actually using the feature.  If the final hash does not have the keyword flag, no changes are made. Methods accepting keyword arguments have more overhead than methods not accepting keyword arguments in CRuby, so there will be a larger percentage difference when calling methods that do not accept keyword arguments.

> I benchmarked MRI, comparing MRI 2.6.5 with MRI 2.7.0preview3, and I see overheads far above 1%, more in the 10%.

Comparing 2.6 to 2.7 is irrelevant in regards to the discussion of the effect of `ruby2_keywords`.  There are many other changes between 2.6 and 2.7 that have a much larger effect than `ruby2_keywords`.  If you want to measure the effect of `ruby2_keywords`, you need to benchmark the master branch against the master branch with the removal of `ruby2_keywords`, as I did in an earlier comment, and as your TruffleRuby benchmark did.

> However, all 3 `foo(*args)` seem to show a general 8-10% slowdown in Ruby 2.7, which I'd guess is due to `ruby2_keywords`.

I think your guess is wrong.  If you want to test the effect of `ruby2_keywords`, you have to benchmark with that change in isolation, not in combination with all of the other changes between 2.6 and 2.7.

Eregon (Benoit Daloze) wrote:
> I also measured on TruffleRuby, and there the diff is minimal, just adding the `ruby2_keywords` check on *splat call sites:
> https://github.com/oracle/truffleruby/commit/d143af3626aae009e2414bfe61833565fe3a0476
> 
> The results are similar (details on https://gist.github.com/eregon/15ebe02ff8f42c0ab964e1066a783f9d ):
> * req: 4.6% slower
> * kw: 10.4% slower
> * kwrest: 10.9% slower
> 
> In summary, I see about 10% slowdown on this micro benchmark, representative of `foo(*args)` calls, just by the extra `ruby2_keywords` check.

Thanks for working on a microbenchmark for TruffleRuby.  This shows about a maximum of 11% slowdown for calls using splats without keyword splats.  Considering the percentage of calls using splats without keyword splats, compared to all other calls, it seems unlikely this change will have a significant effect in a real world benchmark on TruffleRuby.

----------------------------------------
Misc #16188: What are the performance implications of the new keyword arguments in 2.7 and 3.0?
https://bugs.ruby-lang.org/issues/16188#change-82768

* Author: Eregon (Benoit Daloze)
* Status: Open
* Priority: Normal
* Assignee: jeremyevans0 (Jeremy Evans)
----------------------------------------
In #14183, keyword arguments became further separated from positional arguments.

Contrary to the original design though, keyword and positional arguments are not fully separated for methods not accepting keyword arguments.
Example: `foo(key: :value)` will `def foo(hash)` will pass a positional argument.
This is of course better for compatibility, but I wonder what are the performance implications.

The block argument is completely separate in all versions, so no need to concern ourselves about that.

In Ruby <= 2.6:
* The caller never needs to know about the callee's arguments, it can just take all arguments and pass them as an array.
  The last argument might be used to extract keyword, but this is all done at the callee side.
* Splitting kwargs composed of Symbol and non-Symbol keys can be fairly expensive, but it is a rare occurrence.
  If inlining the callee and kwargs are all passed as a literal Hash at the call site, there shouldn't be any overhead compared to positional arguments once JIT'ed.

In Ruby 2.7:
* The caller needs to pass positional and keyword arguments separately, at least when calling a method accepting kwargs.
  But, if it calls a methods not accepting kwargs, then the "kwargs" (e.g. `foo(key: :value)`) should be treated just like a final Hash positional argument.
* (If we had complete separation, then we could always pass positional and keyword arguments separately, so the caller could once again ignore the callee)

How is the logic implemented in MRI for 2.7?

Specializing the caller for a given callee is a well-known technique.
However, it becomes more difficult if different methods are called from the same callsite (polymorphic call), especially if one accepts kwargs and another does not.
In that case, I think we will see a performance cost to this approach, by having to pass arguments differently based on the method to be called.

What about delegation using `ruby2_keywords`?
Which checks does that add (compared to 2.6) in the merged approach with the Hash flag?



-- 
https://bugs.ruby-lang.org/