[ruby-core:39986] problems with Refinements

[ruby-core:39986] problems with Refinements

[Shugo Maeda]

Hi,

2011/10/3 Yukihiro Matsumoto <matz@ruby-lang.org>:
> And if-possible list is:
>
>  * refinement
>  * renaming methods and constant lookup resolution for #include
>
> The current reference implementation slows down method calls to 5-15%
> so that unless we could find how to implement it faster before the
> deadline (Fall 2012 maybe), I think we should give it up for 2.0.

I guess the reason of the performance decrease is that I have added
klass, which corresponds to last_class in Ruby 1.8, to
rb_control_frame_t.  I had to do it to implement super correctly.

Even without Refinements, Ruby 1.9 has some issues such as Bug #2402.
ko1 commented on Bug #3136.

  http://redmine.ruby-lang.org/issues/3136#note-7

If he implemented the idea, we might be able to improve the
performance of Refinements.
I'm not sure, and would like to ask ko1's take.

BTW, is performance the only problem, Matz?
Don't we need local rebinding?

-- 
Shugo Maeda

[ruby-core:39988] Re: problems with Refinements

[Steve Klabnik]

There are also the group of people that think refinements are just a
plain bad idea. See, for example,
http://brightredglow.com/2011/10/3/nikita-the-ruby-secret-agent-slides

Actually, I'm not sure if it's in the slides exactly, but Brian Ford's
talk at RubyConf made a pretty strong case against refinements as a
'feature.'

[ruby-core:39990] Re: problems with Refinements

[Yukihiro Matsumoto]

Hi,

In message "Re: [ruby-core:39988] Re: problems with Refinements"
    on Thu, 6 Oct 2011 23:48:14 +0900, Steve Klabnik <steve@steveklabnik.com> writes:

|There are also the group of people that think refinements are just a
|plain bad idea. See, for example,
|http://brightredglow.com/2011/10/3/nikita-the-ruby-secret-agent-slides
|
|Actually, I'm not sure if it's in the slides exactly, but Brian Ford's
|talk at RubyConf made a pretty strong case against refinements as a
|'feature.'

Unfortunately, I missed Brian's talk, so we have to wait until the
video to check what he said.  But when he claimed against refinements,
there must be reasons behind.  I am open to hear.

							matz.

[ruby-core:39992] Re: problems with Refinements

[Yukihiro Matsumoto]

Hi,
n
In message "Re: [ruby-core:39986] problems with Refinements"
    on Thu, 6 Oct 2011 23:03:17 +0900, Shugo Maeda <shugo@ruby-lang.org> writes:

|BTW, is performance the only problem, Matz?
|Don't we need local rebinding?

I don't think so.  I admit there's need for local rebinding for some
cases, but we can always use monkey patching for that purpose.

							matz.

[ruby-core:39996] Re: problems with Refinements

[Steve Klabnik]

> Unfortunately, I missed Brian's talk, so we have to wait until the
> video to check what he said.  But when he claimed against refinements,
> there must be reasons behind.  I am open to hear.
>
>                                                        matz.

I don't claim to speak for Brian, and I've sent him a link to this
thread, so that he can make his own statements about it, but my
understanding of it is this: refinements provide very little benefit
at the cost of increased complexity for both Ruby programmers and Ruby
language developers.

Refinements for Rubyists:
  Pros:
    - protect a library from interference from other library's monkeypatches
  Cons:
  - End up making libraries harder to understand, since the details of
MyLibrary's String are opaque from outside MyLibrary
  - Are a pretty complex feature to use.
  - This complexity makes code with refinements hard to reason about.

Refinements for Ruby Implementors:
  Pros:
    - Giving users a feature they want
  Cons:
    - Complex features are complex to implement
    - Refinements in particular place a large burden on send (I
think), which is what makes the performance penalty so large.
    - This feature affects Ruby at a very, very deep level, and
therefore deserves significant consideration

Finally,

> "If I had asked people what they wanted, they would have said faster horses." - Henry Ford

Many people _do_ want refinements, but that doesn't mean that it's
good for Ruby. Just like 'optional typing,' when people ask for a
language feature like this, what they actually want is different from
what they're asking for. For example, people want optional typing
because they've heard it will 'make Ruby faster,' not because they
want to use optional typing to write better Ruby code. It's the same
thing with Refinements. People are actually saying, "I don't have a
way to know if my library will conflict with another, and what to do
about it." This is a _tooling_ problem, not a language feature
problem. I also personally feel that the problem is overblown. I've
been doing Ruby for a few years now (though not as long as many on
this list. :) ), and I don't think I've ever been particularly bitten
by this issue. Maybe once or twice, and it wasn't too bad to resolve.
In my mind, paying a 10%-15% performance penalty for a once-a-year bug
is a bad tradeoff. I'm willing to acknowledge that it might be more of
a problem with others, however, I've never actually met someone for
whom that's the case.

[ruby-core:39997] Re: problems with Refinements

[Bill Kelly]

Steve Klabnik wrote:
> 
> Refinements for Rubyists:
>   Pros:
>     - protect a library from interference from other library's monkeypatches
>   Cons:
>   - End up making libraries harder to understand, since the details of
>     MyLibrary's String are opaque from outside MyLibrary
>   - Are a pretty complex feature to use.
>   - This complexity makes code with refinements hard to reason about.
> 
> Refinements for Ruby Implementors:
>   Pros:
>     - Giving users a feature they want
>   Cons:
>     - Complex features are complex to implement
>     - Refinements in particular place a large burden on send (I
>       think), which is what makes the performance penalty so large.
>     - This feature affects Ruby at a very, very deep level, and
>       therefore deserves significant consideration

While I don't disagree with the "Pros" you've listed, I've
tended to view the benefits from a slightly different
perspective:

Refinements for Rubyists:
  Pros:
    - Protect *my* code from from another library's monkeypatches
      (given that some libraries use monkeypatching to facilitate
      their own internal implementations)

Refinements for Ruby Implementors:
  Pros:
    - Allow me to use monkeypatching to add features that
      facilitate the internal implementation of my library
      without polluting the global method space.


However, on the issue of the performance impact: There are
some classes of applications I develop where Ruby's slow
execution speed is already a daily source of concern.  I
would not like to see Ruby become significantly slower.


Regards,

Bill

[ruby-core:39999] Re: problems with Refinements

[Yukihiro Matsumoto]

Hi,

In message "Re: [ruby-core:39996] Re: problems with Refinements"
    on Fri, 7 Oct 2011 04:16:33 +0900, Steve Klabnik <steve@steveklabnik.com> writes:

|I don't claim to speak for Brian, and I've sent him a link to this
|thread, so that he can make his own statements about it, but my
|understanding of it is this: refinements provide very little benefit
|at the cost of increased complexity for both Ruby programmers and Ruby
|language developers.

As Bill Kelly stated in [ruby-core:39997], we consider refinements as
protection of existing code from monkey patching.  I am not sure why
you concluded refinements make libraries hard to understand.  From my
point of view, it is much safer and provide more intuitive behavior
than monkey patching.

For performance concern, I would totally agree with most of you.

							matz.

[ruby-core:40015] Re: problems with Refinements

[Shugo Maeda]

Hi,

2011/10/7 Steve Klabnik <steve@steveklabnik.com>:
>  Cons:
>  - End up making libraries harder to understand, since the details of
> MyLibrary's String are opaque from outside MyLibrary

At this point, I think refinements are better than monkey patching
because affection of refinements are limited lexically.

>  - Are a pretty complex feature to use.
>  - This complexity makes code with refinements hard to reason about.

I admit that refinements are complex, but refinements are a feature
for advanced library/framework developers.
Most application programmers do not need to take care about the
complexity of refinements.

> Refinements for Ruby Implementors:
>  Pros:
>    - Giving users a feature they want
>  Cons:
>    - Complex features are complex to implement
(snip)
>    - This feature affects Ruby at a very, very deep level, and
> therefore deserves significant consideration

Agreed.

I guess implementations other than CRuby have last_class in stack
frames, so for their implementors it may be easier to implement
refinements than CRuby:)

>    - Refinements in particular place a large burden on send (I
> think), which is what makes the performance penalty so large.

I guess the performance decrease came from the change of stack frames,
so implementations other than CRuby may not be affected.

> Many people _do_ want refinements, but that doesn't mean that it's
> good for Ruby.

Agreed.  I think Matz should finally decide.

-- 
Shugo Maeda

[ruby-core:40016] Re: problems with Refinements

[Steve Klabnik]

> I am not sure why
> you concluded refinements make libraries hard to understand.

Sorry, this was terrible wording on my part. What I was trying to say
was the converse; I don't think they make it any _easier_. Which is
almost a different statement. I don't feel that refinements are an
actual win for clarity in any way.


> At this point, I think refinements are better than monkey patching
> because affection of refinements are limited lexically.

Right. I'm concerned that by making them more limited, it'll encourage
the proliferation of doing these kinds of custom extensions. Since
they're 'safer,' they'll be used more often.

Is patching core classes something we want to actively encourage? At
least with monkeypatching being a bit 'dangerous,' people think twice
before they do it. If we make it easier, and people have their own
custom strings everywhere, the extra overhead there will get out of
hand. I want Strings to be Strings 99% of the time, and special
strings only sometimes.

> I admit that refinements are complex, but refinements are a feature
> for advanced library/framework developers.
> Most application programmers do not need to take care about the
> complexity of refinements.

Sure. And I'm not opposed to adding something strictly due to
complexity; some would call blocks complex, for example, but they're
my favorite feature of Ruby. However, what I see in refinements is
adding extra complexity and a drop in performance for very, very
little gain.

> Agreed.  I think Matz should finally decide.

Absolutely, 100%. Of course, it's Matz' language, but that doesn't
mean others can't make a case to persuade him one way or the other. :)
But what he says goes.

[ruby-core:40023] Re: problems with Refinements

[Adam Prescott]

[-- Attachment #1: Type: text/plain, Size: 628 bytes --]

On Fri, Oct 7, 2011 at 6:02 AM, Steve Klabnik <steve@steveklabnik.com>wrote:

> Sure. And I'm not opposed to adding something strictly due to
> complexity; some would call blocks complex, for example, but they're
> my favorite feature of Ruby. However, what I see in refinements is
> adding extra complexity and a drop in performance for very, very
> little gain.
>

If performance weren't affected at all, what would you think about
refinements? Would the concern about proliferation of core class
modifications be enough to be against the change? (I'm asking because I'm
curious to know, I'm not trying to make a point here!)

[-- Attachment #2: Type: text/html, Size: 952 bytes --]

[ruby-core:40033] Re: problems with Refinements

[Steve Klabnik]

> If performance weren't affected at all, what would you think about
> refinements? Would the concern about proliferation of core class
> modifications be enough to be against the change? (I'm asking because I'm
> curious to know, I'm not trying to make a point here!)

Yep. I think it's a bad feature in general, the performance thing is
just an additional great reason not to put it in the language.

Brian said he's writing up something more substantial on this too. And
it's not as if we're the only two people who feel this way, and there
are also people that feel strongly refinements should be in.
Ultimately, what matz says goes.

[ruby-core:40034] Re: problems with Refinements

[Kurt Stephens]

On 10/7/11 2:24 PM, Steve Klabnik wrote:
>> If performance weren't affected at all, what would you think about
>> refinements? Would the concern about proliferation of core class
>> modifications be enough to be against the change? (I'm asking because I'm
>> curious to know, I'm not trying to make a point here!)
>
> Yep. I think it's a bad feature in general, the performance thing is
> just an additional great reason not to put it in the language.
>

I'm hoping that Ruby will actually become smaller language with simpler 
and more consistent semantics, while keeping an eye on making a language 
extension a 1st-class feature.

Simplify core semantics and create a smaller language to create new, 
more powerful languages out of smaller, simpler building blocks. 
Perhaps this is what MRuby is/can be?

For example: small, simple language features, like AnonSymbols/AnonMethods:

  * are easy to understand.
  * are easy to use or *not* use.
  * do not change established core semantics/syntax, very much.
  * are generalizations of current and established semantics/syntax.
  * can be used to create more complex behaviors.
  * don't have a performance cost when *not* used.
  * don't have a major performance cost *when* used.
  * are easy to implement efficiently (in both interpreters and compilers).

Can refinements be written in pure Ruby?

-- KAS

[ruby-core:40042] Re: problems with Refinements

[Yukihiro Matsumoto]

Hi,

In message "Re: [ruby-core:40034] Re: problems with Refinements"
    on Sat, 8 Oct 2011 04:56:03 +0900, Kurt Stephens <ks@kurtstephens.com> writes:

|Can refinements be written in pure Ruby?

I don't think we can.  That's because we are going to add.

If you really like a language that is formed from a few simple core
concepts, we already have the language named Lisp from 50 years ago.

							matz.