[ruby-core:92585] [Ruby trunk Feature#15836] [Proposal] Make Module#name and Symbol#to_s return their internal fstrings

[ruby-core:92585] [Ruby trunk Feature#15836] [Proposal] Make Module#name and Symbol#to_s return their internal fstrings

[jean.boussier]

Issue #15836 has been reported by byroot (Jean Boussier).

----------------------------------------
Feature #15836: [Proposal] Make Module#name and Symbol#to_s return their internal fstrings
https://bugs.ruby-lang.org/issues/15836

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
# Why ?

In many codebases, especially Rails apps, these two methods are the source of quite a lot of object allocations.

`Module#name` is often accessed for various introspection features, autoloading etc.

`Symbol#to_s` is access a lot by HashWithIndifferentAccess other various APIs accepting both symbols and strings. 

Returning fstrings for both of these methods could significantly reduce allocations, as well as sligthly reduce retention as it would reduce some duplications.

Also, more and more Ruby APIs are now returning fstrings. `frozen_string_literal`AFAIK should become the default some day, string used as hash keys are now automatically interned as well.

### Backward compatibilty 

Of course this is not fully backward compatible, it's inevitable that some code in the wild is mutating the strings returned by these methods, but I do believe it's a rare occurence, and easy to fix. 

### Implementation

I implemented it here: https://github.com/ruby/ruby/pull/2175



-- 
https://bugs.ruby-lang.org/

[ruby-core:92586] [Ruby trunk Feature#15836] [Proposal] Make Module#name and Symbol#to_s return their internal fstrings

[mame]

Issue #15836 has been updated by mame (Yusuke Endoh).

Status changed from Open to Feedback

Could you show us a benchmark, especially non-micro one?  I believe that it is definitely required to discuss this proposal.

----------------------------------------
Feature #15836: [Proposal] Make Module#name and Symbol#to_s return their internal fstrings
https://bugs.ruby-lang.org/issues/15836#change-77947

* Author: byroot (Jean Boussier)
* Status: Feedback
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
# Why ?

In many codebases, especially Rails apps, these two methods are the source of quite a lot of object allocations.

`Module#name` is often accessed for various introspection features, autoloading etc.

`Symbol#to_s` is access a lot by HashWithIndifferentAccess other various APIs accepting both symbols and strings. 

Returning fstrings for both of these methods could significantly reduce allocations, as well as sligthly reduce retention as it would reduce some duplications.

Also, more and more Ruby APIs are now returning fstrings. `frozen_string_literal`AFAIK should become the default some day, string used as hash keys are now automatically interned as well.

### Backward compatibilty 

Of course this is not fully backward compatible, it's inevitable that some code in the wild is mutating the strings returned by these methods, but I do believe it's a rare occurence, and easy to fix. 

### Implementation

I implemented it here: https://github.com/ruby/ruby/pull/2175



-- 
https://bugs.ruby-lang.org/

[ruby-core:92587] [Ruby trunk Feature#15836] [Proposal] Make Module#name and Symbol#to_s return their internal fstrings

[hanmac]

Issue #15836 has been updated by Hanmac (Hans Mackowiak).


is the String returned by rb_sym2str not always frozen?

how is the GC handling in case the Symbol was a dynamic generated one?
wouldn't that remove the string that got returned from rb_sym2str too?

----------------------------------------
Feature #15836: [Proposal] Make Module#name and Symbol#to_s return their internal fstrings
https://bugs.ruby-lang.org/issues/15836#change-77948

* Author: byroot (Jean Boussier)
* Status: Feedback
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
# Why ?

In many codebases, especially Rails apps, these two methods are the source of quite a lot of object allocations.

`Module#name` is often accessed for various introspection features, autoloading etc.

`Symbol#to_s` is access a lot by HashWithIndifferentAccess other various APIs accepting both symbols and strings. 

Returning fstrings for both of these methods could significantly reduce allocations, as well as sligthly reduce retention as it would reduce some duplications.

Also, more and more Ruby APIs are now returning fstrings. `frozen_string_literal`AFAIK should become the default some day, string used as hash keys are now automatically interned as well.

### Backward compatibilty 

Of course this is not fully backward compatible, it's inevitable that some code in the wild is mutating the strings returned by these methods, but I do believe it's a rare occurence, and easy to fix. 

### Implementation

I implemented it here: https://github.com/ruby/ruby/pull/2175



-- 
https://bugs.ruby-lang.org/

[ruby-core:92588] [Ruby trunk Feature#15836] [Proposal] Make Module#name and Symbol#to_s return their internal fstrings

[chris]

Issue #15836 has been updated by chrisseaton (Chris Seaton).


> how is the GC handling in case the Symbol was a dynamic generated one?

It copies it - strings and symbols aren’t the same thing so it couldn’t jus return the symbol again.

----------------------------------------
Feature #15836: [Proposal] Make Module#name and Symbol#to_s return their internal fstrings
https://bugs.ruby-lang.org/issues/15836#change-77949

* Author: byroot (Jean Boussier)
* Status: Feedback
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
# Why ?

In many codebases, especially Rails apps, these two methods are the source of quite a lot of object allocations.

`Module#name` is often accessed for various introspection features, autoloading etc.

`Symbol#to_s` is access a lot by HashWithIndifferentAccess other various APIs accepting both symbols and strings. 

Returning fstrings for both of these methods could significantly reduce allocations, as well as sligthly reduce retention as it would reduce some duplications.

Also, more and more Ruby APIs are now returning fstrings. `frozen_string_literal`AFAIK should become the default some day, string used as hash keys are now automatically interned as well.

### Backward compatibilty 

Of course this is not fully backward compatible, it's inevitable that some code in the wild is mutating the strings returned by these methods, but I do believe it's a rare occurence, and easy to fix. 

### Implementation

I implemented it here: https://github.com/ruby/ruby/pull/2175



-- 
https://bugs.ruby-lang.org/

[ruby-core:92589] [Ruby trunk Feature#15836] [Proposal] Make Module#name and Symbol#to_s return their internal fstrings

[jean.boussier]

Issue #15836 has been updated by byroot (Jean Boussier).


@mame very good point. I'll try to run our app against that patch tomorrow. 

----------------------------------------
Feature #15836: [Proposal] Make Module#name and Symbol#to_s return their internal fstrings
https://bugs.ruby-lang.org/issues/15836#change-77950

* Author: byroot (Jean Boussier)
* Status: Feedback
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
# Why ?

In many codebases, especially Rails apps, these two methods are the source of quite a lot of object allocations.

`Module#name` is often accessed for various introspection features, autoloading etc.

`Symbol#to_s` is access a lot by HashWithIndifferentAccess other various APIs accepting both symbols and strings. 

Returning fstrings for both of these methods could significantly reduce allocations, as well as sligthly reduce retention as it would reduce some duplications.

Also, more and more Ruby APIs are now returning fstrings. `frozen_string_literal`AFAIK should become the default some day, string used as hash keys are now automatically interned as well.

### Backward compatibilty 

Of course this is not fully backward compatible, it's inevitable that some code in the wild is mutating the strings returned by these methods, but I do believe it's a rare occurence, and easy to fix. 

### Implementation

I implemented it here: https://github.com/ruby/ruby/pull/2175



-- 
https://bugs.ruby-lang.org/

[ruby-core:92590] [Ruby trunk Feature#15836] [Proposal] Make Module#name and Symbol#to_s return their internal fstrings

[shevegen]

Issue #15836 has been updated by shevegen (Robert A. Heiler).


> frozen_string_literalAFAIK should become the default some day

Matz said that but it will not be for ruby 3.0 at the least.

In my own code bases I am using frozen strings a lot, through the shebang; either
"# frozen_string_literal: false" (initially) but these days more and more
"# frozen_string_literal: true". I think other ruby users may be able to transition
into frozen strings if enough time is given AND recommendations are given from the
ruby core team in due time whenever there are (future) changes.

> string used as hash keys are now automatically interned as well.

Although the use case for HashWithIndifferentAccess (I hate how long that name
is ...) is probably not completely void, with strings being frozen it appears to
me as if one use case (the speed factor) is nullified. There may be still other
use cases probably, such as API design e. g. when people have to make a decision
between "do I have to use a String or a Symbol here". Personally I like both
strings and symbols, though; I think jeremy evans once gave a good explanation
or wrote documentation to emphasis the distinction in the official doc (but I
may misremember).

----------------------------------------
Feature #15836: [Proposal] Make Module#name and Symbol#to_s return their internal fstrings
https://bugs.ruby-lang.org/issues/15836#change-77951

* Author: byroot (Jean Boussier)
* Status: Feedback
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
# Why ?

In many codebases, especially Rails apps, these two methods are the source of quite a lot of object allocations.

`Module#name` is often accessed for various introspection features, autoloading etc.

`Symbol#to_s` is access a lot by HashWithIndifferentAccess other various APIs accepting both symbols and strings. 

Returning fstrings for both of these methods could significantly reduce allocations, as well as sligthly reduce retention as it would reduce some duplications.

Also, more and more Ruby APIs are now returning fstrings. `frozen_string_literal`AFAIK should become the default some day, string used as hash keys are now automatically interned as well.

### Backward compatibilty 

Of course this is not fully backward compatible, it's inevitable that some code in the wild is mutating the strings returned by these methods, but I do believe it's a rare occurence, and easy to fix. 

### Implementation

I implemented it here: https://github.com/ruby/ruby/pull/2175



-- 
https://bugs.ruby-lang.org/

[ruby-core:92592] [Ruby trunk Feature#15836] [Proposal] Make Module#name and Symbol#to_s return their internal fstrings

[jean.boussier]

Issue #15836 has been updated by byroot (Jean Boussier).


> Matz said that but it will not be for ruby 3.0 at the least.

I assumed it was due for 3.0, but good to know it isn't.

> I think other ruby users may be able to transition into frozen strings if enough time is given 

I contribute to many gems, and from what I can see `# frozen_string_literal: true` is extremely common. A good part is likely due to rubocop enforcing it, another is likely due to various article about how freezing strings made many codebases faster (sometimes oversold but that's another topic).

> with strings being frozen it appears to me as if one use case (the speed factor) is nullified

I'm not 100% sure I understood your point correctly. What I meant by hash keys being frozen is:

```
hash = {}
string = :foo.to_s # One string allocated here
hash[string] = true # A second string is allocated here because Hash apply: `-string.dup`
```

If `Symbol#to_s` was to return it's internal fstring, the above snippet would save 2 string allocations.

----------------------------------------
Feature #15836: [Proposal] Make Module#name and Symbol#to_s return their internal fstrings
https://bugs.ruby-lang.org/issues/15836#change-77953

* Author: byroot (Jean Boussier)
* Status: Feedback
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
# Why ?

In many codebases, especially Rails apps, these two methods are the source of quite a lot of object allocations.

`Module#name` is often accessed for various introspection features, autoloading etc.

`Symbol#to_s` is access a lot by HashWithIndifferentAccess other various APIs accepting both symbols and strings. 

Returning fstrings for both of these methods could significantly reduce allocations, as well as sligthly reduce retention as it would reduce some duplications.

Also, more and more Ruby APIs are now returning fstrings. `frozen_string_literal`AFAIK should become the default some day, string used as hash keys are now automatically interned as well.

### Backward compatibilty 

Of course this is not fully backward compatible, it's inevitable that some code in the wild is mutating the strings returned by these methods, but I do believe it's a rare occurence, and easy to fix. 

### Implementation

I implemented it here: https://github.com/ruby/ruby/pull/2175



-- 
https://bugs.ruby-lang.org/

[ruby-core:92593] [Ruby trunk Feature#15836] [Proposal] Make Module#name and Symbol#to_s return their internal fstrings

[ruby-core]

Issue #15836 has been updated by marcandre (Marc-Andre Lafortune).


Is `HashWithIndifferentAccess` the main rationale behind this request?

I have doubts about the usefulness of `HashWithIndifferentAccess` today, now that Rails has protected parameters.

Moreover, now that symbols are garbage collected, shouldn't its implementation use symbols for keys instead of strings?

----------------------------------------
Feature #15836: [Proposal] Make Module#name and Symbol#to_s return their internal fstrings
https://bugs.ruby-lang.org/issues/15836#change-77954

* Author: byroot (Jean Boussier)
* Status: Feedback
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
# Why ?

In many codebases, especially Rails apps, these two methods are the source of quite a lot of object allocations.

`Module#name` is often accessed for various introspection features, autoloading etc.

`Symbol#to_s` is access a lot by HashWithIndifferentAccess other various APIs accepting both symbols and strings. 

Returning fstrings for both of these methods could significantly reduce allocations, as well as sligthly reduce retention as it would reduce some duplications.

Also, more and more Ruby APIs are now returning fstrings. `frozen_string_literal`AFAIK should become the default some day, string used as hash keys are now automatically interned as well.

### Backward compatibilty 

Of course this is not fully backward compatible, it's inevitable that some code in the wild is mutating the strings returned by these methods, but I do believe it's a rare occurence, and easy to fix. 

### Implementation

I implemented it here: https://github.com/ruby/ruby/pull/2175



-- 
https://bugs.ruby-lang.org/

[ruby-core:92595] [Ruby trunk Feature#15836] [Proposal] Make Module#name and Symbol#to_s return their internal fstrings

[jean.boussier]

Issue #15836 has been updated by byroot (Jean Boussier).


> Is HashWithIndifferentAccess the main rationale behind this request?

No. It's simply the poster child of how common `Symbol#to_s` is in code bases.

I shouldn't have mentioned `HashWithIndifferentAccess` because clearly lots of people have a feud with it, and now it's totally shifting the conversation.

### What the proposal is actually about

The question here, is wether `Module#name` and `Symbol#to_s` should return a new string on every call.

My own understanding of why it's like this is because historically all strings were mutable, so the way to prevent them to be mutated was to duplicate them.

But now that frozen strings are very common in code bases, and that `fstring` are a thing, IMHO getting a new string on every call is what is surprising.




----------------------------------------
Feature #15836: [Proposal] Make Module#name and Symbol#to_s return their internal fstrings
https://bugs.ruby-lang.org/issues/15836#change-77955

* Author: byroot (Jean Boussier)
* Status: Feedback
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
# Why ?

In many codebases, especially Rails apps, these two methods are the source of quite a lot of object allocations.

`Module#name` is often accessed for various introspection features, autoloading etc.

`Symbol#to_s` is access a lot by HashWithIndifferentAccess other various APIs accepting both symbols and strings. 

Returning fstrings for both of these methods could significantly reduce allocations, as well as sligthly reduce retention as it would reduce some duplications.

Also, more and more Ruby APIs are now returning fstrings. `frozen_string_literal`AFAIK should become the default some day, string used as hash keys are now automatically interned as well.

### Backward compatibilty 

Of course this is not fully backward compatible, it's inevitable that some code in the wild is mutating the strings returned by these methods, but I do believe it's a rare occurence, and easy to fix. 

### Implementation

I implemented it here: https://github.com/ruby/ruby/pull/2175



-- 
https://bugs.ruby-lang.org/

[ruby-core:92597] [Ruby trunk Feature#15836] [Proposal] Make Module#name and Symbol#to_s return their internal fstrings

[jean.boussier]

Issue #15836 has been updated by byroot (Jean Boussier).


@mame re benchmark

So I decided to run this against redmine boot, using this branch: https://github.com/redmine/redmine/compare/master...byroot:boot-benchmark

Eager loading is enabled so that the entire codebase is loaded, and it uses https://github.com/SamSaffron/memory_profiler to measure allocations and retentions.

Full benchmark output: https://gist.github.com/byroot/845a5877c1cde91c50b43be446dfb20f

Baseline (official 2.6.3):

```
Total allocated: 121.11 MB (1234362 objects)
Total retained:  24.86 MB (200539 objects)

allocated memory by class
-----------------------------------
  63.36 MB  String

allocated objects by class
-----------------------------------
    980623  String
```

With the patch (official 2.6.3 + this patch):

```
Total allocated: 120.01 MB (1206699 objects)
Total retained:  24.82 MB (199397 objects)

allocated memory by class
-----------------------------------
  62.25 MB  String


allocated objects by class
-----------------------------------
    952953  String
```

Diff:

```
-27 663 allocations (-2.24%)
-1.10MB allocations (-0.9%)
-1 142 retentions
-0.4MB retentions  
```

IMHO that is significant, especially for a small sized application like Redmine. However I can't say wether it outweigh the backward compatibility concern or not. 

### Backward compatibility

One thing to note is that I had to patch https://github.com/rails/rails/blob/28aca474d48b6acdbe8c7861d9347e27c65fafd9/activesupport/lib/active_support/ordered_options.rb#L43 because it was mutating the result of `Symbol#to_s`. Also running the Redmine test suite shows a couple breakage in the i18n gem.

IMHO these are fairly simple to fix, but I would totally understand if that was considered as a no-go.

### Typical code benefiting from this change

  - Rails autoloader and Zeitwerk would both benefit from the `Module#name` change as they both keep references to class names as hash keys
  - Various parts of Rails would benefit as well since they use the class names extensively to derive other class names, as well as symbols `belongs_to :post`.
  - `def method_missing` very often call `name.to_s` to match the method name, hence would benefit from the `Symbol#to_s` as well.
  - Serialization of symbols into various formats, e.g. `{foo: 42}.to_json`. That pattern is fairly common IMO.

----------------------------------------
Feature #15836: [Proposal] Make Module#name and Symbol#to_s return their internal fstrings
https://bugs.ruby-lang.org/issues/15836#change-77957

* Author: byroot (Jean Boussier)
* Status: Feedback
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
# Why ?

In many codebases, especially Rails apps, these two methods are the source of quite a lot of object allocations.

`Module#name` is often accessed for various introspection features, autoloading etc.

`Symbol#to_s` is access a lot by HashWithIndifferentAccess other various APIs accepting both symbols and strings. 

Returning fstrings for both of these methods could significantly reduce allocations, as well as sligthly reduce retention as it would reduce some duplications.

Also, more and more Ruby APIs are now returning fstrings. `frozen_string_literal`AFAIK should become the default some day, string used as hash keys are now automatically interned as well.

### Backward compatibilty 

Of course this is not fully backward compatible, it's inevitable that some code in the wild is mutating the strings returned by these methods, but I do believe it's a rare occurence, and easy to fix. 

### Implementation

I implemented it here: https://github.com/ruby/ruby/pull/2175



-- 
https://bugs.ruby-lang.org/

[ruby-core:92598] [Ruby trunk Feature#15836] [Proposal] Make Module#name and Symbol#to_s return their internal fstrings

[pdahorek]

Issue #15836 has been updated by ahorek (Pavel Rosický).


@byroot thanks for sharing the benchmark!

IMO module names should be frozen

even if it's easy to fix, this change could definitely break existing apps that depend on it, see
https://github.com/jruby/jruby/issues/5229

----------------------------------------
Feature #15836: [Proposal] Make Module#name and Symbol#to_s return their internal fstrings
https://bugs.ruby-lang.org/issues/15836#change-77958

* Author: byroot (Jean Boussier)
* Status: Feedback
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
# Why ?

In many codebases, especially Rails apps, these two methods are the source of quite a lot of object allocations.

`Module#name` is often accessed for various introspection features, autoloading etc.

`Symbol#to_s` is access a lot by HashWithIndifferentAccess other various APIs accepting both symbols and strings. 

Returning fstrings for both of these methods could significantly reduce allocations, as well as sligthly reduce retention as it would reduce some duplications.

Also, more and more Ruby APIs are now returning fstrings. `frozen_string_literal`AFAIK should become the default some day, string used as hash keys are now automatically interned as well.

### Backward compatibilty 

Of course this is not fully backward compatible, it's inevitable that some code in the wild is mutating the strings returned by these methods, but I do believe it's a rare occurence, and easy to fix. 

### Implementation

I implemented it here: https://github.com/ruby/ruby/pull/2175



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>

[ruby-core:93090] [Ruby trunk Feature#15836] [Proposal] Make Module#name and Symbol#to_s return their internal fstrings

[duerst]

Issue #15836 has been updated by duerst (Martin Dürst).


Isn't one main purpose of converting a Symbol to a String that you want to change the symbol string? This proposal would make that use case more tedious.

----------------------------------------
Feature #15836: [Proposal] Make Module#name and Symbol#to_s return their internal fstrings
https://bugs.ruby-lang.org/issues/15836#change-78512

* Author: byroot (Jean Boussier)
* Status: Feedback
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
# Why ?

In many codebases, especially Rails apps, these two methods are the source of quite a lot of object allocations.

`Module#name` is often accessed for various introspection features, autoloading etc.

`Symbol#to_s` is access a lot by HashWithIndifferentAccess other various APIs accepting both symbols and strings. 

Returning fstrings for both of these methods could significantly reduce allocations, as well as sligthly reduce retention as it would reduce some duplications.

Also, more and more Ruby APIs are now returning fstrings. `frozen_string_literal`AFAIK should become the default some day, string used as hash keys are now automatically interned as well.

### Backward compatibilty 

Of course this is not fully backward compatible, it's inevitable that some code in the wild is mutating the strings returned by these methods, but I do believe it's a rare occurence, and easy to fix. 

### Implementation

I implemented it here: https://github.com/ruby/ruby/pull/2175



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>

[ruby-core:93091] [Ruby trunk Feature#15836] [Proposal] Make Module#name and Symbol#to_s return their internal fstrings

[matz]

Issue #15836 has been updated by matz (Yukihiro Matsumoto).

Status changed from Feedback to Rejected

The compatibility breakage from changing those methods (especially `Symbol#to_s`) is too big. Sorry.
Maybe we should work on `HashWithIndifferentAccess` to improve (memory) performance.

Matz.

----------------------------------------
Feature #15836: [Proposal] Make Module#name and Symbol#to_s return their internal fstrings
https://bugs.ruby-lang.org/issues/15836#change-78513

* Author: byroot (Jean Boussier)
* Status: Rejected
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
# Why ?

In many codebases, especially Rails apps, these two methods are the source of quite a lot of object allocations.

`Module#name` is often accessed for various introspection features, autoloading etc.

`Symbol#to_s` is access a lot by HashWithIndifferentAccess other various APIs accepting both symbols and strings. 

Returning fstrings for both of these methods could significantly reduce allocations, as well as sligthly reduce retention as it would reduce some duplications.

Also, more and more Ruby APIs are now returning fstrings. `frozen_string_literal`AFAIK should become the default some day, string used as hash keys are now automatically interned as well.

### Backward compatibilty 

Of course this is not fully backward compatible, it's inevitable that some code in the wild is mutating the strings returned by these methods, but I do believe it's a rare occurence, and easy to fix. 

### Implementation

I implemented it here: https://github.com/ruby/ruby/pull/2175



-- 
https://bugs.ruby-lang.org/