* [ruby-core:94349] [Ruby master Feature#16103] Make the dot-colon method reference frozen
[not found] <redmine.issue-16103.20190814122309@ruby-lang.org>
@ 2019-08-14 12:23 ` maciej
2019-08-29 7:37 ` [ruby-core:94658] " ko1
` (2 subsequent siblings)
3 siblings, 0 replies; 4+ messages in thread
From: maciej @ 2019-08-14 12:23 UTC (permalink / raw)
To: ruby-core
Issue #16103 has been reported by maciej.mensfeld (Maciej Mensfeld).
----------------------------------------
Feature #16103: Make the dot-colon method reference frozen
https://bugs.ruby-lang.org/issues/16103
* Author: maciej.mensfeld (Maciej Mensfeld)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
----------------------------------------
I made a PR to freeze the dot-colon method reference result object (https://github.com/ruby/ruby/pull/2267). Nobu asked to make an issue out of that. I initially discussed that with Matz and Ko1 during the hack challenge in Bristol.
Here are some of the reasons why I think it should be done before releasing this feature:
- It's worth keeping the bound methods frozen as one should not modify them in general
- Freezing keeps a window of opportunity for introducing method reference caching in case it would be needed because the method object is immutable.
- I want to work on the "last method" reference cache for that feature when we see more use-cases after 2.7 is released and without having it frozen, the cost and complexity are probably higher than the outcome.
Example of the misuse that makes GC life much harder (note, that freezing does not fix that, but allows further work related to this issue):
```ruby
# frozen_string_literal: true
GC.disable
class Parse
def self.call(string)
string.to_f
end
end
class Normalize
def self.call(number)
number.round
end
end
class Transform
def self.call(int)
int * 2
end
end
# Simulate a long running data producing source with batch results
stream = Array.new(10_000) { Array.new(100) { '100.2' } }
before = GC.stat[:total_allocated_objects]
# This will provide batches to be processed, let's assume an endless data-source
stream.each do |batch|
batch
.map(&Parse.:call)
.map(&Normalize.:call)
.map(&Transform.:call)
end
after = GC.stat[:total_allocated_objects]
p a = after - before # 150001
before = GC.stat[:total_allocated_objects]
# "cache"
parse = Parse.:call
normalize = Normalize.:call
transform = Transform.:call
stream.each do |batch|
batch
.map(&parse)
.map(&normalize)
.map(&transform)
end
after = GC.stat[:total_allocated_objects]
p b = after - before # 120004
p "Difference: #{a - b}" # "Difference: 29997"
```
Things get even more "problematic" when referencing like above is not used on batches but on each of the objects separately:
```ruby
# frozen_string_literal: true
GC.disable
class Parse
def self.call(string)
string.to_f
end
end
class Normalize
def self.call(number)
number.round
end
end
class Transform
def self.call(int)
int * 2
end
end
# Simulate a long running data producing source with batch results
stream = Array.new(10_000) { Array.new(100) { '100.2' } }
before = GC.stat[:total_allocated_objects]
# This will provide batches to be processed, let's assume an endless data-source
stream.each do |batch|
batch.map do |message|
message
.then(&Parse.:call)
.then(&Normalize.:call)
.then(&Transform.:call)
end
end
after = GC.stat[:total_allocated_objects]
p a = after - before # 12010002
before = GC.stat[:total_allocated_objects]
# This will provide batches to be processed, let's assume an endless data-source
parse = Parse.:call
normalize = Normalize.:call
transform = Transform.:call
stream.each do |batch|
batch
.map(&parse)
.map(&normalize)
.map(&transform)
end
after = GC.stat[:total_allocated_objects]
p b = after - before # 120004
p "Difference: #{a - b}" # "Difference: 11889998"
```
I am aware, that this won't help in case we don't reuse the same object method references multiple times but based on many use-cases from here https://bugs.ruby-lang.org/issues/13581 and my own when building data-intense applications, it's not uncommon to build "pipelines" of things applied to each of the batches as a set of functions.
Apart from that I would also vote on freezing the `.method(:method)` result object as well, but I know Matz was against.
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 4+ messages in thread
* [ruby-core:94658] [Ruby master Feature#16103] Make the dot-colon method reference frozen
[not found] <redmine.issue-16103.20190814122309@ruby-lang.org>
2019-08-14 12:23 ` [ruby-core:94349] [Ruby master Feature#16103] Make the dot-colon method reference frozen maciej
@ 2019-08-29 7:37 ` ko1
2019-08-29 7:37 ` [ruby-core:94659] " ko1
2019-11-05 21:16 ` [ruby-core:95709] " eregontp
3 siblings, 0 replies; 4+ messages in thread
From: ko1 @ 2019-08-29 7:37 UTC (permalink / raw)
To: ruby-core
Issue #16103 has been updated by ko1 (Koichi Sasada).
Matz accepted it :tada:
----------------------------------------
Feature #16103: Make the dot-colon method reference frozen
https://bugs.ruby-lang.org/issues/16103#change-81260
* Author: maciej.mensfeld (Maciej Mensfeld)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
----------------------------------------
I made a PR to freeze the dot-colon method reference result object (https://github.com/ruby/ruby/pull/2267). Nobu asked to make an issue out of that. I initially discussed that with Matz and Ko1 during the hack challenge in Bristol.
Here are some of the reasons why I think it should be done before releasing this feature:
- It's worth keeping the bound methods frozen as one should not modify them in general
- Freezing keeps a window of opportunity for introducing method reference caching in case it would be needed because the method object is immutable.
- I want to work on the "last method" reference cache for that feature when we see more use-cases after 2.7 is released and without having it frozen, the cost and complexity are probably higher than the outcome.
Example of the misuse that makes GC life much harder (note, that freezing does not fix that, but allows further work related to this issue):
```ruby
# frozen_string_literal: true
GC.disable
class Parse
def self.call(string)
string.to_f
end
end
class Normalize
def self.call(number)
number.round
end
end
class Transform
def self.call(int)
int * 2
end
end
# Simulate a long running data producing source with batch results
stream = Array.new(10_000) { Array.new(100) { '100.2' } }
before = GC.stat[:total_allocated_objects]
# This will provide batches to be processed, let's assume an endless data-source
stream.each do |batch|
batch
.map(&Parse.:call)
.map(&Normalize.:call)
.map(&Transform.:call)
end
after = GC.stat[:total_allocated_objects]
p a = after - before # 150001
before = GC.stat[:total_allocated_objects]
# "cache"
parse = Parse.:call
normalize = Normalize.:call
transform = Transform.:call
stream.each do |batch|
batch
.map(&parse)
.map(&normalize)
.map(&transform)
end
after = GC.stat[:total_allocated_objects]
p b = after - before # 120004
p "Difference: #{a - b}" # "Difference: 29997"
```
Things get even more "problematic" when referencing like above is not used on batches but on each of the objects separately:
```ruby
# frozen_string_literal: true
GC.disable
class Parse
def self.call(string)
string.to_f
end
end
class Normalize
def self.call(number)
number.round
end
end
class Transform
def self.call(int)
int * 2
end
end
# Simulate a long running data producing source with batch results
stream = Array.new(10_000) { Array.new(100) { '100.2' } }
before = GC.stat[:total_allocated_objects]
# This will provide batches to be processed, let's assume an endless data-source
stream.each do |batch|
batch.map do |message|
message
.then(&Parse.:call)
.then(&Normalize.:call)
.then(&Transform.:call)
end
end
after = GC.stat[:total_allocated_objects]
p a = after - before # 12010002
before = GC.stat[:total_allocated_objects]
# This will provide batches to be processed, let's assume an endless data-source
parse = Parse.:call
normalize = Normalize.:call
transform = Transform.:call
stream.each do |batch|
batch
.map(&parse)
.map(&normalize)
.map(&transform)
end
after = GC.stat[:total_allocated_objects]
p b = after - before # 120004
p "Difference: #{a - b}" # "Difference: 11889998"
```
I am aware, that this won't help in case we don't reuse the same object method references multiple times but based on many use-cases from here https://bugs.ruby-lang.org/issues/13581 and my own when building data-intense applications, it's not uncommon to build "pipelines" of things applied to each of the batches as a set of functions.
Apart from that I would also vote on freezing the `.method(:method)` result object as well, but I know Matz was against.
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 4+ messages in thread
* [ruby-core:94659] [Ruby master Feature#16103] Make the dot-colon method reference frozen
[not found] <redmine.issue-16103.20190814122309@ruby-lang.org>
2019-08-14 12:23 ` [ruby-core:94349] [Ruby master Feature#16103] Make the dot-colon method reference frozen maciej
2019-08-29 7:37 ` [ruby-core:94658] " ko1
@ 2019-08-29 7:37 ` ko1
2019-11-05 21:16 ` [ruby-core:95709] " eregontp
3 siblings, 0 replies; 4+ messages in thread
From: ko1 @ 2019-08-29 7:37 UTC (permalink / raw)
To: ruby-core
Issue #16103 has been updated by ko1 (Koichi Sasada).
Assignee set to ko1 (Koichi Sasada)
----------------------------------------
Feature #16103: Make the dot-colon method reference frozen
https://bugs.ruby-lang.org/issues/16103#change-81261
* Author: maciej.mensfeld (Maciej Mensfeld)
* Status: Open
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* Target version:
----------------------------------------
I made a PR to freeze the dot-colon method reference result object (https://github.com/ruby/ruby/pull/2267). Nobu asked to make an issue out of that. I initially discussed that with Matz and Ko1 during the hack challenge in Bristol.
Here are some of the reasons why I think it should be done before releasing this feature:
- It's worth keeping the bound methods frozen as one should not modify them in general
- Freezing keeps a window of opportunity for introducing method reference caching in case it would be needed because the method object is immutable.
- I want to work on the "last method" reference cache for that feature when we see more use-cases after 2.7 is released and without having it frozen, the cost and complexity are probably higher than the outcome.
Example of the misuse that makes GC life much harder (note, that freezing does not fix that, but allows further work related to this issue):
```ruby
# frozen_string_literal: true
GC.disable
class Parse
def self.call(string)
string.to_f
end
end
class Normalize
def self.call(number)
number.round
end
end
class Transform
def self.call(int)
int * 2
end
end
# Simulate a long running data producing source with batch results
stream = Array.new(10_000) { Array.new(100) { '100.2' } }
before = GC.stat[:total_allocated_objects]
# This will provide batches to be processed, let's assume an endless data-source
stream.each do |batch|
batch
.map(&Parse.:call)
.map(&Normalize.:call)
.map(&Transform.:call)
end
after = GC.stat[:total_allocated_objects]
p a = after - before # 150001
before = GC.stat[:total_allocated_objects]
# "cache"
parse = Parse.:call
normalize = Normalize.:call
transform = Transform.:call
stream.each do |batch|
batch
.map(&parse)
.map(&normalize)
.map(&transform)
end
after = GC.stat[:total_allocated_objects]
p b = after - before # 120004
p "Difference: #{a - b}" # "Difference: 29997"
```
Things get even more "problematic" when referencing like above is not used on batches but on each of the objects separately:
```ruby
# frozen_string_literal: true
GC.disable
class Parse
def self.call(string)
string.to_f
end
end
class Normalize
def self.call(number)
number.round
end
end
class Transform
def self.call(int)
int * 2
end
end
# Simulate a long running data producing source with batch results
stream = Array.new(10_000) { Array.new(100) { '100.2' } }
before = GC.stat[:total_allocated_objects]
# This will provide batches to be processed, let's assume an endless data-source
stream.each do |batch|
batch.map do |message|
message
.then(&Parse.:call)
.then(&Normalize.:call)
.then(&Transform.:call)
end
end
after = GC.stat[:total_allocated_objects]
p a = after - before # 12010002
before = GC.stat[:total_allocated_objects]
# This will provide batches to be processed, let's assume an endless data-source
parse = Parse.:call
normalize = Normalize.:call
transform = Transform.:call
stream.each do |batch|
batch
.map(&parse)
.map(&normalize)
.map(&transform)
end
after = GC.stat[:total_allocated_objects]
p b = after - before # 120004
p "Difference: #{a - b}" # "Difference: 11889998"
```
I am aware, that this won't help in case we don't reuse the same object method references multiple times but based on many use-cases from here https://bugs.ruby-lang.org/issues/13581 and my own when building data-intense applications, it's not uncommon to build "pipelines" of things applied to each of the batches as a set of functions.
Apart from that I would also vote on freezing the `.method(:method)` result object as well, but I know Matz was against.
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 4+ messages in thread
* [ruby-core:95709] [Ruby master Feature#16103] Make the dot-colon method reference frozen
[not found] <redmine.issue-16103.20190814122309@ruby-lang.org>
` (2 preceding siblings ...)
2019-08-29 7:37 ` [ruby-core:94659] " ko1
@ 2019-11-05 21:16 ` eregontp
3 siblings, 0 replies; 4+ messages in thread
From: eregontp @ 2019-11-05 21:16 UTC (permalink / raw)
To: ruby-core
Issue #16103 has been updated by Eregon (Benoit Daloze).
FWIW, I think it would be better to freeze all Method and UnboundMethod objects.
Is there any use-case for adding instance variables to them?
----------------------------------------
Feature #16103: Make the dot-colon method reference frozen
https://bugs.ruby-lang.org/issues/16103#change-82493
* Author: maciej.mensfeld (Maciej Mensfeld)
* Status: Closed
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* Target version:
----------------------------------------
I made a PR to freeze the dot-colon method reference result object (https://github.com/ruby/ruby/pull/2267). Nobu asked to make an issue out of that. I initially discussed that with Matz and Ko1 during the hack challenge in Bristol.
Here are some of the reasons why I think it should be done before releasing this feature:
- It's worth keeping the bound methods frozen as one should not modify them in general
- Freezing keeps a window of opportunity for introducing method reference caching in case it would be needed because the method object is immutable.
- I want to work on the "last method" reference cache for that feature when we see more use-cases after 2.7 is released and without having it frozen, the cost and complexity are probably higher than the outcome.
Example of the misuse that makes GC life much harder (note, that freezing does not fix that, but allows further work related to this issue):
```ruby
# frozen_string_literal: true
GC.disable
class Parse
def self.call(string)
string.to_f
end
end
class Normalize
def self.call(number)
number.round
end
end
class Transform
def self.call(int)
int * 2
end
end
# Simulate a long running data producing source with batch results
stream = Array.new(10_000) { Array.new(100) { '100.2' } }
before = GC.stat[:total_allocated_objects]
# This will provide batches to be processed, let's assume an endless data-source
stream.each do |batch|
batch
.map(&Parse.:call)
.map(&Normalize.:call)
.map(&Transform.:call)
end
after = GC.stat[:total_allocated_objects]
p a = after - before # 150001
before = GC.stat[:total_allocated_objects]
# "cache"
parse = Parse.:call
normalize = Normalize.:call
transform = Transform.:call
stream.each do |batch|
batch
.map(&parse)
.map(&normalize)
.map(&transform)
end
after = GC.stat[:total_allocated_objects]
p b = after - before # 120004
p "Difference: #{a - b}" # "Difference: 29997"
```
Things get even more "problematic" when referencing like above is not used on batches but on each of the objects separately:
```ruby
# frozen_string_literal: true
GC.disable
class Parse
def self.call(string)
string.to_f
end
end
class Normalize
def self.call(number)
number.round
end
end
class Transform
def self.call(int)
int * 2
end
end
# Simulate a long running data producing source with batch results
stream = Array.new(10_000) { Array.new(100) { '100.2' } }
before = GC.stat[:total_allocated_objects]
# This will provide batches to be processed, let's assume an endless data-source
stream.each do |batch|
batch.map do |message|
message
.then(&Parse.:call)
.then(&Normalize.:call)
.then(&Transform.:call)
end
end
after = GC.stat[:total_allocated_objects]
p a = after - before # 12010002
before = GC.stat[:total_allocated_objects]
# This will provide batches to be processed, let's assume an endless data-source
parse = Parse.:call
normalize = Normalize.:call
transform = Transform.:call
stream.each do |batch|
batch
.map(&parse)
.map(&normalize)
.map(&transform)
end
after = GC.stat[:total_allocated_objects]
p b = after - before # 120004
p "Difference: #{a - b}" # "Difference: 11889998"
```
I am aware, that this won't help in case we don't reuse the same object method references multiple times but based on many use-cases from here https://bugs.ruby-lang.org/issues/13581 and my own when building data-intense applications, it's not uncommon to build "pipelines" of things applied to each of the batches as a set of functions.
Apart from that I would also vote on freezing the `.method(:method)` result object as well, but I know Matz was against.
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2019-11-05 21:16 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <redmine.issue-16103.20190814122309@ruby-lang.org>
2019-08-14 12:23 ` [ruby-core:94349] [Ruby master Feature#16103] Make the dot-colon method reference frozen maciej
2019-08-29 7:37 ` [ruby-core:94658] " ko1
2019-08-29 7:37 ` [ruby-core:94659] " ko1
2019-11-05 21:16 ` [ruby-core:95709] " eregontp
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).