* [ruby-core:87907] [Ruby trunk Bug#14908] Enumerator::Lazy creates unnecessary Array objects.
[not found] <redmine.issue-14908.20180711132323@ruby-lang.org>
@ 2018-07-11 13:23 ` chopraanmol1
2018-07-12 0:12 ` [ruby-core:87914] " shyouhei
` (8 subsequent siblings)
9 siblings, 0 replies; 10+ messages in thread
From: chopraanmol1 @ 2018-07-11 13:23 UTC (permalink / raw
To: ruby-core
Issue #14908 has been reported by chopraanmol1 (Anmol Chopra).
----------------------------------------
Bug #14908: Enumerator::Lazy creates unnecessary Array objects.
https://bugs.ruby-lang.org/issues/14908
* Author: chopraanmol1 (Anmol Chopra)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.6.0dev (2018-07-11 trunk 63949) [x86_64-linux]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Benchmark result on trunk:
~~~
user system total real
Lazy: 0.120000 0.000000 0.120000 ( 0.119958)
Normal: 0.056000 0.004000 0.060000 ( 0.062848)
2.142857 0.000000 NaN ( 1.908698)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 122240 bytes (3033 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
120480 Array
880 Proc
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
3012 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
As you may observe an extra array is created for every final element. Enumerator::Yielder#yield method has arity of -1 which wrap every elements in array. The same goes for Enumerator::Yielder#<< method, I'm proposing to change arity for Enumerator::Yielder#<< to 1 from -1. It will also make << method definition consistent with other classes(Array, String & etc).
I've applied the following set of changes to trunk:
~~~
diff --git a/enumerator.c b/enumerator.c
index 050a9ce58f..c21d6f36f1 100644
--- a/enumerator.c
+++ b/enumerator.c
@@ -103,7 +103,7 @@
*/
VALUE rb_cEnumerator;
static VALUE rb_cLazy;
-static ID id_rewind, id_new, id_yield, id_to_enum;
+static ID id_rewind, id_new, id_yield, id_yield_push, id_to_enum;
static ID id_next, id_result, id_lazy, id_receiver, id_arguments, id_memo, id_method, id_force;
static VALUE sym_each, sym_cycle;
@@ -1265,9 +1265,14 @@ yielder_yield(VALUE obj, VALUE args)
/* :nodoc: */
static VALUE
-yielder_yield_push(VALUE obj, VALUE args)
+yielder_yield_push(VALUE obj, VALUE arg)
{
- yielder_yield(obj, args);
+ // VALUE args[1];
+ struct yielder *ptr = yielder_ptr(obj);
+ // args[0] = arg;
+
+ rb_proc_call_with_block(ptr->proc, 1, &arg, Qnil);
+
return obj;
}
@@ -1510,7 +1515,7 @@ lazy_init_yielder(VALUE val, VALUE m, int argc, VALUE *argv)
}
if (cont) {
- rb_funcall2(yielder, id_yield, 1, &(result->memo_value));
+ rb_funcall2(yielder, id_yield_push, 1, &(result->memo_value));
}
if (LAZY_MEMO_BREAK_P(result)) {
rb_iter_break();
@@ -2448,7 +2453,7 @@ InitVM_Enumerator(void)
rb_define_alloc_func(rb_cYielder, yielder_allocate);
rb_define_method(rb_cYielder, "initialize", yielder_initialize, 0);
rb_define_method(rb_cYielder, "yield", yielder_yield, -2);
- rb_define_method(rb_cYielder, "<<", yielder_yield_push, -2);
+ rb_define_method(rb_cYielder, "<<", yielder_yield_push, 1);
rb_provide("enumerator.so"); /* for backward compatibility */
}
@@ -2459,6 +2464,7 @@ Init_Enumerator(void)
{
id_rewind = rb_intern("rewind");
id_yield = rb_intern("yield");
+ id_yield_push = rb_intern("<<");
id_new = rb_intern("new");
id_next = rb_intern("next");
id_result = rb_intern("result");
~~~
which result in the following:
~~~
user system total real
Lazy: 0.108000 0.000000 0.108000 ( 0.108484)
Normal: 0.052000 0.008000 0.060000 ( 0.062528)
2.076923 0.000000 NaN ( 1.734961)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 2240 bytes (33 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
880 Proc
480 Array
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
12 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
This changes reduces the memory utilization and also by a tiny fraction improves performance for the lazy enumerator
---Files--------------------------------
lazy_test.rb (1.26 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [ruby-core:87914] [Ruby trunk Bug#14908] Enumerator::Lazy creates unnecessary Array objects.
[not found] <redmine.issue-14908.20180711132323@ruby-lang.org>
2018-07-11 13:23 ` [ruby-core:87907] [Ruby trunk Bug#14908] Enumerator::Lazy creates unnecessary Array objects chopraanmol1
@ 2018-07-12 0:12 ` shyouhei
2018-07-12 5:03 ` [ruby-core:87918] " chopraanmol1
` (7 subsequent siblings)
9 siblings, 0 replies; 10+ messages in thread
From: shyouhei @ 2018-07-12 0:12 UTC (permalink / raw
To: ruby-core
Issue #14908 has been updated by shyouhei (Shyouhei Urabe).
Understand the problem. The proposed fix however involves spec change. We need to discuss effects of it before applying this patch.
----------------------------------------
Bug #14908: Enumerator::Lazy creates unnecessary Array objects.
https://bugs.ruby-lang.org/issues/14908#change-72924
* Author: chopraanmol1 (Anmol Chopra)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.6.0dev (2018-07-11 trunk 63949) [x86_64-linux]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Benchmark result on trunk:
~~~
user system total real
Lazy: 0.120000 0.000000 0.120000 ( 0.119958)
Normal: 0.056000 0.004000 0.060000 ( 0.062848)
2.142857 0.000000 NaN ( 1.908698)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 122240 bytes (3033 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
120480 Array
880 Proc
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
3012 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
As you may observe an extra array is created for every final element. Enumerator::Yielder#yield method has arity of -1 which wrap every elements in array. The same goes for Enumerator::Yielder#<< method, I'm proposing to change arity for Enumerator::Yielder#<< to 1 from -1. It will also make << method definition consistent with other classes(Array, String & etc).
I've applied the following set of changes to trunk:
~~~
diff --git a/enumerator.c b/enumerator.c
index 050a9ce58f..c21d6f36f1 100644
--- a/enumerator.c
+++ b/enumerator.c
@@ -103,7 +103,7 @@
*/
VALUE rb_cEnumerator;
static VALUE rb_cLazy;
-static ID id_rewind, id_new, id_yield, id_to_enum;
+static ID id_rewind, id_new, id_yield, id_yield_push, id_to_enum;
static ID id_next, id_result, id_lazy, id_receiver, id_arguments, id_memo, id_method, id_force;
static VALUE sym_each, sym_cycle;
@@ -1265,9 +1265,14 @@ yielder_yield(VALUE obj, VALUE args)
/* :nodoc: */
static VALUE
-yielder_yield_push(VALUE obj, VALUE args)
+yielder_yield_push(VALUE obj, VALUE arg)
{
- yielder_yield(obj, args);
+ struct yielder *ptr = yielder_ptr(obj);
+
+ rb_proc_call_with_block(ptr->proc, 1, &arg, Qnil);
+
return obj;
}
@@ -1510,7 +1515,7 @@ lazy_init_yielder(VALUE val, VALUE m, int argc, VALUE *argv)
}
if (cont) {
- rb_funcall2(yielder, id_yield, 1, &(result->memo_value));
+ rb_funcall2(yielder, id_yield_push, 1, &(result->memo_value));
}
if (LAZY_MEMO_BREAK_P(result)) {
rb_iter_break();
@@ -2448,7 +2453,7 @@ InitVM_Enumerator(void)
rb_define_alloc_func(rb_cYielder, yielder_allocate);
rb_define_method(rb_cYielder, "initialize", yielder_initialize, 0);
rb_define_method(rb_cYielder, "yield", yielder_yield, -2);
- rb_define_method(rb_cYielder, "<<", yielder_yield_push, -2);
+ rb_define_method(rb_cYielder, "<<", yielder_yield_push, 1);
rb_provide("enumerator.so"); /* for backward compatibility */
}
@@ -2459,6 +2464,7 @@ Init_Enumerator(void)
{
id_rewind = rb_intern("rewind");
id_yield = rb_intern("yield");
+ id_yield_push = rb_intern("<<");
id_new = rb_intern("new");
id_next = rb_intern("next");
id_result = rb_intern("result");
~~~
which result in the following:
~~~
user system total real
Lazy: 0.108000 0.000000 0.108000 ( 0.108484)
Normal: 0.052000 0.008000 0.060000 ( 0.062528)
2.076923 0.000000 NaN ( 1.734961)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 2240 bytes (33 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
880 Proc
480 Array
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
12 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
This changes reduces the memory utilization and also by a tiny fraction improves performance for the lazy enumerator
---Files--------------------------------
lazy_test.rb (1.26 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [ruby-core:87918] [Ruby trunk Bug#14908] Enumerator::Lazy creates unnecessary Array objects.
[not found] <redmine.issue-14908.20180711132323@ruby-lang.org>
2018-07-11 13:23 ` [ruby-core:87907] [Ruby trunk Bug#14908] Enumerator::Lazy creates unnecessary Array objects chopraanmol1
2018-07-12 0:12 ` [ruby-core:87914] " shyouhei
@ 2018-07-12 5:03 ` chopraanmol1
2018-07-12 9:25 ` [ruby-core:87924] " eregontp
` (6 subsequent siblings)
9 siblings, 0 replies; 10+ messages in thread
From: chopraanmol1 @ 2018-07-12 5:03 UTC (permalink / raw
To: ruby-core
Issue #14908 has been updated by chopraanmol1 (Anmol Chopra).
shyouhei (Shyouhei Urabe) wrote:
> Understand the problem. The proposed fix however involves spec change. We need to discuss effects of it before applying this patch.
If this is a big breaking change than alternate will be to create a different method for Enumerator::Lazy's internal use. I'm also up for updating the patch to reflect that. But I think for future release it will make more sense to have Enumerator::Yielder#<< to have an arity of 1 if you consider syntax use case.
But for the backporting purpose, I'm more inclined to create a new method.
----------------------------------------
Bug #14908: Enumerator::Lazy creates unnecessary Array objects.
https://bugs.ruby-lang.org/issues/14908#change-72928
* Author: chopraanmol1 (Anmol Chopra)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.6.0dev (2018-07-11 trunk 63949) [x86_64-linux]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Benchmark result on trunk:
~~~
user system total real
Lazy: 0.120000 0.000000 0.120000 ( 0.119958)
Normal: 0.056000 0.004000 0.060000 ( 0.062848)
2.142857 0.000000 NaN ( 1.908698)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 122240 bytes (3033 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
120480 Array
880 Proc
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
3012 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
As you may observe an extra array is created for every final element. Enumerator::Yielder#yield method has arity of -1 which wrap every elements in array. The same goes for Enumerator::Yielder#<< method, I'm proposing to change arity for Enumerator::Yielder#<< to 1 from -1. It will also make << method definition consistent with other classes(Array, String & etc).
I've applied the following set of changes to trunk:
~~~
diff --git a/enumerator.c b/enumerator.c
index 050a9ce58f..c21d6f36f1 100644
--- a/enumerator.c
+++ b/enumerator.c
@@ -103,7 +103,7 @@
*/
VALUE rb_cEnumerator;
static VALUE rb_cLazy;
-static ID id_rewind, id_new, id_yield, id_to_enum;
+static ID id_rewind, id_new, id_yield, id_yield_push, id_to_enum;
static ID id_next, id_result, id_lazy, id_receiver, id_arguments, id_memo, id_method, id_force;
static VALUE sym_each, sym_cycle;
@@ -1265,9 +1265,14 @@ yielder_yield(VALUE obj, VALUE args)
/* :nodoc: */
static VALUE
-yielder_yield_push(VALUE obj, VALUE args)
+yielder_yield_push(VALUE obj, VALUE arg)
{
- yielder_yield(obj, args);
+ struct yielder *ptr = yielder_ptr(obj);
+
+ rb_proc_call_with_block(ptr->proc, 1, &arg, Qnil);
+
return obj;
}
@@ -1510,7 +1515,7 @@ lazy_init_yielder(VALUE val, VALUE m, int argc, VALUE *argv)
}
if (cont) {
- rb_funcall2(yielder, id_yield, 1, &(result->memo_value));
+ rb_funcall2(yielder, id_yield_push, 1, &(result->memo_value));
}
if (LAZY_MEMO_BREAK_P(result)) {
rb_iter_break();
@@ -2448,7 +2453,7 @@ InitVM_Enumerator(void)
rb_define_alloc_func(rb_cYielder, yielder_allocate);
rb_define_method(rb_cYielder, "initialize", yielder_initialize, 0);
rb_define_method(rb_cYielder, "yield", yielder_yield, -2);
- rb_define_method(rb_cYielder, "<<", yielder_yield_push, -2);
+ rb_define_method(rb_cYielder, "<<", yielder_yield_push, 1);
rb_provide("enumerator.so"); /* for backward compatibility */
}
@@ -2459,6 +2464,7 @@ Init_Enumerator(void)
{
id_rewind = rb_intern("rewind");
id_yield = rb_intern("yield");
+ id_yield_push = rb_intern("<<");
id_new = rb_intern("new");
id_next = rb_intern("next");
id_result = rb_intern("result");
~~~
which result in the following:
~~~
user system total real
Lazy: 0.108000 0.000000 0.108000 ( 0.108484)
Normal: 0.052000 0.008000 0.060000 ( 0.062528)
2.076923 0.000000 NaN ( 1.734961)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 2240 bytes (33 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
880 Proc
480 Array
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
12 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
This changes reduces the memory utilization and also by a tiny fraction improves performance for the lazy enumerator
---Files--------------------------------
lazy_test.rb (1.26 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [ruby-core:87924] [Ruby trunk Bug#14908] Enumerator::Lazy creates unnecessary Array objects.
[not found] <redmine.issue-14908.20180711132323@ruby-lang.org>
` (2 preceding siblings ...)
2018-07-12 5:03 ` [ruby-core:87918] " chopraanmol1
@ 2018-07-12 9:25 ` eregontp
2018-07-13 4:46 ` [ruby-core:87933] " chopraanmol1
` (5 subsequent siblings)
9 siblings, 0 replies; 10+ messages in thread
From: eregontp @ 2018-07-12 9:25 UTC (permalink / raw
To: ruby-core
Issue #14908 has been updated by Eregon (Benoit Daloze).
Changing Enumerator::Yielder#<< to have arity 1 seems fine to me, as I guess nobody calls << on an Enumerator::Yielder with more than 1 argument, isn't it?
----------------------------------------
Bug #14908: Enumerator::Lazy creates unnecessary Array objects.
https://bugs.ruby-lang.org/issues/14908#change-72934
* Author: chopraanmol1 (Anmol Chopra)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.6.0dev (2018-07-11 trunk 63949) [x86_64-linux]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Benchmark result on trunk:
~~~
user system total real
Lazy: 0.120000 0.000000 0.120000 ( 0.119958)
Normal: 0.056000 0.004000 0.060000 ( 0.062848)
2.142857 0.000000 NaN ( 1.908698)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 122240 bytes (3033 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
120480 Array
880 Proc
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
3012 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
As you may observe an extra array is created for every final element. Enumerator::Yielder#yield method has arity of -1 which wrap every elements in array. The same goes for Enumerator::Yielder#<< method, I'm proposing to change arity for Enumerator::Yielder#<< to 1 from -1. It will also make << method definition consistent with other classes(Array, String & etc).
I've applied the following set of changes to trunk:
~~~
diff --git a/enumerator.c b/enumerator.c
index 050a9ce58f..c21d6f36f1 100644
--- a/enumerator.c
+++ b/enumerator.c
@@ -103,7 +103,7 @@
*/
VALUE rb_cEnumerator;
static VALUE rb_cLazy;
-static ID id_rewind, id_new, id_yield, id_to_enum;
+static ID id_rewind, id_new, id_yield, id_yield_push, id_to_enum;
static ID id_next, id_result, id_lazy, id_receiver, id_arguments, id_memo, id_method, id_force;
static VALUE sym_each, sym_cycle;
@@ -1265,9 +1265,14 @@ yielder_yield(VALUE obj, VALUE args)
/* :nodoc: */
static VALUE
-yielder_yield_push(VALUE obj, VALUE args)
+yielder_yield_push(VALUE obj, VALUE arg)
{
- yielder_yield(obj, args);
+ struct yielder *ptr = yielder_ptr(obj);
+
+ rb_proc_call_with_block(ptr->proc, 1, &arg, Qnil);
+
return obj;
}
@@ -1510,7 +1515,7 @@ lazy_init_yielder(VALUE val, VALUE m, int argc, VALUE *argv)
}
if (cont) {
- rb_funcall2(yielder, id_yield, 1, &(result->memo_value));
+ rb_funcall2(yielder, id_yield_push, 1, &(result->memo_value));
}
if (LAZY_MEMO_BREAK_P(result)) {
rb_iter_break();
@@ -2448,7 +2453,7 @@ InitVM_Enumerator(void)
rb_define_alloc_func(rb_cYielder, yielder_allocate);
rb_define_method(rb_cYielder, "initialize", yielder_initialize, 0);
rb_define_method(rb_cYielder, "yield", yielder_yield, -2);
- rb_define_method(rb_cYielder, "<<", yielder_yield_push, -2);
+ rb_define_method(rb_cYielder, "<<", yielder_yield_push, 1);
rb_provide("enumerator.so"); /* for backward compatibility */
}
@@ -2459,6 +2464,7 @@ Init_Enumerator(void)
{
id_rewind = rb_intern("rewind");
id_yield = rb_intern("yield");
+ id_yield_push = rb_intern("<<");
id_new = rb_intern("new");
id_next = rb_intern("next");
id_result = rb_intern("result");
~~~
which result in the following:
~~~
user system total real
Lazy: 0.108000 0.000000 0.108000 ( 0.108484)
Normal: 0.052000 0.008000 0.060000 ( 0.062528)
2.076923 0.000000 NaN ( 1.734961)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 2240 bytes (33 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
880 Proc
480 Array
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
12 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
This changes reduces the memory utilization and also by a tiny fraction improves performance for the lazy enumerator
---Files--------------------------------
lazy_test.rb (1.26 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [ruby-core:87933] [Ruby trunk Bug#14908] Enumerator::Lazy creates unnecessary Array objects.
[not found] <redmine.issue-14908.20180711132323@ruby-lang.org>
` (3 preceding siblings ...)
2018-07-12 9:25 ` [ruby-core:87924] " eregontp
@ 2018-07-13 4:46 ` chopraanmol1
2018-07-13 4:59 ` [ruby-core:87936] " ruby-core
` (4 subsequent siblings)
9 siblings, 0 replies; 10+ messages in thread
From: chopraanmol1 @ 2018-07-13 4:46 UTC (permalink / raw
To: ruby-core
Issue #14908 has been updated by chopraanmol1 (Anmol Chopra).
Eregon (Benoit Daloze) wrote:
> Changing Enumerator::Yielder#<< to have arity 1 seems fine to me, as I guess nobody calls << on an Enumerator::Yielder with more than 1 argument, isn't it?
Yes, that will be the general case. Exception:
~~~ ruby
.send(:<<,...)
~~~
~~~ ruby
.<<(...)
~~~
----------------------------------------
Bug #14908: Enumerator::Lazy creates unnecessary Array objects.
https://bugs.ruby-lang.org/issues/14908#change-72940
* Author: chopraanmol1 (Anmol Chopra)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.6.0dev (2018-07-11 trunk 63949) [x86_64-linux]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Benchmark result on trunk:
~~~
user system total real
Lazy: 0.120000 0.000000 0.120000 ( 0.119958)
Normal: 0.056000 0.004000 0.060000 ( 0.062848)
2.142857 0.000000 NaN ( 1.908698)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 122240 bytes (3033 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
120480 Array
880 Proc
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
3012 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
As you may observe an extra array is created for every final element. Enumerator::Yielder#yield method has arity of -1 which wrap every elements in array. The same goes for Enumerator::Yielder#<< method, I'm proposing to change arity for Enumerator::Yielder#<< to 1 from -1. It will also make << method definition consistent with other classes(Array, String & etc).
I've applied the following set of changes to trunk:
~~~
diff --git a/enumerator.c b/enumerator.c
index 050a9ce58f..c21d6f36f1 100644
--- a/enumerator.c
+++ b/enumerator.c
@@ -103,7 +103,7 @@
*/
VALUE rb_cEnumerator;
static VALUE rb_cLazy;
-static ID id_rewind, id_new, id_yield, id_to_enum;
+static ID id_rewind, id_new, id_yield, id_yield_push, id_to_enum;
static ID id_next, id_result, id_lazy, id_receiver, id_arguments, id_memo, id_method, id_force;
static VALUE sym_each, sym_cycle;
@@ -1265,9 +1265,14 @@ yielder_yield(VALUE obj, VALUE args)
/* :nodoc: */
static VALUE
-yielder_yield_push(VALUE obj, VALUE args)
+yielder_yield_push(VALUE obj, VALUE arg)
{
- yielder_yield(obj, args);
+ struct yielder *ptr = yielder_ptr(obj);
+
+ rb_proc_call_with_block(ptr->proc, 1, &arg, Qnil);
+
return obj;
}
@@ -1510,7 +1515,7 @@ lazy_init_yielder(VALUE val, VALUE m, int argc, VALUE *argv)
}
if (cont) {
- rb_funcall2(yielder, id_yield, 1, &(result->memo_value));
+ rb_funcall2(yielder, id_yield_push, 1, &(result->memo_value));
}
if (LAZY_MEMO_BREAK_P(result)) {
rb_iter_break();
@@ -2448,7 +2453,7 @@ InitVM_Enumerator(void)
rb_define_alloc_func(rb_cYielder, yielder_allocate);
rb_define_method(rb_cYielder, "initialize", yielder_initialize, 0);
rb_define_method(rb_cYielder, "yield", yielder_yield, -2);
- rb_define_method(rb_cYielder, "<<", yielder_yield_push, -2);
+ rb_define_method(rb_cYielder, "<<", yielder_yield_push, 1);
rb_provide("enumerator.so"); /* for backward compatibility */
}
@@ -2459,6 +2464,7 @@ Init_Enumerator(void)
{
id_rewind = rb_intern("rewind");
id_yield = rb_intern("yield");
+ id_yield_push = rb_intern("<<");
id_new = rb_intern("new");
id_next = rb_intern("next");
id_result = rb_intern("result");
~~~
which result in the following:
~~~
user system total real
Lazy: 0.108000 0.000000 0.108000 ( 0.108484)
Normal: 0.052000 0.008000 0.060000 ( 0.062528)
2.076923 0.000000 NaN ( 1.734961)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 2240 bytes (33 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
880 Proc
480 Array
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
12 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
This changes reduces the memory utilization and also by a tiny fraction improves performance for the lazy enumerator
---Files--------------------------------
lazy_test.rb (1.26 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [ruby-core:87936] [Ruby trunk Bug#14908] Enumerator::Lazy creates unnecessary Array objects.
[not found] <redmine.issue-14908.20180711132323@ruby-lang.org>
` (4 preceding siblings ...)
2018-07-13 4:46 ` [ruby-core:87933] " chopraanmol1
@ 2018-07-13 4:59 ` ruby-core
2018-08-01 7:44 ` [ruby-core:88254] " kelvinbhoss
` (3 subsequent siblings)
9 siblings, 0 replies; 10+ messages in thread
From: ruby-core @ 2018-07-13 4:59 UTC (permalink / raw
To: ruby-core
Issue #14908 has been updated by marcandre (Marc-Andre Lafortune).
Indeed, as long as `Yielder#yield` is kept with arity -1 (as in this patch), indeed I don't think that would be an "incompatibility" we should worry about.
----------------------------------------
Bug #14908: Enumerator::Lazy creates unnecessary Array objects.
https://bugs.ruby-lang.org/issues/14908#change-72943
* Author: chopraanmol1 (Anmol Chopra)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.6.0dev (2018-07-11 trunk 63949) [x86_64-linux]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Benchmark result on trunk:
~~~
user system total real
Lazy: 0.120000 0.000000 0.120000 ( 0.119958)
Normal: 0.056000 0.004000 0.060000 ( 0.062848)
2.142857 0.000000 NaN ( 1.908698)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 122240 bytes (3033 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
120480 Array
880 Proc
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
3012 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
As you may observe an extra array is created for every final element. Enumerator::Yielder#yield method has arity of -1 which wrap every elements in array. The same goes for Enumerator::Yielder#<< method, I'm proposing to change arity for Enumerator::Yielder#<< to 1 from -1. It will also make << method definition consistent with other classes(Array, String & etc).
I've applied the following set of changes to trunk:
~~~
diff --git a/enumerator.c b/enumerator.c
index 050a9ce58f..c21d6f36f1 100644
--- a/enumerator.c
+++ b/enumerator.c
@@ -103,7 +103,7 @@
*/
VALUE rb_cEnumerator;
static VALUE rb_cLazy;
-static ID id_rewind, id_new, id_yield, id_to_enum;
+static ID id_rewind, id_new, id_yield, id_yield_push, id_to_enum;
static ID id_next, id_result, id_lazy, id_receiver, id_arguments, id_memo, id_method, id_force;
static VALUE sym_each, sym_cycle;
@@ -1265,9 +1265,14 @@ yielder_yield(VALUE obj, VALUE args)
/* :nodoc: */
static VALUE
-yielder_yield_push(VALUE obj, VALUE args)
+yielder_yield_push(VALUE obj, VALUE arg)
{
- yielder_yield(obj, args);
+ struct yielder *ptr = yielder_ptr(obj);
+
+ rb_proc_call_with_block(ptr->proc, 1, &arg, Qnil);
+
return obj;
}
@@ -1510,7 +1515,7 @@ lazy_init_yielder(VALUE val, VALUE m, int argc, VALUE *argv)
}
if (cont) {
- rb_funcall2(yielder, id_yield, 1, &(result->memo_value));
+ rb_funcall2(yielder, id_yield_push, 1, &(result->memo_value));
}
if (LAZY_MEMO_BREAK_P(result)) {
rb_iter_break();
@@ -2448,7 +2453,7 @@ InitVM_Enumerator(void)
rb_define_alloc_func(rb_cYielder, yielder_allocate);
rb_define_method(rb_cYielder, "initialize", yielder_initialize, 0);
rb_define_method(rb_cYielder, "yield", yielder_yield, -2);
- rb_define_method(rb_cYielder, "<<", yielder_yield_push, -2);
+ rb_define_method(rb_cYielder, "<<", yielder_yield_push, 1);
rb_provide("enumerator.so"); /* for backward compatibility */
}
@@ -2459,6 +2464,7 @@ Init_Enumerator(void)
{
id_rewind = rb_intern("rewind");
id_yield = rb_intern("yield");
+ id_yield_push = rb_intern("<<");
id_new = rb_intern("new");
id_next = rb_intern("next");
id_result = rb_intern("result");
~~~
which result in the following:
~~~
user system total real
Lazy: 0.108000 0.000000 0.108000 ( 0.108484)
Normal: 0.052000 0.008000 0.060000 ( 0.062528)
2.076923 0.000000 NaN ( 1.734961)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 2240 bytes (33 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
880 Proc
480 Array
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
12 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
This changes reduces the memory utilization and also by a tiny fraction improves performance for the lazy enumerator
---Files--------------------------------
lazy_test.rb (1.26 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [ruby-core:88254] [Ruby trunk Bug#14908] Enumerator::Lazy creates unnecessary Array objects.
[not found] <redmine.issue-14908.20180711132323@ruby-lang.org>
` (5 preceding siblings ...)
2018-07-13 4:59 ` [ruby-core:87936] " ruby-core
@ 2018-08-01 7:44 ` kelvinbhoss
2018-09-13 7:53 ` [ruby-core:88986] " knu
` (2 subsequent siblings)
9 siblings, 0 replies; 10+ messages in thread
From: kelvinbhoss @ 2018-08-01 7:44 UTC (permalink / raw
To: ruby-core
Issue #14908 has been updated by kelvinbhoss (Kelvin Hoss).
I must say that emulators are always good when it comes to get the same experience with limited sources. https://psncoders.com
----------------------------------------
Bug #14908: Enumerator::Lazy creates unnecessary Array objects.
https://bugs.ruby-lang.org/issues/14908#change-73276
* Author: chopraanmol1 (Anmol Chopra)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.6.0dev (2018-07-11 trunk 63949) [x86_64-linux]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Benchmark result on trunk:
~~~
user system total real
Lazy: 0.120000 0.000000 0.120000 ( 0.119958)
Normal: 0.056000 0.004000 0.060000 ( 0.062848)
2.142857 0.000000 NaN ( 1.908698)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 122240 bytes (3033 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
120480 Array
880 Proc
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
3012 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
As you may observe an extra array is created for every final element. Enumerator::Yielder#yield method has arity of -1 which wrap every elements in array. The same goes for Enumerator::Yielder#<< method, I'm proposing to change arity for Enumerator::Yielder#<< to 1 from -1. It will also make << method definition consistent with other classes(Array, String & etc).
I've applied the following set of changes to trunk(Pull Request for the same: https://github.com/ruby/ruby/pull/1912):
~~~
diff --git a/enumerator.c b/enumerator.c
index 050a9ce58f..c21d6f36f1 100644
--- a/enumerator.c
+++ b/enumerator.c
@@ -103,7 +103,7 @@
*/
VALUE rb_cEnumerator;
static VALUE rb_cLazy;
-static ID id_rewind, id_new, id_yield, id_to_enum;
+static ID id_rewind, id_new, id_yield, id_yield_push, id_to_enum;
static ID id_next, id_result, id_lazy, id_receiver, id_arguments, id_memo, id_method, id_force;
static VALUE sym_each, sym_cycle;
@@ -1265,9 +1265,14 @@ yielder_yield(VALUE obj, VALUE args)
/* :nodoc: */
static VALUE
-yielder_yield_push(VALUE obj, VALUE args)
+yielder_yield_push(VALUE obj, VALUE arg)
{
- yielder_yield(obj, args);
+ struct yielder *ptr = yielder_ptr(obj);
+
+ rb_proc_call_with_block(ptr->proc, 1, &arg, Qnil);
+
return obj;
}
@@ -1510,7 +1515,7 @@ lazy_init_yielder(VALUE val, VALUE m, int argc, VALUE *argv)
}
if (cont) {
- rb_funcall2(yielder, id_yield, 1, &(result->memo_value));
+ rb_funcall2(yielder, id_yield_push, 1, &(result->memo_value));
}
if (LAZY_MEMO_BREAK_P(result)) {
rb_iter_break();
@@ -2448,7 +2453,7 @@ InitVM_Enumerator(void)
rb_define_alloc_func(rb_cYielder, yielder_allocate);
rb_define_method(rb_cYielder, "initialize", yielder_initialize, 0);
rb_define_method(rb_cYielder, "yield", yielder_yield, -2);
- rb_define_method(rb_cYielder, "<<", yielder_yield_push, -2);
+ rb_define_method(rb_cYielder, "<<", yielder_yield_push, 1);
rb_provide("enumerator.so"); /* for backward compatibility */
}
@@ -2459,6 +2464,7 @@ Init_Enumerator(void)
{
id_rewind = rb_intern("rewind");
id_yield = rb_intern("yield");
+ id_yield_push = rb_intern("<<");
id_new = rb_intern("new");
id_next = rb_intern("next");
id_result = rb_intern("result");
~~~
which result in the following:
~~~
user system total real
Lazy: 0.108000 0.000000 0.108000 ( 0.108484)
Normal: 0.052000 0.008000 0.060000 ( 0.062528)
2.076923 0.000000 NaN ( 1.734961)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 2240 bytes (33 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
880 Proc
480 Array
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
12 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
This changes reduces the memory utilization and also by a tiny fraction improves performance for the lazy enumerator
---Files--------------------------------
lazy_test.rb (1.26 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [ruby-core:88986] [Ruby trunk Bug#14908] Enumerator::Lazy creates unnecessary Array objects.
[not found] <redmine.issue-14908.20180711132323@ruby-lang.org>
` (6 preceding siblings ...)
2018-08-01 7:44 ` [ruby-core:88254] " kelvinbhoss
@ 2018-09-13 7:53 ` knu
2018-09-18 5:48 ` [ruby-core:89054] " chopraanmol1
2018-09-18 6:43 ` [ruby-core:89055] " shyouhei
9 siblings, 0 replies; 10+ messages in thread
From: knu @ 2018-09-13 7:53 UTC (permalink / raw
To: ruby-core
Issue #14908 has been updated by knu (Akinori MUSHA).
I'm the original author of Enumerator & Yielder and I don't think I particularly intended to make `<<` accept many arguments. `<<` was an alias of `yield`, and I just didn't bother to make a separate function. So, I'd say go ahead.
----------------------------------------
Bug #14908: Enumerator::Lazy creates unnecessary Array objects.
https://bugs.ruby-lang.org/issues/14908#change-74015
* Author: chopraanmol1 (Anmol Chopra)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.6.0dev (2018-07-11 trunk 63949) [x86_64-linux]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Benchmark result on trunk:
~~~
user system total real
Lazy: 0.120000 0.000000 0.120000 ( 0.119958)
Normal: 0.056000 0.004000 0.060000 ( 0.062848)
2.142857 0.000000 NaN ( 1.908698)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 122240 bytes (3033 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
120480 Array
880 Proc
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
3012 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
As you may observe an extra array is created for every final element. Enumerator::Yielder#yield method has arity of -1 which wrap every elements in array. The same goes for Enumerator::Yielder#<< method, I'm proposing to change arity for Enumerator::Yielder#<< to 1 from -1 and use it internally for the lazy enum. It will also make << method definition consistent with other classes(Array, String & etc).
I've applied the following set of changes:
**https://github.com/ruby/ruby/pull/1912**
which result in the following:
~~~
user system total real
Lazy: 0.108000 0.000000 0.108000 ( 0.108484)
Normal: 0.052000 0.008000 0.060000 ( 0.062528)
2.076923 0.000000 NaN ( 1.734961)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 2240 bytes (33 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
880 Proc
480 Array
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
12 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
This changes reduces the memory utilization and also by a tiny fraction improves performance for the lazy enumerator
---Files--------------------------------
lazy_test.rb (1.26 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 10+ messages in thread
* [ruby-core:89054] [Ruby trunk Bug#14908] Enumerator::Lazy creates unnecessary Array objects.
[not found] <redmine.issue-14908.20180711132323@ruby-lang.org>
` (7 preceding siblings ...)
2018-09-13 7:53 ` [ruby-core:88986] " knu
@ 2018-09-18 5:48 ` chopraanmol1
2018-09-18 6:43 ` [ruby-core:89055] " shyouhei
9 siblings, 0 replies; 10+ messages in thread
From: chopraanmol1 @ 2018-09-18 5:48 UTC (permalink / raw
To: ruby-core
Issue #14908 has been updated by chopraanmol1 (Anmol Chopra).
All the response of the proposed patch is positive till date. @shyouhei can we go ahead with this now?
----------------------------------------
Bug #14908: Enumerator::Lazy creates unnecessary Array objects.
https://bugs.ruby-lang.org/issues/14908#change-74079
* Author: chopraanmol1 (Anmol Chopra)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.6.0dev (2018-07-11 trunk 63949) [x86_64-linux]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Benchmark result on trunk:
~~~
user system total real
Lazy: 0.120000 0.000000 0.120000 ( 0.119958)
Normal: 0.056000 0.004000 0.060000 ( 0.062848)
2.142857 0.000000 NaN ( 1.908698)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 122240 bytes (3033 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
120480 Array
880 Proc
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
3012 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
As you may observe an extra array is created for every final element. Enumerator::Yielder#yield method has arity of -1 which wrap every elements in array. The same goes for Enumerator::Yielder#<< method, I'm proposing to change arity for Enumerator::Yielder#<< to 1 from -1 and use it internally for the lazy enum. It will also make << method definition consistent with other classes(Array, String & etc).
I've applied the following set of changes:
**https://github.com/ruby/ruby/pull/1912**
which result in the following:
~~~
user system total real
Lazy: 0.108000 0.000000 0.108000 ( 0.108484)
Normal: 0.052000 0.008000 0.060000 ( 0.062528)
2.076923 0.000000 NaN ( 1.734961)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 2240 bytes (33 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
880 Proc
480 Array
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
12 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
This changes reduces the memory utilization and also by a tiny fraction improves performance for the lazy enumerator
---Files--------------------------------
lazy_test.rb (1.26 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 10+ messages in thread
* [ruby-core:89055] [Ruby trunk Bug#14908] Enumerator::Lazy creates unnecessary Array objects.
[not found] <redmine.issue-14908.20180711132323@ruby-lang.org>
` (8 preceding siblings ...)
2018-09-18 5:48 ` [ruby-core:89054] " chopraanmol1
@ 2018-09-18 6:43 ` shyouhei
9 siblings, 0 replies; 10+ messages in thread
From: shyouhei @ 2018-09-18 6:43 UTC (permalink / raw
To: ruby-core
Issue #14908 has been updated by shyouhei (Shyouhei Urabe).
No objection. I guess @nobu is trying to fix it.
----------------------------------------
Bug #14908: Enumerator::Lazy creates unnecessary Array objects.
https://bugs.ruby-lang.org/issues/14908#change-74080
* Author: chopraanmol1 (Anmol Chopra)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.6.0dev (2018-07-11 trunk 63949) [x86_64-linux]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Benchmark result on trunk:
~~~
user system total real
Lazy: 0.120000 0.000000 0.120000 ( 0.119958)
Normal: 0.056000 0.004000 0.060000 ( 0.062848)
2.142857 0.000000 NaN ( 1.908698)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 122240 bytes (3033 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
120480 Array
880 Proc
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
3012 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
As you may observe an extra array is created for every final element. Enumerator::Yielder#yield method has arity of -1 which wrap every elements in array. The same goes for Enumerator::Yielder#<< method, I'm proposing to change arity for Enumerator::Yielder#<< to 1 from -1 and use it internally for the lazy enum. It will also make << method definition consistent with other classes(Array, String & etc).
I've applied the following set of changes:
**https://github.com/ruby/ruby/pull/1912**
which result in the following:
~~~
user system total real
Lazy: 0.108000 0.000000 0.108000 ( 0.108484)
Normal: 0.052000 0.008000 0.060000 ( 0.062528)
2.076923 0.000000 NaN ( 1.734961)
++++++++++++++++++++++++++++++++++++++++++++++++++
Lazy:
Total allocated: 2240 bytes (33 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
880 Proc
480 Array
384 Enumerator::Lazy
264 Object
168 Enumerator::Generator
64 Enumerator::Yielder
allocated objects by class
--------------------------------------------------
12 Array
11 Proc
3 Enumerator::Generator
3 Enumerator::Lazy
3 Object
1 Enumerator::Yielder
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
Normal:
Total allocated: 72120 bytes (3 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
--------------------------------------------------
72120 Array
allocated objects by class
--------------------------------------------------
3 Array
++++++++++++++++++++++++++++++++++++++++++++++++++
~~~
This changes reduces the memory utilization and also by a tiny fraction improves performance for the lazy enumerator
---Files--------------------------------
lazy_test.rb (1.26 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2018-09-18 6:43 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <redmine.issue-14908.20180711132323@ruby-lang.org>
2018-07-11 13:23 ` [ruby-core:87907] [Ruby trunk Bug#14908] Enumerator::Lazy creates unnecessary Array objects chopraanmol1
2018-07-12 0:12 ` [ruby-core:87914] " shyouhei
2018-07-12 5:03 ` [ruby-core:87918] " chopraanmol1
2018-07-12 9:25 ` [ruby-core:87924] " eregontp
2018-07-13 4:46 ` [ruby-core:87933] " chopraanmol1
2018-07-13 4:59 ` [ruby-core:87936] " ruby-core
2018-08-01 7:44 ` [ruby-core:88254] " kelvinbhoss
2018-09-13 7:53 ` [ruby-core:88986] " knu
2018-09-18 5:48 ` [ruby-core:89054] " chopraanmol1
2018-09-18 6:43 ` [ruby-core:89055] " shyouhei
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).