[ruby-core:99474] [Ruby master Feature#17103] Add a :since option to ObjectSpace.dump_all

[ruby-core:99474] [Ruby master Feature#17103] Add a :since option to ObjectSpace.dump_all

[jean.boussier]

Issue #17103 has been reported by byroot (Jean Boussier).

----------------------------------------
Feature #17103: Add a :since option to ObjectSpace.dump_all
https://bugs.ruby-lang.org/issues/17103

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
----------------------------------------
Patch: https://github.com/ruby/ruby/pull/3368

This is useful to see what a block of code allocated, e.g.

```ruby
GC.start
GC.disable
gc_gen = GC.count
ObjectSpace.trace_object_allocations do
  # run some code
end
allocations = ObjectSpace.dump_all(output: :file, since: gc_gen)
GC.enable
GC.start
retentions = ObjectSpace.dump_all(output: :file, since: gc_gen)
```

For context this is what I do in https://github.com/Shopify/heap-profiler, except I have to dump the entire three times, and then do a diff. This new API would allow me to dump the entire heap only once, and then do two much faster single generations dumps, and avoid having to do the diff.




-- 
https://bugs.ruby-lang.org/

[ruby-core:99569] [Ruby master Feature#17103] Add a :since option to ObjectSpace.dump_all

[ko1]

Issue #17103 has been updated by ko1 (Koichi Sasada).


sorry I can't understand the specification.

also https://github.com/ruby/ruby/pull/3368/files doesn't have a doc.
could you explain more?


----------------------------------------
Feature #17103: Add a :since option to ObjectSpace.dump_all
https://bugs.ruby-lang.org/issues/17103#change-87035

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
----------------------------------------
Patch: https://github.com/ruby/ruby/pull/3368

This is useful for seeing what a block of code allocated, e.g.

```ruby
GC.start
GC.disable
gc_gen = GC.count
ObjectSpace.trace_object_allocations do
  # run some code
end
allocations = ObjectSpace.dump_all(output: :file, since: gc_gen)
GC.enable
GC.start
retentions = ObjectSpace.dump_all(output: :file, since: gc_gen)
```

For context, this is what I do in https://github.com/Shopify/heap-profiler, except that I have to dump the entire heap three times and then do a diff. This new API would allow me to dump the entire heap only once and then do two much-faster single-generation dumps without doing the diff.



-- 
https://bugs.ruby-lang.org/

[ruby-core:99570] [Ruby master Feature#17103] Add a :since option to ObjectSpace.dump_all

[jean.boussier]

Issue #17103 has been updated by byroot (Jean Boussier).


@ko1 this parameter allow to do partial heap dumps. e.g. `dump_all(since: 42)`, means to only dump objects of the 42th generation and later. Basically like adding a `if ObjectSpace.allocation_generation(obj) >= 42`.

This would be very useful to me, because a minimal heap dump of our application is over 1GiB and take almost a minute to be performed. Often times I'm only interested by allocations past a certain point, such filtering criterias would allow for much smaller and faster dumps, with less "noise" in them.

Please let me know if that is still unclear.

> also https://github.com/ruby/ruby/pull/3368/files doesn't have a doc.

That's an oversight on my part, I'll update the PR to add a documentation for that new parameter as soon as possible.

----------------------------------------
Feature #17103: Add a :since option to ObjectSpace.dump_all
https://bugs.ruby-lang.org/issues/17103#change-87037

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
----------------------------------------
Patch: https://github.com/ruby/ruby/pull/3368

This is useful for seeing what a block of code allocated, e.g.

```ruby
GC.start
GC.disable
gc_gen = GC.count
ObjectSpace.trace_object_allocations do
  # run some code
end
allocations = ObjectSpace.dump_all(output: :file, since: gc_gen)
GC.enable
GC.start
retentions = ObjectSpace.dump_all(output: :file, since: gc_gen)
```

For context, this is what I do in https://github.com/Shopify/heap-profiler, except that I have to dump the entire heap three times and then do a diff. This new API would allow me to dump the entire heap only once and then do two much-faster single-generation dumps without doing the diff.



-- 
https://bugs.ruby-lang.org/

[ruby-core:99678] [Ruby master Feature#17103] Add a :since option to ObjectSpace.dump_all

[ko1]

Issue #17103 has been updated by ko1 (Koichi Sasada).


this feature strongly connected with `trace_object_allocations`, right? so please write it on document.
what happens on without `trace_object_allocations`? it is unclear.

----------------------------------------
Feature #17103: Add a :since option to ObjectSpace.dump_all
https://bugs.ruby-lang.org/issues/17103#change-87165

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
----------------------------------------
Patch: https://github.com/ruby/ruby/pull/3368

This is useful for seeing what a block of code allocated, e.g.

```ruby
GC.start
GC.disable
gc_gen = GC.count
ObjectSpace.trace_object_allocations do
  # run some code
end
allocations = ObjectSpace.dump_all(output: :file, since: gc_gen)
GC.enable
GC.start
retentions = ObjectSpace.dump_all(output: :file, since: gc_gen)
```

For context, this is what I do in https://github.com/Shopify/heap-profiler, except that I have to dump the entire heap three times and then do a diff. This new API would allow me to dump the entire heap only once and then do two much-faster single-generation dumps without doing the diff.



-- 
https://bugs.ruby-lang.org/

[ruby-core:99679] [Ruby master Feature#17103] Add a :since option to ObjectSpace.dump_all

[jean.boussier]

Issue #17103 has been updated by byroot (Jean Boussier).


> this feature strongly connected with trace_object_allocations, right?

Indeed.

> so please write it on document. what happens on without trace_object_allocations? it is unclear.

I changed:

```
Objects that were allocated with object allocation tracing disabled are not dumped.
```

to:

```
Objects that were allocated without object allocation tracing enabled
are ignored. See ::trace_object_allocations for more information and
examples.
```

Is that acceptable?

----------------------------------------
Feature #17103: Add a :since option to ObjectSpace.dump_all
https://bugs.ruby-lang.org/issues/17103#change-87166

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
----------------------------------------
Patch: https://github.com/ruby/ruby/pull/3368

This is useful for seeing what a block of code allocated, e.g.

```ruby
GC.start
GC.disable
gc_gen = GC.count
ObjectSpace.trace_object_allocations do
  # run some code
end
allocations = ObjectSpace.dump_all(output: :file, since: gc_gen)
GC.enable
GC.start
retentions = ObjectSpace.dump_all(output: :file, since: gc_gen)
```

For context, this is what I do in https://github.com/Shopify/heap-profiler, except that I have to dump the entire heap three times and then do a diff. This new API would allow me to dump the entire heap only once and then do two much-faster single-generation dumps without doing the diff.



-- 
https://bugs.ruby-lang.org/

[ruby-core:99718] [Ruby master Feature#17103] Add a :since option to ObjectSpace.dump_all

[ko1]

Issue #17103 has been updated by ko1 (Koichi Sasada).


I missed "allocation tracing disabled are not dumped" sentence. And changed version is more helpful.

Now I don't have any objection.
One suggestion is, "since" integer is not clear yet (even if `trace_object_allocations` describes it), so it is helpful to explain it is integer from `GC.count`.

----------------------------------------
Feature #17103: Add a :since option to ObjectSpace.dump_all
https://bugs.ruby-lang.org/issues/17103#change-87208

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
----------------------------------------
Patch: https://github.com/ruby/ruby/pull/3368

This is useful for seeing what a block of code allocated, e.g.

```ruby
GC.start
GC.disable
gc_gen = GC.count
ObjectSpace.trace_object_allocations do
  # run some code
end
allocations = ObjectSpace.dump_all(output: :file, since: gc_gen)
GC.enable
GC.start
retentions = ObjectSpace.dump_all(output: :file, since: gc_gen)
```

For context, this is what I do in https://github.com/Shopify/heap-profiler, except that I have to dump the entire heap three times and then do a diff. This new API would allow me to dump the entire heap only once and then do two much-faster single-generation dumps without doing the diff.



-- 
https://bugs.ruby-lang.org/

[ruby-core:99726] [Ruby master Feature#17103] Add a :since option to ObjectSpace.dump_all

[jean.boussier]

Issue #17103 has been updated by byroot (Jean Boussier).


That's a good point. I added:

> The current generation can be accessed using GC::count.

----------------------------------------
Feature #17103: Add a :since option to ObjectSpace.dump_all
https://bugs.ruby-lang.org/issues/17103#change-87219

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
----------------------------------------
Patch: https://github.com/ruby/ruby/pull/3368

This is useful for seeing what a block of code allocated, e.g.

```ruby
GC.start
GC.disable
gc_gen = GC.count
ObjectSpace.trace_object_allocations do
  # run some code
end
allocations = ObjectSpace.dump_all(output: :file, since: gc_gen)
GC.enable
GC.start
retentions = ObjectSpace.dump_all(output: :file, since: gc_gen)
```

For context, this is what I do in https://github.com/Shopify/heap-profiler, except that I have to dump the entire heap three times and then do a diff. This new API would allow me to dump the entire heap only once and then do two much-faster single-generation dumps without doing the diff.



-- 
https://bugs.ruby-lang.org/

[ruby-core:99792] [Ruby master Feature#17103] Add a :since option to ObjectSpace.dump_all

[matz]

Issue #17103 has been updated by matz (Yukihiro Matsumoto).


OK. Accepted.

Matz.


----------------------------------------
Feature #17103: Add a :since option to ObjectSpace.dump_all
https://bugs.ruby-lang.org/issues/17103#change-87309

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
----------------------------------------
Patch: https://github.com/ruby/ruby/pull/3368

This is useful for seeing what a block of code allocated, e.g.

```ruby
GC.start
GC.disable
gc_gen = GC.count
ObjectSpace.trace_object_allocations do
  # run some code
end
allocations = ObjectSpace.dump_all(output: :file, since: gc_gen)
GC.enable
GC.start
retentions = ObjectSpace.dump_all(output: :file, since: gc_gen)
```

For context, this is what I do in https://github.com/Shopify/heap-profiler, except that I have to dump the entire heap three times and then do a diff. This new API would allow me to dump the entire heap only once and then do two much-faster single-generation dumps without doing the diff.



-- 
https://bugs.ruby-lang.org/

[ruby-core:102138] [Ruby master Feature#17103] Add a :since option to ObjectSpace.dump_all

[jean.boussier]

Issue #17103 has been updated by byroot (Jean Boussier).


This was merged in Ruby 3.0 and can now be closed.

----------------------------------------
Feature #17103: Add a :since option to ObjectSpace.dump_all
https://bugs.ruby-lang.org/issues/17103#change-89989

* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
----------------------------------------
Patch: https://github.com/ruby/ruby/pull/3368

This is useful for seeing what a block of code allocated, e.g.

```ruby
GC.start
GC.disable
gc_gen = GC.count
ObjectSpace.trace_object_allocations do
  # run some code
end
allocations = ObjectSpace.dump_all(output: :file, since: gc_gen)
GC.enable
GC.start
retentions = ObjectSpace.dump_all(output: :file, since: gc_gen)
```

For context, this is what I do in https://github.com/Shopify/heap-profiler, except that I have to dump the entire heap three times and then do a diff. This new API would allow me to dump the entire heap only once and then do two much-faster single-generation dumps without doing the diff.



-- 
https://bugs.ruby-lang.org/