[ruby-core:75948] [Ruby trunk Bug#12480] Restarting Coverage does not capture additional coverage for already loaded files

[ruby-core:75948] [Ruby trunk Bug#12480] Restarting Coverage does not capture additional coverage for already loaded files

[michael]

Issue #12480 has been reported by Michael Grosser.

----------------------------------------
Bug #12480: Restarting Coverage does not capture additional coverage for already loaded files
https://bugs.ruby-lang.org/issues/12480

* Author: Michael Grosser
* Status: Open
* Priority: Normal
* Assignee: 
* ruby -v: 2.3.1
* Backport: 2.1: UNKNOWN, 2.2: UNKNOWN, 2.3: UNKNOWN
----------------------------------------
I'm trying to combine coverage from before fork and after fork to make coverage reporting work in a forking test runner.
The problem I ran into is 2-fold:
 - A: when forking, previous coverage is lost
 - B: when restarting coverage, old files do not get added to

I could work around issue A by storing the old result and then merging it with the new result post fork, but issue B makes that impossible.

Please fix either A or B ... 

Reproduction steps for A:

~~~
# reproduce.rb
require 'coverage'

Coverage.start
require_relative 'test'
a

fork do
  b
  new = Coverage.result
  puts "NEW: #{new}"
end


# test.rb
def a
  1
end

def b
  1
end
~~~

NEW: {"/Users/mgrosser/Code/tools/forking_test_runner/test.rb"=>[0, 0, nil, nil, 0, 1, nil]}
-> missing coverage information for method `a`

Reproduction steps for B:
~~~
# reproduce.rb
require 'coverage'

Coverage.start
require_relative 'test'
a
old = Coverage.result
Coverage.start
b
new = Coverage.result
puts "OLD: #{old} -- NEW: #{new}"

# test.rb
def a
  1
end

def b
  1
end
~~~

OLD: {"test.rb"=>[1, 1, nil, nil, 1, 0, nil]} -- NEW: {"test.rb"=>[]}
-> missing coverage information for method `b`



-- 
https://bugs.ruby-lang.org/

[ruby-core:75950] [Ruby trunk Bug#12480][Feedback] Restarting Coverage does not capture additional coverage for already loaded files

[mame]

Issue #12480 has been updated by Yusuke Endoh.

Status changed from Open to Feedback
Assignee set to Yusuke Endoh

Michael Grosser wrote:
>  - A: when forking, previous coverage is lost

This is intended.  Consider the following program:

~~~
# reproduce.rb
require "coverage"

Coverage.start
require_relative "test"

def save_coverage
  r = Coverage.result

  # merge coverage data
  if File.readable?("coverage.dat")
    r2 = Marshal.load(File.binread("coverage.dat"))
    r = merge(r, r2)
  end

  File.binwrite("coverage.dat", Marshal.dump(r))
end

simple_test

fork do
  heavy_test_1
  save_coverage
end

heavy_test_2
save_coverage
~~~

Note that the `simple_test` method is executed once.  If `fork` does not clear the previous coverage, the method will be wrongly counted twice.


>  - B: when restarting coverage, old files do not get added to

This is also intended.  See #12220.



I recommend you not to use `fork`.  If you want to do so at any cost, you can pass the coverage data via pipe and do manual merge:

~~~
require "coverage"
Coverage.start

require_relative "test"

a

r, w = IO.pipe("BINARY")
fork do
  w.close
  c1 = Marshal.load(r.read)

  b

  c2 = Coverage.result
  puts "NEW: #{ merge(c1, c2) }"
end

r.close
w << Marshal.dump(Coverage.result)
w.close
~~~

----------------------------------------
Bug #12480: Restarting Coverage does not capture additional coverage for already loaded files
https://bugs.ruby-lang.org/issues/12480#change-59144

* Author: Michael Grosser
* Status: Feedback
* Priority: Normal
* Assignee: Yusuke Endoh
* ruby -v: 2.3.1
* Backport: 2.1: UNKNOWN, 2.2: UNKNOWN, 2.3: UNKNOWN
----------------------------------------
I'm trying to combine coverage from before fork and after fork to make coverage reporting work in a forking test runner.
The problem I ran into is 2-fold:
 - A: when forking, previous coverage is lost
 - B: when restarting coverage, old files do not get added to

I could work around issue A by storing the old result and then merging it with the new result post fork, but issue B makes that impossible.

Please fix either A or B ... 

Reproduction steps for A:

~~~
# reproduce.rb
require 'coverage'

Coverage.start
require_relative 'test'
a

fork do
  b
  new = Coverage.result
  puts "NEW: #{new}"
end


# test.rb
def a
  1
end

def b
  1
end
~~~

NEW: {"/Users/mgrosser/Code/tools/forking_test_runner/test.rb"=>[0, 0, nil, nil, 0, 1, nil]}
-> missing coverage information for method `a`

Reproduction steps for B:
~~~
# reproduce.rb
require 'coverage'

Coverage.start
require_relative 'test'
a
old = Coverage.result
Coverage.start
b
new = Coverage.result
puts "OLD: #{old} -- NEW: #{new}"

# test.rb
def a
  1
end

def b
  1
end
~~~

OLD: {"test.rb"=>[1, 1, nil, nil, 1, 0, nil]} -- NEW: {"test.rb"=>[]}
-> missing coverage information for method `b`



-- 
https://bugs.ruby-lang.org/

[ruby-core:76122] [Ruby trunk Bug#12480] Restarting Coverage does not capture additional coverage for already loaded files

[michael]

Issue #12480 has been updated by Michael Grosser.


Solved this by using Coverage.peek_result in the before-fork part of my code to capture the current state and then merged it after the fork is done :D 

Feel free to close this now!

----------------------------------------
Bug #12480: Restarting Coverage does not capture additional coverage for already loaded files
https://bugs.ruby-lang.org/issues/12480#change-59326

* Author: Michael Grosser
* Status: Feedback
* Priority: Normal
* Assignee: Yusuke Endoh
* ruby -v: 2.3.1
* Backport: 2.1: UNKNOWN, 2.2: UNKNOWN, 2.3: UNKNOWN
----------------------------------------
I'm trying to combine coverage from before fork and after fork to make coverage reporting work in a forking test runner.
The problem I ran into is 2-fold:
 - A: when forking, previous coverage is lost
 - B: when restarting coverage, old files do not get added to

I could work around issue A by storing the old result and then merging it with the new result post fork, but issue B makes that impossible.

Please fix either A or B ... 

Reproduction steps for A:

~~~
# reproduce.rb
require 'coverage'

Coverage.start
require_relative 'test'
a

fork do
  b
  new = Coverage.result
  puts "NEW: #{new}"
end


# test.rb
def a
  1
end

def b
  1
end
~~~

NEW: {"/Users/mgrosser/Code/tools/forking_test_runner/test.rb"=>[0, 0, nil, nil, 0, 1, nil]}
-> missing coverage information for method `a`

Reproduction steps for B:
~~~
# reproduce.rb
require 'coverage'

Coverage.start
require_relative 'test'
a
old = Coverage.result
Coverage.start
b
new = Coverage.result
puts "OLD: #{old} -- NEW: #{new}"

# test.rb
def a
  1
end

def b
  1
end
~~~

OLD: {"test.rb"=>[1, 1, nil, nil, 1, 0, nil]} -- NEW: {"test.rb"=>[]}
-> missing coverage information for method `b`



-- 
https://bugs.ruby-lang.org/

[ruby-core:91205] [Ruby trunk Bug#12480] Restarting Coverage does not capture additional coverage for already loaded files

[mame]

Issue #12480 has been updated by mame (Yusuke Endoh).

Status changed from Feedback to Closed

Closing due to OP's request.

FYI: Since 2.6, coverage.so has supported `Coverage.result(stop: false, clear: false)`.  You may want to check it out.

----------------------------------------
Bug #12480: Restarting Coverage does not capture additional coverage for already loaded files
https://bugs.ruby-lang.org/issues/12480#change-76442

* Author: grosser (Michael Grosser)
* Status: Closed
* Priority: Normal
* Assignee: mame (Yusuke Endoh)
* Target version: 
* ruby -v: 2.3.1
* Backport: 2.1: UNKNOWN, 2.2: UNKNOWN, 2.3: UNKNOWN
----------------------------------------
I'm trying to combine coverage from before fork and after fork to make coverage reporting work in a forking test runner.
The problem I ran into is 2-fold:
 - A: when forking, previous coverage is lost
 - B: when restarting coverage, old files do not get added to

I could work around issue A by storing the old result and then merging it with the new result post fork, but issue B makes that impossible.

Please fix either A or B ... 

Reproduction steps for A:

~~~
# reproduce.rb
require 'coverage'

Coverage.start
require_relative 'test'
a

fork do
  b
  new = Coverage.result
  puts "NEW: #{new}"
end


# test.rb
def a
  1
end

def b
  1
end
~~~

NEW: {"/Users/mgrosser/Code/tools/forking_test_runner/test.rb"=>[0, 0, nil, nil, 0, 1, nil]}
-> missing coverage information for method `a`

Reproduction steps for B:
~~~
# reproduce.rb
require 'coverage'

Coverage.start
require_relative 'test'
a
old = Coverage.result
Coverage.start
b
new = Coverage.result
puts "OLD: #{old} -- NEW: #{new}"

# test.rb
def a
  1
end

def b
  1
end
~~~

OLD: {"test.rb"=>[1, 1, nil, nil, 1, 0, nil]} -- NEW: {"test.rb"=>[]}
-> missing coverage information for method `b`



-- 
https://bugs.ruby-lang.org/