From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-2.4 required=3.0 tests=AWL,BAYES_00, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 2C4D21F453 for ; Mon, 21 Jan 2019 08:01:22 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 7C1E41210A5; Mon, 21 Jan 2019 17:01:18 +0900 (JST) Received: from o1678916x28.outbound-mail.sendgrid.net (o1678916x28.outbound-mail.sendgrid.net [167.89.16.28]) by neon.ruby-lang.org (Postfix) with ESMTPS id 3B39B120F14 for ; Mon, 21 Jan 2019 17:01:12 +0900 (JST) Received: by filter0035p3iad2.sendgrid.net with SMTP id filter0035p3iad2-15445-5C457C46-41 2019-01-21 08:01:10.765611453 +0000 UTC m=+556862.793143988 Received: from herokuapp.com (ec2-54-205-253-189.compute-1.amazonaws.com [54.205.253.189]) by ismtpd0050p1iad1.sendgrid.net (SG) with ESMTP id Cf0tTQgiRSSuBi7Ch6C4KA for ; Mon, 21 Jan 2019 08:01:10.348 +0000 (UTC) Date: Mon, 21 Jan 2019 08:01:11 +0000 (UTC) From: mame@ruby-lang.org To: ruby-core@ruby-lang.org Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 66651 X-Redmine-Project: ruby-trunk X-Redmine-Issue-Id: 9508 X-Redmine-Issue-Author: srawlins X-Redmine-Issue-Assignee: mame X-Redmine-Sender: mame X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: ync6xU2WACa70kv/Ymy4QrNMhiuLXJG8OTL2vJD1yS4zjgNaOCnnVmdjXhRaxnMZTrfs5BU/LqC3Wu 9btZ6LulEorjGnnnfUyjVpk+IFXMow4E4ZlV7qGjQK3gE3iNgpiETre4c2KeGNZVaQlNJVWV5Alisv X9kxRXSWfQrVbRZY1IsTzRH+YgNWOzz5ZUvgOXcQAuXiza3CyqhFLOsouA== X-ML-Name: ruby-core X-Mail-Count: 91203 Subject: [ruby-core:91203] [Ruby trunk Feature#9508] Add method coverage and branch coverage metrics X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #9508 has been updated by mame (Yusuke Endoh). Status changed from Assigned to Closed Branch coverage has been finally implemented in #13901. Thanks! ---------------------------------------- Feature #9508: Add method coverage and branch coverage metrics https://bugs.ruby-lang.org/issues/9508#change-76440 * Author: srawlins (Sam Rawlins) * Status: Closed * Priority: Normal * Assignee: mame (Yusuke Endoh) * Target version: ---------------------------------------- Since the Coverage extension was introduced in Ruby 1.9, Ruby has had built-in line code coverage. Ruby should support more of the basic code coverage metrics [1]. I have a pull request on GitHub ( https://github.com/ruby/ruby/pull/511 ) to add Method Coverage (Function Coverage) and Branch Coverage. I'd love feedback to improve it. Currently, with this feature, Coverage.result would look like: {"/Users/sam/code/ruby/cov_method.rb" => { lines: [1, 2, 2, 20, nil, nil, 2, 2, 2, nil, 0, nil, nil, nil, 1, 0, nil, nil, 1, 1, nil, nil, 1], methods: {1=>2, 15=>0, 19=>1}, branches: {8=>2, 11=>0} }} which includes * the current Ruby line coverage report, * as well as a method report (The method defined on line 1 was called 2 times; the method on line 15 was called 0 times; ...), * and a branch report (the branch on line 8 was called 2 times; the branch on line 11 was called 0 times). Branches -------- Branches include the bodies of if, elsif, else, unless, and when statements, which are all tracked with this new feature. However, this feature is not aware of void bodies, for example: if foo :ok end will report that only one branch exists in the file. It would be better to declare that there is a branch body on line 2, and a void branch body on line 3, or perhaps line 1. This would require the keys of the [:branch] Hash to be something other than line numbers. Perhaps label_no? Perhaps nd_type(node) paired with line or label_no? More Coverage ------------- I think that Statement Coverage, and Condition Coverage could be added to this feature, using the same techniques. Caveats ------- I was not very clear on the bit-arrays used in ruby.h, and just used values for the new macros that seemed to work. Also, I would much rather use Ranges to identify a branch, so that a Coverage analyzer like SimpleCov won't need any kind of Ruby parser to identify and highlight a full chunk of code as a tested branch, or a not tested branch. I'm trying to find how that could be implemented... [1] Wikipedia has good definitions: http://en.wikipedia.org/wiki/Code_coverage ---Files-------------------------------- pull-request-511.patch (26.7 KB) pull-request-511.patch (38.5 KB) pull-request-511.patch (57 KB) -- https://bugs.ruby-lang.org/