From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: poffice@blade.nagaokaut.ac.jp Delivered-To: poffice@blade.nagaokaut.ac.jp Received: from kankan.nagaokaut.ac.jp (kankan.nagaokaut.ac.jp [133.44.2.24]) by blade.nagaokaut.ac.jp (Postfix) with ESMTP id A0BC117D6D55 for ; Sat, 10 May 2014 09:32:10 +0900 (JST) Received: from funfun.nagaokaut.ac.jp (smtp.nagaokaut.ac.jp [133.44.2.201]) by kankan.nagaokaut.ac.jp (Postfix) with ESMTP id 66988B5D86B for ; Sat, 10 May 2014 09:41:46 +0900 (JST) Received: from funfun.nagaokaut.ac.jp (localhost.nagaokaut.ac.jp [127.0.0.1]) by funfun.nagaokaut.ac.jp (Postfix) with ESMTP id 74D2D97A826 for ; Sat, 10 May 2014 09:41:47 +0900 (JST) X-Virus-Scanned: amavisd-new at nagaokaut.ac.jp Authentication-Results: funfun.nagaokaut.ac.jp (amavisd-new); dkim=fail (1024-bit key) reason="fail (message has been altered)" header.d=sendgrid.me Received: from funfun.nagaokaut.ac.jp ([127.0.0.1]) by funfun.nagaokaut.ac.jp (funfun.nagaokaut.ac.jp [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fMlf4mwlFE_l for ; Sat, 10 May 2014 09:41:47 +0900 (JST) Received: from voscc.nagaokaut.ac.jp (voscc.nagaokaut.ac.jp [133.44.1.100]) by funfun.nagaokaut.ac.jp (Postfix) with ESMTP id 2D74197A820 for ; Sat, 10 May 2014 09:41:47 +0900 (JST) Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by voscc.nagaokaut.ac.jp (Postfix) with ESMTP id 9DF3F95241A for ; Sat, 10 May 2014 09:41:45 +0900 (JST) Received: from [221.186.184.76] (unknown [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 6674A12033D; Sat, 10 May 2014 09:41:37 +0900 (JST) X-Original-To: ruby-core@ruby-lang.org Delivered-To: ruby-core@ruby-lang.org Received: from o10.shared.sendgrid.net (o10.shared.sendgrid.net [173.193.132.135]) by neon.ruby-lang.org (Postfix) with SMTP id 9C00E120049 for ; Sat, 10 May 2014 09:41:33 +0900 (JST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.me; h=from:to:references:subject:mime-version:content-type:content-transfer-encoding:list-id; s=smtpapi; bh=7WArwjFyRVUnp5bBzw70CG4AFJw=; b=UdMCLbRDuo13mEX/U7 P5TDUBMHnwuwLUATXoNMNZ+Ox25Ca4lQQuWpqoWGb9W/OH2GHOqk/qIE95z7Kqg7 ne8OfwYyqle0MI7CRolg2pNiL3hiKp4eUKO33chEZS0Zsx2iQV5J7u8DACp1oudj mIwiC2H00H07Xxn1paNorzH2M= Received: by mf132.sendgrid.net with SMTP id mf132.5803.536D75BC3 Sat, 10 May 2014 00:41:32 +0000 (UTC) Received: from herokuapp.com (ec2-23-20-252-47.compute-1.amazonaws.com [23.20.252.47]) by ismtpd-027 (SG) with ESMTP id 145e393e745.7cd7.2c085 Sat, 10 May 2014 00:41:32 +0000 (GMT) Date: Sat, 10 May 2014 00:41:31 +0000 From: mame@tsg.ne.jp To: ruby-core@ruby-lang.org Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Redmine-MailingListIntegration-Message-Ids: 36439 X-Redmine-Project: ruby-trunk X-Redmine-Issue-Id: 9508 X-Redmine-Issue-Author: srawlins X-Redmine-Issue-Assignee: mame X-Redmine-Sender: mame X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: OOF Auto-Submitted: auto-generated X-SG-EID: ync6xU2WACa70kv/Ymy4QrNMhiuLXJG8OTL2vJD1yS5qiXigqwmJffYonr49w0vqNLC45BB8DFYth4o1WFj43e3ZdNoBcJpD2B8BxqnFnil7xmF2H3TTk3HQ4cCs5GVzG95aoktlpeVpKEVa1uQSy3yrqqNoDmVvYYIO+0ADkqY= X-ML-Name: ruby-core X-Mail-Count: 62489 Subject: [ruby-core:62489] [ruby-trunk - Feature #9508] [Assigned] Add method coverage and branch coverage metrics X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #9508 has been updated by Yusuke Endoh. Category set to core Status changed from Feedback to Assigned Assignee set to Yusuke Endoh Target version set to current: 2.2.0 Sam Rawlins wrote: > I was largely inspired by the visualization and analysis of Istanbul [1], the standard Javascript coverage library. Here is a great example of a coverage report: http://gotwarlost.github.io/istanbul/public/coverage/lcov-report/istanbul/lib/report/html.js.html Thank you for the explanation, I understood well what you want to do. A minor question: Istanbul ignores ternary operators, but the proposed API can not distinguish if/else branches and ternary operators. Is that okay? I think that the last major problem for this proposal is that ko1 is not very keen to add a new type of events. I'll talk with him. Thank you, -- Yusuke Endoh ---------------------------------------- Feature #9508: Add method coverage and branch coverage metrics https://bugs.ruby-lang.org/issues/9508#change-46653 * Author: Sam Rawlins * Status: Assigned * Priority: Normal * Assignee: Yusuke Endoh * Category: core * Target version: current: 2.2.0 ---------------------------------------- Since the Coverage extension was introduced in Ruby 1.9, Ruby has had built-in line code coverage. Ruby should support more of the basic code coverage metrics [1]. I have a pull request on GitHub ( https://github.com/ruby/ruby/pull/511 ) to add Method Coverage (Function Coverage) and Branch Coverage. I'd love feedback to improve it. Currently, with this feature, Coverage.result would look like: {"/Users/sam/code/ruby/cov_method.rb" => { lines: [1, 2, 2, 20, nil, nil, 2, 2, 2, nil, 0, nil, nil, nil, 1, 0, nil, nil, 1, 1, nil, nil, 1], methods: {1=>2, 15=>0, 19=>1}, branches: {8=>2, 11=>0} }} which includes * the current Ruby line coverage report, * as well as a method report (The method defined on line 1 was called 2 times; the method on line 15 was called 0 times; ...), * and a branch report (the branch on line 8 was called 2 times; the branch on line 11 was called 0 times). Branches -------- Branches include the bodies of if, elsif, else, unless, and when statements, which are all tracked with this new feature. However, this feature is not aware of void bodies, for example: if foo :ok end will report that only one branch exists in the file. It would be better to declare that there is a branch body on line 2, and a void branch body on line 3, or perhaps line 1. This would require the keys of the [:branch] Hash to be something other than line numbers. Perhaps label_no? Perhaps nd_type(node) paired with line or label_no? More Coverage ------------- I think that Statement Coverage, and Condition Coverage could be added to this feature, using the same techniques. Caveats ------- I was not very clear on the bit-arrays used in ruby.h, and just used values for the new macros that seemed to work. Also, I would much rather use Ranges to identify a branch, so that a Coverage analyzer like SimpleCov won't need any kind of Ruby parser to identify and highlight a full chunk of code as a tested branch, or a not tested branch. I'm trying to find how that could be implemented... [1] Wikipedia has good definitions: http://en.wikipedia.org/wiki/Code_coverage ---Files-------------------------------- pull-request-511.patch (26.7 KB) pull-request-511.patch (38.5 KB) pull-request-511.patch (57 KB) -- https://bugs.ruby-lang.org/