From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: poffice@blade.nagaokaut.ac.jp Delivered-To: poffice@blade.nagaokaut.ac.jp Received: from kankan.nagaokaut.ac.jp (kankan.nagaokaut.ac.jp [133.44.2.24]) by blade.nagaokaut.ac.jp (Postfix) with ESMTP id E1F4E17D5CD5 for ; Fri, 21 Feb 2014 17:06:43 +0900 (JST) Received: from funfun.nagaokaut.ac.jp (funfun.nagaokaut.ac.jp [133.44.2.201]) by kankan.nagaokaut.ac.jp (Postfix) with ESMTP id 5AD9AB5D920 for ; Fri, 21 Feb 2014 16:58:45 +0900 (JST) Received: from funfun.nagaokaut.ac.jp (localhost.nagaokaut.ac.jp [127.0.0.1]) by funfun.nagaokaut.ac.jp (Postfix) with ESMTP id 3B34697A826 for ; Fri, 21 Feb 2014 16:58:47 +0900 (JST) X-Virus-Scanned: amavisd-new at nagaokaut.ac.jp Authentication-Results: funfun.nagaokaut.ac.jp (amavisd-new); dkim=fail (1024-bit key) reason="fail (message has been altered)" header.d=sendgrid.me Received: from funfun.nagaokaut.ac.jp ([127.0.0.1]) by funfun.nagaokaut.ac.jp (funfun.nagaokaut.ac.jp [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IezCQbM6eNZP for ; Fri, 21 Feb 2014 16:58:46 +0900 (JST) Received: from voscc.nagaokaut.ac.jp (voscc.nagaokaut.ac.jp [133.44.1.100]) by funfun.nagaokaut.ac.jp (Postfix) with ESMTP id CAAF497A820 for ; Fri, 21 Feb 2014 16:58:46 +0900 (JST) Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by voscc.nagaokaut.ac.jp (Postfix) with ESMTP id 5D704952439 for ; Fri, 21 Feb 2014 16:58:44 +0900 (JST) Received: from [221.186.184.76] (unknown [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id D779E1207AA; Fri, 21 Feb 2014 16:58:38 +0900 (JST) X-Original-To: ruby-core@ruby-lang.org Delivered-To: ruby-core@ruby-lang.org Received: from o10.shared.sendgrid.net (o10.shared.sendgrid.net [173.193.132.135]) by neon.ruby-lang.org (Postfix) with SMTP id 2F72912058E for ; Fri, 21 Feb 2014 16:58:36 +0900 (JST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.me; h=from:to:references:subject:mime-version:content-type:content-transfer-encoding:list-id; s=smtpapi; bh=BUZEm4L2/OU3s9k9iqEQ11yTsZI=; b=c9TfWmyOrh+aBw7vxl gNuWUGfpIEYmmhG49n9Q3e3skeooW9JJYv0jEue6oAMXjMrJp4bNUgHMk91wJjcN JVG8Z1CuMUrKcgw5YuVP6P2gkCQIOkcJqj5eSIMJzNnkfVvcza+SgKbVGwkM0AHS 1lXWkYWdgrxW2AFOKQmvjVFIE= Received: by mf210.sendgrid.net with SMTP id mf210.10896.5307072B1 Fri, 21 Feb 2014 07:58:35 +0000 (UTC) Received: from herokuapp.com (ec2-54-211-220-117.compute-1.amazonaws.com [54.211.220.117]) by ismtpd-003 (SG) with ESMTP id 1445373ff91.6109.65aca6 Fri, 21 Feb 2014 07:58:34 +0000 (GMT) Date: Fri, 21 Feb 2014 07:58:34 +0000 From: normalperson@yhbt.net To: ruby-core@ruby-lang.org Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Redmine-MailingListIntegration-Message-Ids: 34736 X-Redmine-Project: ruby-trunk X-Redmine-Issue-Id: 9508 X-Redmine-Issue-Author: srawlins X-Redmine-Sender: normalperson X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: OOF Auto-Submitted: auto-generated X-SG-EID: ync6xU2WACa70kv/Ymy4QrNMhiuLXJG8OTL2vJD1yS42f2v/jDdJOQOD52w0KH1br+o0b1uFnEDhddoNLnbzn8ppp+qQIE1VjRSoeRtg0QNjLKiYiD2CSUb6ONLL8XH8c3DPOhPYlZsllmTIOt5VXdwnVdHE2ZnPZ5Fxm9tuIew= X-ML-Name: ruby-core X-Mail-Count: 60932 Subject: [ruby-core:60932] [ruby-trunk - Feature #9508] Add method coverage and branch coverage metrics X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #9508 has been updated by Eric Wong. sam.rawlins@gmail.com wrote: > Good call Eric. I've carried out your suggestion [1], and attached the cumulative patch. > > [1] this commit: https://github.com/srawlins/ruby/commit/cc50eab44f5ce0a4febdc05bdd99a09708e78b7e Thanks Sam! I didn't check very closely, but in places where before where you checked for "iseq->coverage" being true, now jumps straight to "iseq->coverage->(methods|branches|lines)". Wouldn't that crash if iseq->coverage isn't set at all? Unless I'm missing another check elsewhere, perhaps checking: (iseq->coverage && iseq->coverage->FOO) is safer. ---------------------------------------- Feature #9508: Add method coverage and branch coverage metrics https://bugs.ruby-lang.org/issues/9508#change-45328 * Author: Sam Rawlins * Status: Open * Priority: Normal * Assignee: * Category: * Target version: ---------------------------------------- Since the Coverage extension was introduced in Ruby 1.9, Ruby has had built-in line code coverage. Ruby should support more of the basic code coverage metrics [1]. I have a pull request on GitHub ( https://github.com/ruby/ruby/pull/511 ) to add Method Coverage (Function Coverage) and Branch Coverage. I'd love feedback to improve it. Currently, with this feature, Coverage.result would look like: {"/Users/sam/code/ruby/cov_method.rb" => { lines: [1, 2, 2, 20, nil, nil, 2, 2, 2, nil, 0, nil, nil, nil, 1, 0, nil, nil, 1, 1, nil, nil, 1], methods: {1=>2, 15=>0, 19=>1}, branches: {8=>2, 11=>0} }} which includes * the current Ruby line coverage report, * as well as a method report (The method defined on line 1 was called 2 times; the method on line 15 was called 0 times; ...), * and a branch report (the branch on line 8 was called 2 times; the branch on line 11 was called 0 times). Branches -------- Branches include the bodies of if, elsif, else, unless, and when statements, which are all tracked with this new feature. However, this feature is not aware of void bodies, for example: if foo :ok end will report that only one branch exists in the file. It would be better to declare that there is a branch body on line 2, and a void branch body on line 3, or perhaps line 1. This would require the keys of the [:branch] Hash to be something other than line numbers. Perhaps label_no? Perhaps nd_type(node) paired with line or label_no? More Coverage ------------- I think that Statement Coverage, and Condition Coverage could be added to this feature, using the same techniques. Caveats ------- I was not very clear on the bit-arrays used in ruby.h, and just used values for the new macros that seemed to work. Also, I would much rather use Ranges to identify a branch, so that a Coverage analyzer like SimpleCov won't need any kind of Ruby parser to identify and highlight a full chunk of code as a tested branch, or a not tested branch. I'm trying to find how that could be implemented... [1] Wikipedia has good definitions: http://en.wikipedia.org/wiki/Code_coverage ---Files-------------------------------- pull-request-511.patch (26.7 KB) pull-request-511.patch (38.5 KB) -- http://bugs.ruby-lang.org/