From: "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: peff@peff.net, Junio C Hamano <gitster@pobox.com>,
Derrick Stolee <dstolee@microsoft.com>
Subject: [PATCH v3 1/1] contrib: add coverage-diff script
Date: Fri, 21 Sep 2018 08:15:55 -0700 (PDT) [thread overview]
Message-ID: <21214cc321f80cf2e9eb0cdb1ec3ebb869ea496d.1537542952.git.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.40.v3.git.gitgitgadget@gmail.com>
From: Derrick Stolee <dstolee@microsoft.com>
We have coverage targets in our Makefile for using gcov to display line
coverage based on our test suite. The way I like to do it is to run:
make coverage-test
make coverage-report
This leaves the repo in a state where every X.c file that was covered has
an X.c.gcov file containing the coverage counts for every line, and "#####"
at every uncovered line.
There have been a few bugs in recent patches what would have been caught
if the test suite covered those blocks (including a few of mine). I want
to work towards a "sensible" amount of coverage on new topics. In my opinion,
this means that any logic should be covered, but the 'die()' blocks covering
very unlikely (or near-impossible) situations may not warrant coverage.
It is important to not measure the coverage of the codebase by what old code
is not covered. To help, I created the 'contrib/coverage-diff.sh' script.
After creating the coverage statistics at a version (say, 'topic') you can
then run
contrib/coverage-diff.sh base topic
to see the lines added between 'base' and 'topic' that are not covered by the
test suite. The output uses 'git blame -c' format so you can find the commits
responsible and view the line numbers for quick access to the context.
Helped-by: Junio C Hamano <gister@pobox.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
contrib/coverage-diff.sh | 79 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 79 insertions(+)
create mode 100755 contrib/coverage-diff.sh
diff --git a/contrib/coverage-diff.sh b/contrib/coverage-diff.sh
new file mode 100755
index 0000000000..48b9a3ae96
--- /dev/null
+++ b/contrib/coverage-diff.sh
@@ -0,0 +1,79 @@
+#!/bin/sh
+
+# Usage: Run 'contrib/coverage-diff.sh <version1> <version2>' from source-root
+# after running
+#
+# make coverage-test
+# make coverage-report
+#
+# while checked out at <version2>. This script combines the *.gcov files
+# generated by the 'make' commands above with 'git diff <version1> <version2>'
+# to report new lines that are not covered by the test suite.
+
+V1=$1
+V2=$2
+
+diff_lines () {
+ perl -e '
+ my $line_num;
+ while (<>) {
+ # Hunk header? Grab the beginning in postimage.
+ if (/^@@ -\d+(?:,\d+)? \+(\d+)(?:,\d+)? @@/) {
+ $line_num = $1;
+ next;
+ }
+
+ # Have we seen a hunk? Ignore "diff --git" etc.
+ next unless defined $line_num;
+
+ # Deleted line? Ignore.
+ if (/^-/) {
+ next;
+ }
+
+ # Show only the line number of added lines.
+ if (/^\+/) {
+ print "$line_num\n";
+ }
+ # Either common context or added line appear in
+ # the postimage. Count it.
+ $line_num++;
+ }
+ '
+}
+
+files=$(git diff --name-only $V1 $V2 -- *.c)
+
+for file in $files
+do
+ git diff $V1 $V2 -- $file \
+ | diff_lines \
+ | sort >new_lines.txt
+
+ if ! test -s new_lines.txt
+ then
+ continue
+ fi
+
+ hash_file=$(echo $file | sed "s/\//\#/")
+ sed -ne '/#####:/{
+ s/ #####://
+ s/:.*//
+ s/ //g
+ p
+ }' "$hash_file.gcov" \
+ | sort >uncovered_lines.txt
+
+ comm -12 uncovered_lines.txt new_lines.txt \
+ | sed -e 's/$/\)/' \
+ | sed -e 's/^/\t/' \
+ >uncovered_new_lines.txt
+
+ grep -q '[^[:space:]]' < uncovered_new_lines.txt && \
+ echo $file && \
+ git blame -c $file \
+ | grep -f uncovered_new_lines.txt
+
+ rm -f new_lines.txt uncovered_lines.txt uncovered_new_lines.txt
+done
+
--
gitgitgadget
next prev parent reply other threads:[~2018-09-21 15:15 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-09-12 16:45 [PATCH 0/1] contrib: Add script to show uncovered "new" lines Derrick Stolee via GitGitGadget
2018-09-12 16:45 ` [PATCH 1/1] contrib: add coverage-diff script Derrick Stolee via GitGitGadget
2018-09-12 22:13 ` Junio C Hamano
2018-09-12 22:54 ` Junio C Hamano
2018-09-13 12:21 ` Derrick Stolee
2018-09-13 14:59 ` Junio C Hamano
2018-09-12 19:14 ` [PATCH 0/1] contrib: Add script to show uncovered "new" lines Derrick Stolee
2018-09-12 19:33 ` Ben Peart
2018-09-12 19:53 ` Junio C Hamano
2018-09-13 14:56 ` [PATCH v2 " Derrick Stolee via GitGitGadget
2018-09-13 14:56 ` [PATCH v2 1/1] contrib: add coverage-diff script Derrick Stolee via GitGitGadget
2018-09-13 17:40 ` Junio C Hamano
2018-09-13 18:15 ` Derrick Stolee
2018-09-13 17:54 ` Eric Sunshine
2018-09-21 15:15 ` [PATCH v3 0/1] contrib: Add script to show uncovered "new" lines Derrick Stolee via GitGitGadget
2018-09-21 15:15 ` Derrick Stolee via GitGitGadget [this message]
2018-09-25 18:36 ` [PATCH v3 1/1] contrib: add coverage-diff script Junio C Hamano
2018-09-21 15:20 ` [PATCH v3 0/1] contrib: Add script to show uncovered "new" lines Derrick Stolee
2018-10-08 14:52 ` [PATCH v4 " Derrick Stolee via GitGitGadget
2018-10-08 14:52 ` [PATCH v4 1/1] contrib: add coverage-diff script Derrick Stolee via GitGitGadget
2018-10-12 3:01 ` [PATCH v4 0/1] contrib: Add script to show uncovered "new" lines Junio C Hamano
2018-10-12 12:09 ` Derrick Stolee
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=21214cc321f80cf2e9eb0cdb1ec3ebb869ea496d.1537542952.git.gitgitgadget@gmail.com \
--to=gitgitgadget@gmail.com \
--cc=dstolee@microsoft.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).