From: Derrick Stolee <stolee@gmail.com>
To: Jeff King <peff@peff.net>
Cc: "git@vger.kernel.org" <git@vger.kernel.org>
Subject: Re: Git Test Coverage Report (Thursday, Aug 15)
Date: Tue, 20 Aug 2019 08:54:20 -0400 [thread overview]
Message-ID: <a081ccdb-bb0d-00a1-136a-c8fa54296639@gmail.com> (raw)
In-Reply-To: <20190816181103.GA13894@sigill.intra.peff.net>
On 8/16/2019 2:11 PM, Jeff King wrote:
> On Thu, Aug 15, 2019 at 10:01:04PM -0400, Derrick Stolee wrote:
>
>> Here is today's test coverage report.
>
> Are the scripts you use to generate these available somewhere?
The scripts I originally used are available in contrib/coverage-diff.sh.
However, to allow for some more advanced workflows, I created the following
C# project: https://github.com/derrickstolee/git-test-coverage/tree/master/TestCoverageReport
The runs are done via the following Azure Pipelines build:
https://dev.azure.com/git/git/_build/results?buildId=879
There, you can see how the code is built, tested, and the gcov output
is collected in the TestCoverageReport tool to generate the .txt and .htm
files.
> I think it's useful to look at uncovered code, but I often struggle to
> figure out whether the parts attached to my name are relevant. In
> particular, I think two changes to the report format might help:
>
> 1. Include names alongside commit ids when listing uncovered lines. I
> know that will end up with some overly-long lines, but it makes it
> easy to grep for one's name to find relevant sections of the file
> (as opposed to finding your name at the bottom and
> cross-referencing with actual content lines).
>
> Seeing that an uncovered line is a BUG(), for example, makes it
> easy to know that it's not really an interesting uncovered case in
> the code.
The HTML reports [1] have the following feature: click on a commit in the
commit summary and it highlights the lines from that commit.
[1] https://derrickstolee.github.io/git-test-coverage/reports/2019-08-15.htm
However, the plain-text report _could_ add names. Maybe instead the report
should just group the output by commit instead of branch.
> 2. Include more context. Just taking a random example from this email:
>
>> builtin/rebase.c
>> e191cc8b 129) strbuf_addstr(&buf, strategy_opts);
>
> We know what the uncovered line was trying to do, but more interesting
> is likely the conditional that causes it to be uncovered. In this case
> the surrounding code is:
>
> if (opts->ignore_whitespace) {
> struct strbuf buf = STRBUF_INIT;
>
> if (strategy_opts)
> strbuf_addstr(&buf, strategy_opts);
>
> strbuf_addstr(&buf, " --ignore-space-change");
> free(strategy_opts);
> strategy_opts = strbuf_detach(&buf, NULL);
> }
>
> even the usual 3 lines of diff context would make it a lot quicker to
> understand what's going on (it only kicks in when multiple strategy
> options are used).
In this case, the additional context helps, but only if you expand by
several lines. In other cases, the necessary context could be dozens of
lines.
Perhaps a deeper report could include something like "git format-patch
--function-context" to naturally extend to the appropriate context. It
may be better to have a "block context" for these conditionals. In such
a case, it would be important to mark the "new" lines explicitly so the
context lines don't become too noisy.
> (As an aside, this code leaks the newly allocated buffer and leaves a
> dangling pointer in opts->strategy_opts, but that's all orthogonal to
> the uncovered line; I'll send a separate message to the original
> author).
>
>
> Anyway, I wonder if we could adjust the output of the script to make
> reading it that way a bit easier.
I'll think about this, for sure. When I have time to go deep into the
report (not just the code I wrote) I look for lines that seem to be
non-trivial, then go find their context by looking at the commit.
Here is another example of why the HTML report is helpful: it will
link to the commit diff view [2] or the exact line of the file [3].
(Note: the exact line link only works if we point to the branch that
was tested, not the commit that made the change.)
[2] https://github.com/git/git/commit/e191cc8b8080f63a0080031bf1276269c6bf42dd
[3] https://github.com/git/git/blob/6cf2e4a04610efe882eb663cd36436b80b343fa0/builtin/rebase.c#L129
Thanks,
-Stolee
prev parent reply other threads:[~2019-08-20 12:54 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-16 2:01 Git Test Coverage Report (Thursday, Aug 15) Derrick Stolee
2019-08-16 18:11 ` Jeff King
2019-08-16 18:15 ` Jeff King
2019-08-20 12:54 ` Derrick Stolee [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a081ccdb-bb0d-00a1-136a-c8fa54296639@gmail.com \
--to=stolee@gmail.com \
--cc=git@vger.kernel.org \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).