Re: Git Test Coverage Report (Tuesday, Sept 25)

[Ben Peart <peartben@gmail.com>]

On 9/26/2018 2:44 PM, Derrick Stolee wrote:
> On 9/26/2018 1:59 PM, Junio C Hamano wrote:
>> Derrick Stolee <stolee@gmail.com> writes:
>>
>>> In an effort to ensure new code is reasonably covered by the test
>>> suite, we now have contrib/coverage-diff.sh to combine the gcov output
>>> from 'make coverage-test ; make coverage-report' with the output from
>>> 'git diff A B' to discover _new_ lines of code that are not covered.
>>>
>>> This report takes the output of these results after running on four
>>> branches:
>>>
>>>          pu: 80e728fa913dc3a1165b6dec9a7afa6052a86325
>>>
>>>         jch: 0c10634844314ab89666ed0a1c7d36dde7ac9689
>>>
>>>        next: 76f2f5c1e34c4dbef1029e2984c2892894c444ce
>>>
>>>      master: fe8321ec057f9231c26c29b364721568e58040f7
>>>
>>> master@{1}: 2d3b1c576c85b7f5db1f418907af00ab88e0c303
>>>
>>> I ran the test suite on each of these branches on an Ubuntu Linux VM,
>>> and I'm missing some dependencies (like apache, svn, and perforce) so
>>> not all tests are run.
>> Thanks.
>>
>>> master@{1}..master:
>>>
>>> builtin/remote.c
>>> 5025425dfff     (   Shulhan     2018-09-13 20:18:33 +0700
>>> 864)            return error(_("No such remote: '%s'"), name);
>>> commit-reach.c
>>> b67f6b26e35     (Derrick Stolee 2018-09-21 08:05:26 -0700
>>> 559)                    continue;
>>> b67f6b26e35     (Derrick Stolee 2018-09-21 08:05:26 -0700
>>> 569)                    from->objects[i].item->flags |= assign_flag;
>>> b67f6b26e35     (Derrick Stolee 2018-09-21 08:05:26 -0700
>>> 570)                    continue;
>>> b67f6b26e35     (Derrick Stolee 2018-09-21 08:05:26 -0700
>>> 576)                    result = 0;
>>> b67f6b26e35     (Derrick Stolee 2018-09-21 08:05:26 -0700
>>> 577)                    goto cleanup;
>>> cat: compat#mingw.c.gcov: No such file or directory
>>> ll-merge.c
>>> d64324cb60e     (Torsten Bögershausen   2018-09-12 21:32:02
>>> +0200       379)                    marker_size =
>>> DEFAULT_CONFLICT_MARKER_SIZE;
>>> remote-curl.c
>>> c3b9bc94b9b     (Elijah Newren  2018-09-05 10:03:07 -0700
>>> 181)            options.filter = xstrdup(value);
>> As I think the data presented here is very valuable, let me ignore
>> the findings of this specific run (which will be fixed by individual
>> authors as/if necessary), and focus on the way the data is presented
>> (which will affect the ease of consumption by authors of future
>> commits).
>>
>> These wrapped blame output lines are harder to view.  Can we have
>> this in plain/text without format=flowed at least?
> 
> Perhaps removing the middle columns of data and just "<sha> <num>) 
> <line>" would be easier? We could also remove tabs to save space. For 
> example:
> 
> builtin/remote.c
> 5025425dfff  864) return error(_("No such remote: '%s'"), name);
> 
> commit-reach.c
> b67f6b26e35 559) continue;
> b67f6b26e35 569) from->objects[i].item->flags |= assign_flag;
> b67f6b26e35 570) continue;
> b67f6b26e35 576) result = 0;
> b67f6b26e35 577) goto cleanup;
> 
> ll-merge.c
> d64324cb60e 379) marker_size = DEFAULT_CONFLICT_MARKER_SIZE;
> 
> remote-curl.c
> c3b9bc94b9b  181) options.filter = xstrdup(value);
> 
> This does still pad the data by a bit, but should be more readable. Most 
> "uncovered" code will be indented at least one level.
> 
> We do lose the author information, but keen readers could identify code 
> they are interested in by filename and then look up the commit by OID 
> later.
> 

I personally find the author data very useful as it makes it trivial for 
me to scan for and find changes I'm responsible for.  Just scanning the 
output of the mail and looking for file names I may have changed lately 
is much more laborious - meaning I'm much less likely to actually do it :-).

Perhaps a reasonable compromise would be to put the author name once 
with the block of changes (like you are doing for the file name) rather 
than on every line that changed and wasn't executed.

>>
>> I personally do not mind a monospaced and non-wrapping website, just
>> I do not mind visiting travis-ci.org for recent results from time to
>> time.  Others may differ.
>>
>> There is an error message from "cat" in it, by the way.
> Thanks! I'll add an 'if' statement when there is no gcov file. This 
> happens for the compat layers that are not compiled in and for the 
> t/helper directory, it seems.
>>
>>> preload-index.c
>>> ae9af12287b     (Nguyễn Thái Ngọc Duy   2018-09-15 19:56:04
>>> +0200       73)                     struct progress_data *pd =
>>> p->progress;
>>> ae9af12287b     (Nguyễn Thái Ngọc Duy   2018-09-15 19:56:04
>>> +0200       75) pthread_mutex_lock(&pd->mutex);
>>> ae9af12287b     (Nguyễn Thái Ngọc Duy   2018-09-15 19:56:04
>>> +0200       76)                     pd->n += last_nr - nr;
>>> ae9af12287b     (Nguyễn Thái Ngọc Duy   2018-09-15 19:56:04
>>> +0200       77) display_progress(pd->progress, pd->n);
>>> ae9af12287b     (Nguyễn Thái Ngọc Duy   2018-09-15 19:56:04
>>> +0200       78) pthread_mutex_unlock(&pd->mutex);
>>> ae9af12287b     (Nguyễn Thái Ngọc Duy   2018-09-15 19:56:04
>>> +0200       79)                     last_nr = nr;
>> I wonder how much we can save the effort that is needed to scan the
>> output if we somehow coalesce these consecutive lines that are
>> attributed to the same commit.
> 
> It could be possible to group consecutive lines together, but hopefully 
> reducing the total data is enough, and we can keep the actual lines 
> visible.
> 
> Thanks,
> 
> -Stolee
>