Re: Git Test Coverage Report (Thurs. June 27)

[Derrick Stolee <stolee@gmail.com>]

On 6/28/2019 7:59 PM, Jeff King wrote:
> On Fri, Jun 28, 2019 at 08:23:49AM -0400, Derrick Stolee wrote:
> 
>> On 6/28/2019 2:45 AM, Jeff King wrote:
>>> On Thu, Jun 27, 2019 at 01:35:17PM -0400, Derrick Stolee wrote:
>>>
>>>>> t/helper/test-example-decorate.c
>>>>> 0ebbcf70 29) one = lookup_unknown_object(&one_oid);
>>>>> 0ebbcf70 30) two = lookup_unknown_object(&two_oid);
>>>>> 0ebbcf70 59) three = lookup_unknown_object(&three_oid);
>>>>
>>>> Peff: again interesting that these lines you refactored were not covered, especially
>>>> because they are part of a test helper. Perhaps the tests they were intended for are
>>>> now defunct?
>>>
>>> They should be run by t9004 (and if I replace them with a `die`, they
>>> clearly are). Are you sure your coverage script is not mistaken?
>>
>> It looks like I'm missing the 9000+ tests. The following line was in the script
>> I adapted from another CI job:
>>
>> 	rm -f t/t9*.sh
>>
>> This was probably because the job I adapted from needed to run quickly, but for
>> this coverage report we should do the hard work of running whatever t9*.sh tests
>> we can.
> 
> I suspect most of those _are_ low-value. The git-p4 tests, for instance,
> are mostly exercising the p4 script and not our C code, and the same
> with git-svn. However I wouldn't be surprised if there are a few dusty
> corners they manage to hit that aren't covered elsewhere.
> 
> Still, if it's not too painful to add them in time-wise, it probably
> makes sense for the coverage tests to be as exhaustive as possible.

Unfortunately, even running the t9*.sh tests once (among the two runs:
first with default options and then with several GIT_TEST_* options)
causes the build to go beyond the three hour limit, and the builds time
out.

I'll just need to keep this in mind and do some more diligence myself
to check if things are covered in the 9000 tests before bugging people
about coverage.

Thanks,
-Stolee