Re: [PATCH 0/2] t/lib-gpg: a gpgsm fix, a minor improvement, and a question

[Todd Zullinger <tmz@pobox.com>]

SZEDER Gábor wrote:
> On Thu, Feb 07, 2019 at 10:17:44PM -0500, Todd Zullinger wrote:
>> Looking through the build logs for the fedora git packages, I noticed it
>> was missing the GPGSM prereq.
> 
> Just curious: how did you noticed the missing GPGSM prereq?

I just grep the build output for '# SKIP|skipped:' and then
filter out those which I expect (thing like MINGW and
NATIVE_CRLF that aren't likely to be in a Fedora build).

Far more manual than the slick method you have below. :)

> I'm asking because I use a patch for a good couple of months now that
> collects the prereqs missed by test cases and prints them at the end
> of 'make test'.  Its output looks like this:
> 
>   https://travis-ci.org/szeder/git/jobs/490944032#L2358
> 
> Since you seem to be interested in that sort of thing as well, perhaps
> it would be worth to have something like this in git.git?  It's just
> that I have been too wary of potentially annoying other contributors
> by adding (what might be perceived as) clutter to their 'make test'
> output :)

Indeed, I think that would be useful.  At the very least,
the .missing_prereqs files look quite handy.  I wouldn't
mind the output from 'make test' either, but building
packages surely shifts my perspective toward more verbose
build logs than someone hacking on git regularly and reading
the 'make test' output.

>> I don't know if there are other packagers or builders who run into this,
>> so maybe it's not worth much effort to try and have the test suite cope
>> better.  It took me longer than I would have liked to track it down, so
>> I thought I'd mention it in case anyone else has run into this or has
>> thoughts on how to improve lib-gpg.sh while waiting for GnuPG to improve
>> this area.
> 
> I stumbled upon this when Dscho inadvertently broke a test script on
> setups without gpg last year; sorry for not speaking about it.  I
> noticed it in our Travis CI builds on macOS, because it (macOS itself
> or Homebrew? I don't know) defaulted to gpg2 already back then, and to
> make matters worse its sun_path is on the shorter end, and the path
> to the build dir on Travis CI includes the GitHub user/repo as well.

Heh, I figured it was a rather specific group of folks that
might run into this.

>> A GIT_TEST_GNUPGHOME_ROOT var to set the root path for the GNUPGHOME
>> dirs in the tests is one thought I had, but didn't try to put it into
>> patch form.  Setting the --root test option is probably enough control
>> for most cases.
> 
> A potential issue I see with GIT_TEST_GNUPGHOME_ROOT is that there are
> several test scripts involving gpg, and if GIT_TEST_GNUPGHOME_ROOT is
> set for the whole 'make test', then they might interfere with each
> other when they happen to be run at the same time.

Yeah, I was envisioning that var as something which set the
base dir, under which the normal test directories would
live.  Basically, like setting --root, but only for the
GnuPG bits.

I'm not impressed by that idea (and I'm even less so after
realizing how it would most likely make it harder to gather
up the results in the CI scripts).  I mainly tossed it out
in the hope someone would reply with a better method. ;)

> In the meantime I came up with a '--short-trash-dir' option to
> test-lib, which turns 'trash directory.t7612-merge-verify-signatures'
> into 'trash dir.t7612'.  It works, but I don't really like it, and it
> required various adjustments to the CI build scripts, notably to the
> part in 'ci/print-test-failures.sh' that includes the trash dir of
> failed test scripts in the build log.

I can certainly live with setting '--root' to a shorter path
and waiting to see if GnuPG upstream will come up with
something a little more friendly to users like us - running
gpg in a test suite.  Though if we do just wait it out,
maybe we could/should add a note in t/README or t/lib-gpg.sh
about this to warn others?

-- 
Todd