[PATCH] gpg-interface: fix for gpgsm v2.3

[PATCH] gpg-interface: fix for gpgsm v2.3

[Fabian Stelzer]

gpgsm v2.3 changed some details about its output:
 - instead of displaying `fingerprint:` for keys it will print `sha1
   fpr:` and `sha2 fpr:`
 - some wording of errors has changed
 - signing will omit an extra debug output line before the [GNUPG]: tag

This change adjusts the gpgsm test prerequisite to work with v2.3 as
well by accepting `sha1 fpr:` as well as `fingerprint:` and allows both
variants of errors for unknown certs.
Checking for successful signature creation will omit the leading NL in
its search string.
---

I am not a user of gpgsm but noticed that the GPGSM test prereq was disabled 
on my runs so i investigated. The `fix` seems rather trivial and I tried to 
test this as thorough as possible. I ran the test suite on machines 
available to me (fedora35, centos8) and did a full CI run on github without 
any issues.
But if you actually use gpgsm with git please give this a go and let me know 
if I missed anything.


 gpg-interface.c | 2 +-
 t/lib-gpg.sh    | 4 ++--
 t/t4202-log.sh  | 5 ++++-
 3 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/gpg-interface.c b/gpg-interface.c
index b52eb0e2e0..299e7f588a 100644
--- a/gpg-interface.c
+++ b/gpg-interface.c
@@ -939,7 +939,7 @@ static int sign_buffer_gpg(struct strbuf *buffer, struct strbuf *signature,
 			   signature, 1024, &gpg_status, 0);
 	sigchain_pop(SIGPIPE);
 
-	ret |= !strstr(gpg_status.buf, "\n[GNUPG:] SIG_CREATED ");
+	ret |= !strstr(gpg_status.buf, "[GNUPG:] SIG_CREATED ");
 	strbuf_release(&gpg_status);
 	if (ret)
 		return error(_("gpg failed to sign the data"));
diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
index 3e7ee1386a..6c2dd4b14b 100644
--- a/t/lib-gpg.sh
+++ b/t/lib-gpg.sh
@@ -73,8 +73,8 @@ test_lazy_prereq GPGSM '
 		--import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 &&
 
 	gpgsm --homedir "${GNUPGHOME}" -K |
-	grep fingerprint: |
-	cut -d" " -f4 |
+	grep -E "(fingerprint|sha1 fpr):" |
+	cut -d":" -f2- | tr -d " " |
 	tr -d "\\n" >"${GNUPGHOME}/trustlist.txt" &&
 
 	echo " S relax" >>"${GNUPGHOME}/trustlist.txt" &&
diff --git a/t/t4202-log.sh b/t/t4202-log.sh
index 5049559861..08556493ce 100755
--- a/t/t4202-log.sh
+++ b/t/t4202-log.sh
@@ -1931,7 +1931,10 @@ test_expect_success GPGSM 'log --graph --show-signature for merged tag x509 miss
 	git merge --no-ff -m msg signed_tag_x509_nokey &&
 	GNUPGHOME=. git log --graph --show-signature -n1 plain-x509-nokey >actual &&
 	grep "^|\\\  merged tag" actual &&
-	grep "^| | gpgsm: certificate not found" actual
+	(
+		grep "^| | gpgsm: certificate not found" actual ||
+		grep "^| | gpgsm: failed to find the certificate: Not found" actual
+	)
 '
 
 test_expect_success GPGSM 'log --graph --show-signature for merged tag x509 bad signature' '
-- 
2.34.1

Re: [PATCH] gpg-interface: fix for gpgsm v2.3

[Junio C Hamano]

Fabian Stelzer <fs@gigacodes.de> writes:

> gpgsm v2.3 changed some details about its output:
>  - instead of displaying `fingerprint:` for keys it will print `sha1
>    fpr:` and `sha2 fpr:`
>  - some wording of errors has changed
>  - signing will omit an extra debug output line before the [GNUPG]: tag
>
> This change adjusts the gpgsm test prerequisite to work with v2.3 as
> well by accepting `sha1 fpr:` as well as `fingerprint:` and allows both
> variants of errors for unknown certs.

OK, so the change is meant to add support for the new behaviour
without deprecating/removing the support for the older one.  Good.

> Checking for successful signature creation will omit the leading NL in
> its search string.

I think this is to adjust for "will omit an extra debug output"; as
long as we still ensure that the "[GNUPG:] SIG_CREATED" comes at the
beginning of a line with some other means, I think that is a good
change.

> I am not a user of gpgsm but noticed that the GPGSM test prereq was disabled 
> on my runs so i investigated. The `fix` seems rather trivial and I tried to 
> test this as thorough as possible. I ran the test suite on machines 
> available to me (fedora35, centos8) and did a full CI run on github without 
> any issues.
> But if you actually use gpgsm with git please give this a go and let me know 
> if I missed anything.

Yup, thanks for a call for help.  I am not gpgsm user either and
testing by actual users is very much appreciated.

> @@ -939,7 +939,7 @@ static int sign_buffer_gpg(struct strbuf *buffer, struct strbuf *signature,
>  			   signature, 1024, &gpg_status, 0);
>  	sigchain_pop(SIGPIPE);
>  
> -	ret |= !strstr(gpg_status.buf, "\n[GNUPG:] SIG_CREATED ");
> +	ret |= !strstr(gpg_status.buf, "[GNUPG:] SIG_CREATED ");

This I am not sure about.  I understand that the intention is to
allow this at the beginning of gpg_status.buf, but not to allow the
substring appear in the middle of an otherwise unrelated line.  I am
afraid that the new check is a bit too lose for that.

Totally untested but just to illustrate the idea...

 gpg-interface.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git c/gpg-interface.c w/gpg-interface.c
index b52eb0e2e0..4238e60dfa 100644
--- c/gpg-interface.c
+++ w/gpg-interface.c
@@ -920,6 +920,7 @@ static int sign_buffer_gpg(struct strbuf *buffer, struct strbuf *signature,
 	struct child_process gpg = CHILD_PROCESS_INIT;
 	int ret;
 	size_t bottom;
+	const char *cp;
 	struct strbuf gpg_status = STRBUF_INIT;
 
 	strvec_pushl(&gpg.args,
@@ -939,7 +940,13 @@ static int sign_buffer_gpg(struct strbuf *buffer, struct strbuf *signature,
 			   signature, 1024, &gpg_status, 0);
 	sigchain_pop(SIGPIPE);
 
-	ret |= !strstr(gpg_status.buf, "\n[GNUPG:] SIG_CREATED ");
+	for (cp = gpg_status.buf;
+	     cp && (cp = strstr(cp, "[GNUPG:] SIG_CREATED "));
+	     cp++) {
+		if (cp == gpg_status.buf || cp[-1] == '\n')
+			break; /* found */
+	}
+	ret |= !cp;
 	strbuf_release(&gpg_status);
 	if (ret)
 		return error(_("gpg failed to sign the data"));

Re: [PATCH] gpg-interface: fix for gpgsm v2.3

[Todd Zullinger]

Hi Fabian,

Fabian Stelzer wrote:
> gpgsm v2.3 changed some details about its output:
>  - instead of displaying `fingerprint:` for keys it will print `sha1
>    fpr:` and `sha2 fpr:`
>  - some wording of errors has changed
>  - signing will omit an extra debug output line before the [GNUPG]: tag

Thanks for sending this.  I noticed these as well, as Fedora
started shipping gnupg-2.3 a few months back.  I have been
trying (and failing) to make time to submit (when I know I
won't be too distracted to actually converse about them).
The commits I made for the tests in Fedora are all here:

    https://src.fedoraproject.org/rpms/git/c/a7d2f7e53

I don't intend that as something anyone here should feel the
need to chase down.  But since they provide some additional
context on the changes I made in the same area, it might
help if anyone's curious.

> diff --git a/gpg-interface.c b/gpg-interface.c
> index b52eb0e2e0..299e7f588a 100644
> --- a/gpg-interface.c
> +++ b/gpg-interface.c
> @@ -939,7 +939,7 @@ static int sign_buffer_gpg(struct strbuf *buffer, struct strbuf *signature,
>  			   signature, 1024, &gpg_status, 0);
>  	sigchain_pop(SIGPIPE);
>  
> -	ret |= !strstr(gpg_status.buf, "\n[GNUPG:] SIG_CREATED ");
> +	ret |= !strstr(gpg_status.buf, "[GNUPG:] SIG_CREATED ");
>  	strbuf_release(&gpg_status);
>  	if (ret)
>  		return error(_("gpg failed to sign the data"));

As Junio noted, this loosens the GPG parsing a good bit.  I
worried it could lead to security issues as well.  The
simple fix I made in Fedora was to add a newline to the
gpg_status string buffer before adding the command output to
it:

    diff --git a/gpg-interface.c b/gpg-interface.c
    index 3e7255a2a9..d179dfb3ab 100644
    --- a/gpg-interface.c
    +++ b/gpg-interface.c
    @@ -859,6 +859,12 @@ static int sign_buffer_gpg(struct strbuf *buffer, struct strbuf *signature,
     
	bottom = signature->len;
     
    +	/*
    +	 * Ensure gpg_status begins with a newline or we'll fail to match if
    +	 * the SIG_CREATED line is at the start of the gpg output.
    +	 */
    +	strbuf_addch(&gpg_status, '\n');
    +
	/*
	* When the username signingkey is bad, program could be terminated
	* because gpg exits without reading and then write gets SIGPIPE.

https://src.fedoraproject.org/rpms/git/blob/a7d2f7e53/f/0005-gpg-interface-match-SIG_CREATED-if-it-s-the-first-li.patch

But that seemed like a bit of a hack.  What I had queued up
to submit for discussion (as I'm not sure that it isn't
entirely horrible) used the string-list API to parse the gpg
output:

    diff --git a/gpg-interface.c b/gpg-interface.c
    index b52eb0e2e0..e63ccdcb11 100644
    --- a/gpg-interface.c
    +++ b/gpg-interface.c
    @@ -921,6 +921,7 @@ static int sign_buffer_gpg(struct strbuf *buffer, struct strbuf *signature,
     	int ret;
     	size_t bottom;
     	struct strbuf gpg_status = STRBUF_INIT;
    +	struct string_list lines = { .cmp = starts_with };
     
     	strvec_pushl(&gpg.args,
     		     use_format->program,
    @@ -939,8 +940,11 @@ static int sign_buffer_gpg(struct strbuf *buffer, struct strbuf *signature,
     			   signature, 1024, &gpg_status, 0);
     	sigchain_pop(SIGPIPE);
     
    -	ret |= !strstr(gpg_status.buf, "\n[GNUPG:] SIG_CREATED ");
    +	string_list_split_in_place(&lines, gpg_status.buf, '\n', -1);
    +	ret |= !unsorted_string_list_has_string(&lines, "[GNUPG:] SIG_CREATED ");
     	strbuf_release(&gpg_status);
    +	string_list_clear(&lines, 0);
    +
     	if (ret)
     		return error(_("gpg failed to sign the data"));

That's the commit I was most in doubt about though, as my C
"skills" are close to non-existant.  I'd rather have
something ugly and clear (like the `strbuf_addch(...)`
above) than clever and wrong in gpg-interface.c.

(To be clear, I mean "clever and wrong" in regard to my use
of the string list API, not anyone else's code.)

> diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
> index 3e7ee1386a..6c2dd4b14b 100644
> --- a/t/lib-gpg.sh
> +++ b/t/lib-gpg.sh
> @@ -73,8 +73,8 @@ test_lazy_prereq GPGSM '
>  		--import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 &&
>  
>  	gpgsm --homedir "${GNUPGHOME}" -K |
> -	grep fingerprint: |
> -	cut -d" " -f4 |
> +	grep -E "(fingerprint|sha1 fpr):" |
> +	cut -d":" -f2- | tr -d " " |
>  	tr -d "\\n" >"${GNUPGHOME}/trustlist.txt" &&

I think this whole thing can (and should) be simplified by
using gpg's --with-colons output which is intended to be
machine parsable.

If we'd been using that previously, we wouldn't need to make
any further changes here.

I think we're making our lives difficult by screen scraping
here wher we don't need to do so.

The change I made for the Fedora package to fix this does
it like this:

    --- a/t/lib-gpg.sh
    +++ b/t/lib-gpg.sh
    @@ -72,12 +72,10 @@ test_lazy_prereq GPGSM '
                    --passphrase-fd 0 --pinentry-mode loopback \
                    --import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 &&

    -	gpgsm --homedir "${GNUPGHOME}" -K |
    -	grep fingerprint: |
    -	cut -d" " -f4 |
    -	tr -d "\\n" >"${GNUPGHOME}/trustlist.txt" &&
    +	gpgsm --homedir "${GNUPGHOME}" -K --with-colons |
    +	awk -F ":" "/^fpr:/ {printf \"%s S relax\\n\", \$10}" \
    +		>"${GNUPGHOME}/trustlist.txt" &&

    -	echo " S relax" >>"${GNUPGHOME}/trustlist.txt" &&
            echo hello | gpgsm --homedir "${GNUPGHOME}" >/dev/null \
                   -u committer@example.com -o /dev/null --sign -
     '

With a commit message:

    https://src.fedoraproject.org/rpms/git/blob/a7d2f7e53/f/0001-t-lib-gpg-use-with-colons-when-parsing-gpgsm-output.patch

I was hoping to submit that small series in the next day or
two, while I've got a few days away from $work.  If doing it
that way is appealing, I can submit them.  But only if that
looks like a reasonable improvement to you and others.

>  	echo " S relax" >>"${GNUPGHOME}/trustlist.txt" &&
> diff --git a/t/t4202-log.sh b/t/t4202-log.sh
> index 5049559861..08556493ce 100755
> --- a/t/t4202-log.sh
> +++ b/t/t4202-log.sh
> @@ -1931,7 +1931,10 @@ test_expect_success GPGSM 'log --graph --show-signature for merged tag x509 miss
>  	git merge --no-ff -m msg signed_tag_x509_nokey &&
>  	GNUPGHOME=. git log --graph --show-signature -n1 plain-x509-nokey >actual &&
>  	grep "^|\\\  merged tag" actual &&
> -	grep "^| | gpgsm: certificate not found" actual
> +	(
> +		grep "^| | gpgsm: certificate not found" actual ||
> +		grep "^| | gpgsm: failed to find the certificate: Not found" actual
> +	)
>  '
>  
>  test_expect_success GPGSM 'log --graph --show-signature for merged tag x509 bad signature' '

Can we make this simpler by adjusting the grep pattern?
It's certainly a slight trade-off in ease of reading, but it
saves a subshell and an extra command:

    -	grep "^| | gpgsm: certificate not found" actual
    +	grep -Ei "^| | gpgsm:( failed to find the)? certificate:? not found" actual

I did that in a separate patch:

    https://src.fedoraproject.org/rpms/git/blob/a7d2f7e53/f/0004-t4202-match-gpgsm-output-from-GnuPG-2.3.patch

IMO, this is a bug in gnupg-2.3.  I submitted a patch to
resolve it back in November, but have not gotten any
response as yet. :(

    https://lists.gnupg.org/pipermail/gnupg-devel/2021-November/034991.html

Not that it will preclude us from having to fix this for the
test suite, but it's worth noting why the change is needed
(and when it will no longer be relevant -- if/when we don't
care to support the few early gnupg-2.3.x releases).

Thanks,

-- 
Todd

Re: [PATCH] gpg-interface: fix for gpgsm v2.3

[Junio C Hamano]

Todd Zullinger <tmz@pobox.com> writes:

>     -	ret |= !strstr(gpg_status.buf, "\n[GNUPG:] SIG_CREATED ");
>     +	string_list_split_in_place(&lines, gpg_status.buf, '\n', -1);
>     +	ret |= !unsorted_string_list_has_string(&lines, "[GNUPG:] SIG_CREATED ");

Is "SIG_CREATED " supposed to be at the end of that line?  I thought
that has_string() asks for an exact match, and unfortunately(?)
there is not the string_list_has_string_that_has_this_prefix()
function.  So...

Re: [PATCH] gpg-interface: fix for gpgsm v2.3

[Todd Zullinger]

Junio C Hamano wrote:
> Todd Zullinger <tmz@pobox.com> writes:
> 
>>     -	ret |= !strstr(gpg_status.buf, "\n[GNUPG:] SIG_CREATED ");
>>     +	string_list_split_in_place(&lines, gpg_status.buf, '\n', -1);
>>     +	ret |= !unsorted_string_list_has_string(&lines, "[GNUPG:] SIG_CREATED ");
> 
> Is "SIG_CREATED " supposed to be at the end of that line?  I thought
> that has_string() asks for an exact match, and unfortunately(?)
> there is not the string_list_has_string_that_has_this_prefix()
> function.  So...

By default, yes.  The string_list struct uses strcmp() if no
cmp function is given.  That's why the previous chunk has:

    struct string_list lines = { .cmp = starts_with };

There aren't any similar uses in the code, which is just one
of the reasons I wasn't confident that it was a good idea or
even a good implementation.

-- 
Todd

Re: [PATCH] gpg-interface: fix for gpgsm v2.3

[Junio C Hamano]

Todd Zullinger <tmz@pobox.com> writes:

> By default, yes.  The string_list struct uses strcmp() if no
> cmp function is given.  That's why the previous chunk has:
>
>     struct string_list lines = { .cmp = starts_with };

Ahh, I missed that part.  Sounds correct to me, then.

Splitting only to iterate over these lines sounds wasteful to me,
though, since we do not need access to these lines only one at a
time and there is no need to keep an array of them.

Thanks.

Re: [PATCH] gpg-interface: fix for gpgsm v2.3

[Fabian Stelzer]

On 03.02.2022 15:01, Todd Zullinger wrote:
>Hi Fabian,
>
>Fabian Stelzer wrote:
>> gpgsm v2.3 changed some details about its output:
>>  - instead of displaying `fingerprint:` for keys it will print `sha1
>>    fpr:` and `sha2 fpr:`
>>  - some wording of errors has changed
>>  - signing will omit an extra debug output line before the [GNUPG]: tag
>
>Thanks for sending this.  I noticed these as well, as Fedora
>started shipping gnupg-2.3 a few months back.  I have been
>trying (and failing) to make time to submit (when I know I
>won't be too distracted to actually converse about them).
>The commits I made for the tests in Fedora are all here:
>
>    https://src.fedoraproject.org/rpms/git/c/a7d2f7e53
>
>I don't intend that as something anyone here should feel the
>need to chase down.  But since they provide some additional
>context on the changes I made in the same area, it might
>help if anyone's curious.
>
>> diff --git a/gpg-interface.c b/gpg-interface.c
>> index b52eb0e2e0..299e7f588a 100644
>> --- a/gpg-interface.c
>> +++ b/gpg-interface.c
>> @@ -939,7 +939,7 @@ static int sign_buffer_gpg(struct strbuf *buffer, struct strbuf *signature,
>>  			   signature, 1024, &gpg_status, 0);
>>  	sigchain_pop(SIGPIPE);
>>
>> -	ret |= !strstr(gpg_status.buf, "\n[GNUPG:] SIG_CREATED ");
>> +	ret |= !strstr(gpg_status.buf, "[GNUPG:] SIG_CREATED ");
>>  	strbuf_release(&gpg_status);
>>  	if (ret)
>>  		return error(_("gpg failed to sign the data"));
>
>As Junio noted, this loosens the GPG parsing a good bit.  I
>worried it could lead to security issues as well.  The
>simple fix I made in Fedora was to add a newline to the
>gpg_status string buffer before adding the command output to
>it:
>
>    diff --git a/gpg-interface.c b/gpg-interface.c
>    index 3e7255a2a9..d179dfb3ab 100644
>    --- a/gpg-interface.c
>    +++ b/gpg-interface.c
>    @@ -859,6 +859,12 @@ static int sign_buffer_gpg(struct strbuf *buffer, struct strbuf *signature,
>
>	bottom = signature->len;
>
>    +	/*
>    +	 * Ensure gpg_status begins with a newline or we'll fail to match if
>    +	 * the SIG_CREATED line is at the start of the gpg output.
>    +	 */
>    +	strbuf_addch(&gpg_status, '\n');
>    +
>	/*
>	* When the username signingkey is bad, program could be terminated
>	* because gpg exits without reading and then write gets SIGPIPE.
>
>https://src.fedoraproject.org/rpms/git/blob/a7d2f7e53/f/0005-gpg-interface-match-SIG_CREATED-if-it-s-the-first-li.patch
>
>But that seemed like a bit of a hack.  What I had queued up
>to submit for discussion (as I'm not sure that it isn't
>entirely horrible) used the string-list API to parse the gpg
>output:
>
>    diff --git a/gpg-interface.c b/gpg-interface.c
>    index b52eb0e2e0..e63ccdcb11 100644
>    --- a/gpg-interface.c
>    +++ b/gpg-interface.c
>    @@ -921,6 +921,7 @@ static int sign_buffer_gpg(struct strbuf *buffer, struct strbuf *signature,
>     	int ret;
>     	size_t bottom;
>     	struct strbuf gpg_status = STRBUF_INIT;
>    +	struct string_list lines = { .cmp = starts_with };
>
>     	strvec_pushl(&gpg.args,
>     		     use_format->program,
>    @@ -939,8 +940,11 @@ static int sign_buffer_gpg(struct strbuf *buffer, struct strbuf *signature,
>     			   signature, 1024, &gpg_status, 0);
>     	sigchain_pop(SIGPIPE);
>
>    -	ret |= !strstr(gpg_status.buf, "\n[GNUPG:] SIG_CREATED ");
>    +	string_list_split_in_place(&lines, gpg_status.buf, '\n', -1);
>    +	ret |= !unsorted_string_list_has_string(&lines, "[GNUPG:] SIG_CREATED ");
>     	strbuf_release(&gpg_status);
>    +	string_list_clear(&lines, 0);
>    +
>     	if (ret)
>     		return error(_("gpg failed to sign the data"));
>
>That's the commit I was most in doubt about though, as my C
>"skills" are close to non-existant.  I'd rather have
>something ugly and clear (like the `strbuf_addch(...)`
>above) than clever and wrong in gpg-interface.c.
>
>(To be clear, I mean "clever and wrong" in regard to my use
>of the string list API, not anyone else's code.)

string_list_split seems a bit like overkill.  My first thought was actually 
sth like:

cp = strstr(gpg_status.buf, "[GNUPG]: SIG_CREATED");
if (cp == gpg_status.buf || --cp == '\n')
	// found

But that would fail in case the string came up before the actual success 
message at the beginning of a line. So Junios variant of using the for() 
loop is more robust and would normally find the correct string on its first 
iteration anyway.

>
>> diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
>> index 3e7ee1386a..6c2dd4b14b 100644
>> --- a/t/lib-gpg.sh
>> +++ b/t/lib-gpg.sh
>> @@ -73,8 +73,8 @@ test_lazy_prereq GPGSM '
>>  		--import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 &&
>>
>>  	gpgsm --homedir "${GNUPGHOME}" -K |
>> -	grep fingerprint: |
>> -	cut -d" " -f4 |
>> +	grep -E "(fingerprint|sha1 fpr):" |
>> +	cut -d":" -f2- | tr -d " " |
>>  	tr -d "\\n" >"${GNUPGHOME}/trustlist.txt" &&
>
>I think this whole thing can (and should) be simplified by
>using gpg's --with-colons output which is intended to be
>machine parsable.

I looked for sth like this but gpgs --help does not list it so i didn't dig 
deeper. I've checked the blame and it seems like this was introduced >19 
years ago. So i guess we can probably use this ^^

>
>If we'd been using that previously, we wouldn't need to make
>any further changes here.
>
>I think we're making our lives difficult by screen scraping
>here wher we don't need to do so.
>
>The change I made for the Fedora package to fix this does
>it like this:
>
>    --- a/t/lib-gpg.sh
>    +++ b/t/lib-gpg.sh
>    @@ -72,12 +72,10 @@ test_lazy_prereq GPGSM '
>                    --passphrase-fd 0 --pinentry-mode loopback \
>                    --import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 &&
>
>    -	gpgsm --homedir "${GNUPGHOME}" -K |
>    -	grep fingerprint: |
>    -	cut -d" " -f4 |
>    -	tr -d "\\n" >"${GNUPGHOME}/trustlist.txt" &&
>    +	gpgsm --homedir "${GNUPGHOME}" -K --with-colons |
>    +	awk -F ":" "/^fpr:/ {printf \"%s S relax\\n\", \$10}" \
>    +		>"${GNUPGHOME}/trustlist.txt" &&

This does not quite work for me. It will add the fingerprint without the 
colons into the trustlist which is not valid :/
It would need sth like:
+       gpgsm --with-colons --homedir "${GNUPGHOME}" -K |
+       awk -F ":" "/^(fpr|fingerprint):/ {gsub(/.{2}/, \"&:\", \$10); 
printf \"%s S relax\\n\", substr(\$10, 1, length(\$10)-1)}" \
+        >"${GNUPGHOME}/trustlist.txt" &&

which looks needlessly complicated. There is probably some better way to do 
this with/without awk.

>
>    -	echo " S relax" >>"${GNUPGHOME}/trustlist.txt" &&
>            echo hello | gpgsm --homedir "${GNUPGHOME}" >/dev/null \
>                   -u committer@example.com -o /dev/null --sign -
>     '
>
>With a commit message:
>
>    https://src.fedoraproject.org/rpms/git/blob/a7d2f7e53/f/0001-t-lib-gpg-use-with-colons-when-parsing-gpgsm-output.patch
>
>I was hoping to submit that small series in the next day or
>two, while I've got a few days away from $work.  If doing it
>that way is appealing, I can submit them.  But only if that
>looks like a reasonable improvement to you and others.
>
>>  	echo " S relax" >>"${GNUPGHOME}/trustlist.txt" &&
>> diff --git a/t/t4202-log.sh b/t/t4202-log.sh
>> index 5049559861..08556493ce 100755
>> --- a/t/t4202-log.sh
>> +++ b/t/t4202-log.sh
>> @@ -1931,7 +1931,10 @@ test_expect_success GPGSM 'log --graph --show-signature for merged tag x509 miss
>>  	git merge --no-ff -m msg signed_tag_x509_nokey &&
>>  	GNUPGHOME=. git log --graph --show-signature -n1 plain-x509-nokey >actual &&
>>  	grep "^|\\\  merged tag" actual &&
>> -	grep "^| | gpgsm: certificate not found" actual
>> +	(
>> +		grep "^| | gpgsm: certificate not found" actual ||
>> +		grep "^| | gpgsm: failed to find the certificate: Not found" actual
>> +	)
>>  '
>>
>>  test_expect_success GPGSM 'log --graph --show-signature for merged tag x509 bad signature' '
>
>Can we make this simpler by adjusting the grep pattern?
>It's certainly a slight trade-off in ease of reading, but it
>saves a subshell and an extra command:
>
>    -	grep "^| | gpgsm: certificate not found" actual
>    +	grep -Ei "^| | gpgsm:( failed to find the)? certificate:? not found" actual

Thanks, will do.

>
>I did that in a separate patch:
>
>    https://src.fedoraproject.org/rpms/git/blob/a7d2f7e53/f/0004-t4202-match-gpgsm-output-from-GnuPG-2.3.patch
>
>IMO, this is a bug in gnupg-2.3.  I submitted a patch to
>resolve it back in November, but have not gotten any
>response as yet. :(
>
>    https://lists.gnupg.org/pipermail/gnupg-devel/2021-November/034991.html
>
>Not that it will preclude us from having to fix this for the
>test suite, but it's worth noting why the change is needed
>(and when it will no longer be relevant -- if/when we don't
>care to support the few early gnupg-2.3.x releases).
>
>Thanks,
>
>-- 
>Todd

Re: [PATCH] gpg-interface: fix for gpgsm v2.3

[Todd Zullinger]

Hi Fabien,

Fabian Stelzer wrote:
> On 03.02.2022 15:01, Todd Zullinger wrote:
>> (To be clear, I mean "clever and wrong" in regard to my use
>> of the string list API, not anyone else's code.)
>
> string_list_split seems a bit like overkill.

I have little doubt that the string_list_split() method is
far from ideal. :)

> I looked for sth like this but gpgs --help does not list it so i didn't dig
> deeper. I've checked the blame and it seems like this was introduced >19
> years ago. So i guess we can probably use this ^^

Indeed, the --with-colons output goes much further back in
the GnuPG history than Git will ever have to care about.

>>    --- a/t/lib-gpg.sh
>>    +++ b/t/lib-gpg.sh
>>    @@ -72,12 +72,10 @@ test_lazy_prereq GPGSM '
>>                    --passphrase-fd 0 --pinentry-mode loopback \
>>                    --import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 &&
>> 
>>    -	gpgsm --homedir "${GNUPGHOME}" -K |
>>    -	grep fingerprint: |
>>    -	cut -d" " -f4 |
>>    -	tr -d "\\n" >"${GNUPGHOME}/trustlist.txt" &&
>>    +	gpgsm --homedir "${GNUPGHOME}" -K --with-colons |
>>    +	awk -F ":" "/^fpr:/ {printf \"%s S relax\\n\", \$10}" \
>>    +		>"${GNUPGHOME}/trustlist.txt" &&
> 
> This does not quite work for me. It will add the fingerprint without the
> colons into the trustlist which is not valid :/

The colons are optional, and have been documented as such
since cb1840720 ((Agent Configuration): New section.,
2005-04-20).  The text in the gpg-agent docs from GnuPG 2.2
say:

    Colons may optionally be used to separate the bytes of a
    fingerprint; this enables cutting and pasting the
    fingerprint from a key listing output.

Source: https://dev.gnupg.org/source/gnupg/browse/STABLE-BRANCH-2-2/doc/gpg-agent.texi;8021fe7670c79d5c698ec3fb600b02a9e5afb415$756?as=source&blame=off

How did it fail for you?  It passes all the tests when I've
run it against Fedora and RHEL-based hosts.  If it's flaky
on other systems, that would put a damper on doing it this
way.  Though it _should_ work.

[Note to myself] We don't just generate the key data,
trustlist, etc. and store it in t/lib-gpg like we do with
some other files per b41a36e635 (tests: create gpg homedir
on the fly, 2014-12-12).  That was because the gnupg home
directory layout changed a bit between 2.0 and 2.1.

Thanks,

-- 
Todd

Re: [PATCH] gpg-interface: fix for gpgsm v2.3

[Fabian Stelzer]

On 07.02.2022 11:38, Todd Zullinger wrote:
>Hi Fabien,
>
>Fabian Stelzer wrote:
>> On 03.02.2022 15:01, Todd Zullinger wrote:
>>> (To be clear, I mean "clever and wrong" in regard to my use
>>> of the string list API, not anyone else's code.)
>>
>> string_list_split seems a bit like overkill.
>
>I have little doubt that the string_list_split() method is
>far from ideal. :)
>
>> I looked for sth like this but gpgs --help does not list it so i didn't dig
>> deeper. I've checked the blame and it seems like this was introduced >19
>> years ago. So i guess we can probably use this ^^
>
>Indeed, the --with-colons output goes much further back in
>the GnuPG history than Git will ever have to care about.
>
>>>    --- a/t/lib-gpg.sh
>>>    +++ b/t/lib-gpg.sh
>>>    @@ -72,12 +72,10 @@ test_lazy_prereq GPGSM '
>>>                    --passphrase-fd 0 --pinentry-mode loopback \
>>>                    --import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 &&
>>>
>>>    -	gpgsm --homedir "${GNUPGHOME}" -K |
>>>    -	grep fingerprint: |
>>>    -	cut -d" " -f4 |
>>>    -	tr -d "\\n" >"${GNUPGHOME}/trustlist.txt" &&
>>>    +	gpgsm --homedir "${GNUPGHOME}" -K --with-colons |
>>>    +	awk -F ":" "/^fpr:/ {printf \"%s S relax\\n\", \$10}" \
>>>    +		>"${GNUPGHOME}/trustlist.txt" &&
>>
>> This does not quite work for me. It will add the fingerprint without the
>> colons into the trustlist which is not valid :/
>
>The colons are optional, and have been documented as such
>since cb1840720 ((Agent Configuration): New section.,
>2005-04-20).  The text in the gpg-agent docs from GnuPG 2.2
>say:
>
>    Colons may optionally be used to separate the bytes of a
>    fingerprint; this enables cutting and pasting the
>    fingerprint from a key listing output.
>
>Source: https://dev.gnupg.org/source/gnupg/browse/STABLE-BRANCH-2-2/doc/gpg-agent.texi;8021fe7670c79d5c698ec3fb600b02a9e5afb415$756?as=source&blame=off
>
>How did it fail for you?  It passes all the tests when I've
>run it against Fedora and RHEL-based hosts.  If it's flaky
>on other systems, that would put a damper on doing it this
>way.  Though it _should_ work.

Sorry for the delays, I'm a bit busy with other things at the moment. I did 
get an interactive popup asking if I would like to trust the key when I ran 
the t4202 test. This never happened with the old variant.

>
>[Note to myself] We don't just generate the key data,
>trustlist, etc. and store it in t/lib-gpg like we do with
>some other files per b41a36e635 (tests: create gpg homedir
>on the fly, 2014-12-12).  That was because the gnupg home
>directory layout changed a bit between 2.0 and 2.1.
>
>Thanks,
>
>-- 
>Todd

Re: [PATCH] gpg-interface: fix for gpgsm v2.3

[Todd Zullinger]

Fabian Stelzer wrote:
> On 07.02.2022 11:38, Todd Zullinger wrote:
>> How did it fail for you?  It passes all the tests when I've
>> run it against Fedora and RHEL-based hosts.  If it's flaky
>> on other systems, that would put a damper on doing it this
>> way.  Though it _should_ work.
> 
> Sorry for the delays, I'm a bit busy with other things at the moment.

No apologies needed.  This is something I worked on back in
November and had yet to send to the list, so I'm the last
person to rush another. :)

> I did get an interactive popup asking if I would like to
> trust the key when I ran the t4202 test. This never
> happened with the old variant.

Interesting.  I do have a patch in my gnupg-2.3 series to
reload the gpg agent after changing the trustlist, as the
changes were not picked up prior to that.  In my case, I was
running the tests in an environment where gpg could not
prompt me.  (It also seems like we should try harder to have
the test suite reject such prompts).

--- 8< ---
Subject: [PATCH] t/lib-gpg: reload gpg components after updating trustlist

With gpgsm from gnupg-2.3, the changes to the trustlist.txt do not
appear to be picked up without refreshing the gpg-agent.  Use the 'all'
keyword to reload all of the gpg components.  The scdaemon is started as
a child of gpg-agent, for example.

We used to have a --kill at this spot, but I removed it in 2e285e7803
(t/lib-gpg: drop redundant killing of gpg-agent, 2019-02-07).  It seems
like it might be necessary (again) for 2.3.

Signed-off-by: Todd Zullinger <tmz@pobox.com>
---

Notes:
    An alternative to doing this dance with the trustlist.txt and having to
    kill and/or reload the gpg-agent to pick up the change right after the
    import in each test might be to make this part of the steps used when
    adding/updating/removing certificates in t/lib-gpg.
    
    If not as a one-time affair when a cert is added/update/removed, then
    perhaps as a step taken by/in t/lib-gpg.sh only once.  It could populate
    a gpghome to be copied into the trash dir for each test which used
    gpg/gpgsm.  I haven't measured the effect of the extra reload precisely,
    but I'm sure it's not free.
    
    (For what it's worth, it didn't add any noticeable amount of time to the
    full builds/test runs I made while working on this, so it's a seemingly
    small cost, at least.)
    
    Also, hello Henning,
    
    Way back, in February 2019¹, when I submitted 2e285e7803 to remove the
    "redundant" killing of the gpg-agent, you said:
    
    > Killing the agent once should be enough, i remember manually killing
    > it many times as i was looking for a way to generate certs and trust
    > (configure gpgsm for the test). That is probably why i copied it over
    > in the first place.
    
    As I wrote this patch to partially restore the gpg-agent killing (now
    just a reload), I thought this might have been the sort of issue that
    you hit while testing.
    
    It could be unrelated, but it sounds quite similar to what I found with
    gnupg-2.3 when trying to get it to pick up the trustlist.txt changes.  I
    thought you might at least enjoy seeing it come back around.  :)
    
    ¹ <20190208093324.7b17f270@md1za8fc.ad001.siemens.net>

 t/lib-gpg.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
index 6bc083ca77..38e2c0f4fb 100644
--- a/t/lib-gpg.sh
+++ b/t/lib-gpg.sh
@@ -75,6 +75,7 @@ test_lazy_prereq GPGSM '
 	gpgsm --homedir "${GNUPGHOME}" -K --with-colons |
 	awk -F ":" "/^fpr:/ {printf \"%s S relax\\n\", \$10}" \
 		>"${GNUPGHOME}/trustlist.txt" &&
+	(gpgconf --reload all || : ) &&
 
 	echo hello | gpgsm --homedir "${GNUPGHOME}" >/dev/null \
 	       -u committer@example.com -o /dev/null --sign -

--- 8< ---

I have another patch which changes the earlier gpgconf call
which kills gpg-agent to kill all gpg daemons, as there are
some others which could potentially interfere with the
tests:

--- 8< ---
Subject: [PATCH] t/lib-gpg: kill all gpg components, not just gpg-agent

The gpg-agent is one of several processes that newer releases of GnuPG
start automatically.  Issue a kill to each of them to ensure they do not
affect separate tests.  (Yes, the separate GNUPGHOME should do that
already. If we find that is case, we could drop the --kill entirely.)

In terms of compatibility, the 'all' keyword was added to the --kill &
--reload options in GnuPG 2.1.18.  Debian and RHEL are often used as
indicators of how a change might affect older systems we often try to
support.

    - Debian Strech (old old stable), which has limited security support
      until June 2022, has GnuPG 2.1.18 (or 2.2.x in backports).

    - CentOS/RHEL 7, which is supported until June 2024, has GnuPG
      2.0.22, which lacks the --kill option, so the change won't have
      any impact.

Signed-off-by: Todd Zullinger <tmz@pobox.com>
---
 t/lib-gpg.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
index d675698a2d..2bb309a8c1 100644
--- a/t/lib-gpg.sh
+++ b/t/lib-gpg.sh
@@ -40,7 +40,7 @@ test_lazy_prereq GPG '
 		#		> lib-gpg/ownertrust
 		mkdir "$GNUPGHOME" &&
 		chmod 0700 "$GNUPGHOME" &&
-		(gpgconf --kill gpg-agent || : ) &&
+		(gpgconf --kill all || : ) &&
 		gpg --homedir "${GNUPGHOME}" --import \
 			"$TEST_DIRECTORY"/lib-gpg/keyring.gpg &&
 		gpg --homedir "${GNUPGHOME}" --import-ownertrust \
--- 8< ---

I have the series in the gpg-misc-fixes branch:

    https://github.com/tmzullinger/git/commits/gpg-misc-fixes

(That series does differ in that it has `string_list_split()`
instead of the simpler `strbuf_addch(&gpg_status, '\n');` to
fix the gpgsm parsing.)

Iff you think it would be useful or helpful, I can post that
series.

Thanks,

-- 
Todd

Re: [PATCH] gpg-interface: fix for gpgsm v2.3

[Fabian Stelzer]

On 09.02.2022 11:20, Todd Zullinger wrote:
>Fabian Stelzer wrote:
>> On 07.02.2022 11:38, Todd Zullinger wrote:
>>> How did it fail for you?  It passes all the tests when I've
>>> run it against Fedora and RHEL-based hosts.  If it's flaky
>>> on other systems, that would put a damper on doing it this
>>> way.  Though it _should_ work.
>>
>> Sorry for the delays, I'm a bit busy with other things at the moment.
>
>No apologies needed.  This is something I worked on back in
>November and had yet to send to the list, so I'm the last
>person to rush another. :)
>
>> I did get an interactive popup asking if I would like to
>> trust the key when I ran the t4202 test. This never
>> happened with the old variant.
>
>Interesting.  I do have a patch in my gnupg-2.3 series to
>reload the gpg agent after changing the trustlist, as the
>changes were not picked up prior to that.  In my case, I was
>running the tests in an environment where gpg could not
>prompt me.  (It also seems like we should try harder to have
>the test suite reject such prompts).
>

Yes, gpg-agent in general can be problematic for the tests. I'm not familiar 
enough with gpg but I don't know if we can get by without it?

>--- 8< ---
>Subject: [PATCH] t/lib-gpg: reload gpg components after updating trustlist
>
>With gpgsm from gnupg-2.3, the changes to the trustlist.txt do not
>appear to be picked up without refreshing the gpg-agent.  Use the 'all'
>keyword to reload all of the gpg components.  The scdaemon is started as
>a child of gpg-agent, for example.
>
>We used to have a --kill at this spot, but I removed it in 2e285e7803
>(t/lib-gpg: drop redundant killing of gpg-agent, 2019-02-07).  It seems
>like it might be necessary (again) for 2.3.
>
>Signed-off-by: Todd Zullinger <tmz@pobox.com>
>---
>
>Notes:
>    An alternative to doing this dance with the trustlist.txt and having to
>    kill and/or reload the gpg-agent to pick up the change right after the
>    import in each test might be to make this part of the steps used when
>    adding/updating/removing certificates in t/lib-gpg.
>
>    If not as a one-time affair when a cert is added/update/removed, then
>    perhaps as a step taken by/in t/lib-gpg.sh only once.  It could populate
>    a gpghome to be copied into the trash dir for each test which used
>    gpg/gpgsm.  I haven't measured the effect of the extra reload precisely,
>    but I'm sure it's not free.
>
>    (For what it's worth, it didn't add any noticeable amount of time to the
>    full builds/test runs I made while working on this, so it's a seemingly
>    small cost, at least.)
>
>    Also, hello Henning,
>
>    Way back, in February 2019¹, when I submitted 2e285e7803 to remove the
>    "redundant" killing of the gpg-agent, you said:
>
>    > Killing the agent once should be enough, i remember manually killing
>    > it many times as i was looking for a way to generate certs and trust
>    > (configure gpgsm for the test). That is probably why i copied it over
>    > in the first place.
>
>    As I wrote this patch to partially restore the gpg-agent killing (now
>    just a reload), I thought this might have been the sort of issue that
>    you hit while testing.
>
>    It could be unrelated, but it sounds quite similar to what I found with
>    gnupg-2.3 when trying to get it to pick up the trustlist.txt changes.  I
>    thought you might at least enjoy seeing it come back around.  :)
>
>    ¹ <20190208093324.7b17f270@md1za8fc.ad001.siemens.net>
>
> t/lib-gpg.sh | 1 +
> 1 file changed, 1 insertion(+)
>
>diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
>index 6bc083ca77..38e2c0f4fb 100644
>--- a/t/lib-gpg.sh
>+++ b/t/lib-gpg.sh
>@@ -75,6 +75,7 @@ test_lazy_prereq GPGSM '
> 	gpgsm --homedir "${GNUPGHOME}" -K --with-colons |
> 	awk -F ":" "/^fpr:/ {printf \"%s S relax\\n\", \$10}" \
> 		>"${GNUPGHOME}/trustlist.txt" &&
>+	(gpgconf --reload all || : ) &&
>
> 	echo hello | gpgsm --homedir "${GNUPGHOME}" >/dev/null \
> 	       -u committer@example.com -o /dev/null --sign -
>
>--- 8< ---

This patch fixes it for me.

>
>I have another patch which changes the earlier gpgconf call
>which kills gpg-agent to kill all gpg daemons, as there are
>some others which could potentially interfere with the
>tests:
>
>--- 8< ---
>Subject: [PATCH] t/lib-gpg: kill all gpg components, not just gpg-agent
>
>The gpg-agent is one of several processes that newer releases of GnuPG
>start automatically.  Issue a kill to each of them to ensure they do not
>affect separate tests.  (Yes, the separate GNUPGHOME should do that
>already. If we find that is case, we could drop the --kill entirely.)
>
>In terms of compatibility, the 'all' keyword was added to the --kill &
>--reload options in GnuPG 2.1.18.  Debian and RHEL are often used as
>indicators of how a change might affect older systems we often try to
>support.
>
>    - Debian Strech (old old stable), which has limited security support
>      until June 2022, has GnuPG 2.1.18 (or 2.2.x in backports).
>
>    - CentOS/RHEL 7, which is supported until June 2024, has GnuPG
>      2.0.22, which lacks the --kill option, so the change won't have
>      any impact.
>
>Signed-off-by: Todd Zullinger <tmz@pobox.com>
>---
> t/lib-gpg.sh | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
>diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
>index d675698a2d..2bb309a8c1 100644
>--- a/t/lib-gpg.sh
>+++ b/t/lib-gpg.sh
>@@ -40,7 +40,7 @@ test_lazy_prereq GPG '
> 		#		> lib-gpg/ownertrust
> 		mkdir "$GNUPGHOME" &&
> 		chmod 0700 "$GNUPGHOME" &&
>-		(gpgconf --kill gpg-agent || : ) &&
>+		(gpgconf --kill all || : ) &&
> 		gpg --homedir "${GNUPGHOME}" --import \
> 			"$TEST_DIRECTORY"/lib-gpg/keyring.gpg &&
> 		gpg --homedir "${GNUPGHOME}" --import-ownertrust \
>--- 8< ---
>
>I have the series in the gpg-misc-fixes branch:
>
>    https://github.com/tmzullinger/git/commits/gpg-misc-fixes
>
>(That series does differ in that it has `string_list_split()`
>instead of the simpler `strbuf_addch(&gpg_status, '\n');` to
>fix the gpgsm parsing.)
>
>Iff you think it would be useful or helpful, I can post that
>series.

I have prepared the patch with the simple strstr() matching I can post in a 
bit. I would add your two gpg test lib patches to it if thats ok?

Thanks

>
>Thanks,
>
>-- 
>Todd

Re: [PATCH] gpg-interface: fix for gpgsm v2.3

[Todd Zullinger]

Fabian Stelzer wrote:
> On 09.02.2022 11:20, Todd Zullinger wrote:
>> Interesting.  I do have a patch in my gnupg-2.3 series to
>> reload the gpg agent after changing the trustlist, as the
>> changes were not picked up prior to that.  In my case, I was
>> running the tests in an environment where gpg could not
>> prompt me.  (It also seems like we should try harder to have
>> the test suite reject such prompts).
>> 
> 
> Yes, gpg-agent in general can be problematic for the tests. I'm not familiar
> enough with gpg but I don't know if we can get by without it?

With modern gnupg, the secret keyring access is handled by
gpg-agent.  So it's no longer optional, which is mildly
unfortunate for automated tests..

>> diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
>> index 6bc083ca77..38e2c0f4fb 100644
>> --- a/t/lib-gpg.sh
>> +++ b/t/lib-gpg.sh
>> @@ -75,6 +75,7 @@ test_lazy_prereq GPGSM '
>> 	gpgsm --homedir "${GNUPGHOME}" -K --with-colons |
>> 	awk -F ":" "/^fpr:/ {printf \"%s S relax\\n\", \$10}" \
>> 		>"${GNUPGHOME}/trustlist.txt" &&
>> +	(gpgconf --reload all || : ) &&
>> 
>> 	echo hello | gpgsm --homedir "${GNUPGHOME}" >/dev/null \
>> 	       -u committer@example.com -o /dev/null --sign -
>> 
>> --- 8< ---
> 
> This patch fixes it for me.

Excellent.

> I have prepared the patch with the simple strstr() matching I can post in a
> bit. I would add your two gpg test lib patches to it if thats ok?

Absolutely.  Thank you for working on this and pulling it
together.

Cheers,

-- 
Todd

[PATCH 1/3] gpg-interface/gpgsm: fix for v2.3

[Fabian Stelzer]

gpgsm v2.3 changed some details about its output:
 - instead of displaying `fingerprint:` for keys it will print `sha1
   fpr:` and `sha2 fpr:`
 - some wording of errors has changed
 - signing will omit an extra debug output line before the [GNUPG]: tag

This change adjusts the gpgsm test prerequisite to work with v2.3 as
well by accepting `sha1 fpr:` as well as `fingerprint:`. To make this
parsing more robust switch to gpg's `--with-colons` output format.
Also allow both variants of errors for unknown certs.
Checking if signing was successful will now accept '[GNUPG]:
SIG_CREATED' on any beginning of a line. Not just explictly the second
one anymore.

Helped-By: Junio C Hamano <gitster@pobox.com>
Helped-By: Todd Zullinger <tmz@pobox.com>
---
 gpg-interface.c | 9 ++++++++-
 t/lib-gpg.sh    | 8 +++-----
 t/t4202-log.sh  | 2 +-
 3 files changed, 12 insertions(+), 7 deletions(-)

diff --git a/gpg-interface.c b/gpg-interface.c
index 17b1e44baa..94abb3090b 100644
--- a/gpg-interface.c
+++ b/gpg-interface.c
@@ -934,6 +934,7 @@ static int sign_buffer_gpg(struct strbuf *buffer, struct strbuf *signature,
 	struct child_process gpg = CHILD_PROCESS_INIT;
 	int ret;
 	size_t bottom;
+	const char *cp;
 	struct strbuf gpg_status = STRBUF_INIT;
 
 	strvec_pushl(&gpg.args,
@@ -953,7 +954,13 @@ static int sign_buffer_gpg(struct strbuf *buffer, struct strbuf *signature,
 			   signature, 1024, &gpg_status, 0);
 	sigchain_pop(SIGPIPE);
 
-	ret |= !strstr(gpg_status.buf, "\n[GNUPG:] SIG_CREATED ");
+	for (cp = gpg_status.buf;
+	     cp && (cp = strstr(cp, "[GNUPG:] SIG_CREATED "));
+	     cp++) {
+		if (cp == gpg_status.buf || cp[-1] == '\n')
+			break; /* found */
+	}
+	ret |= !cp;
 	strbuf_release(&gpg_status);
 	if (ret)
 		return error(_("gpg failed to sign the data"));
diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
index 3e7ee1386a..e997ce10ea 100644
--- a/t/lib-gpg.sh
+++ b/t/lib-gpg.sh
@@ -72,12 +72,10 @@ test_lazy_prereq GPGSM '
 		--passphrase-fd 0 --pinentry-mode loopback \
 		--import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 &&
 
-	gpgsm --homedir "${GNUPGHOME}" -K |
-	grep fingerprint: |
-	cut -d" " -f4 |
-	tr -d "\\n" >"${GNUPGHOME}/trustlist.txt" &&
+	gpgsm --homedir "${GNUPGHOME}" -K --with-colons |
+	awk -F ":" "/^(fpr|fingerprint):/ {printf \"%s S relax\\n\", \$10}" \
+		>"${GNUPGHOME}/trustlist.txt" &&
 
-	echo " S relax" >>"${GNUPGHOME}/trustlist.txt" &&
 	echo hello | gpgsm --homedir "${GNUPGHOME}" >/dev/null \
 	       -u committer@example.com -o /dev/null --sign -
 '
diff --git a/t/t4202-log.sh b/t/t4202-log.sh
index 544f0aa82e..493e376e73 100755
--- a/t/t4202-log.sh
+++ b/t/t4202-log.sh
@@ -2013,7 +2013,7 @@ test_expect_success GPGSM 'log --graph --show-signature for merged tag x509 miss
 	git merge --no-ff -m msg signed_tag_x509_nokey &&
 	GNUPGHOME=. git log --graph --show-signature -n1 plain-x509-nokey >actual &&
 	grep "^|\\\  merged tag" actual &&
-	grep "^| | gpgsm: certificate not found" actual
+	grep -Ei "^| | gpgsm:( failed to find the)? certificate:? not found" actual
 '
 
 test_expect_success GPGSM 'log --graph --show-signature for merged tag x509 bad signature' '
-- 
2.35.1

[PATCH 2/3] t/lib-gpg: reload gpg components after updating trustlist

[Fabian Stelzer]

From: Todd Zullinger <tmz@pobox.com>

With gpgsm from gnupg-2.3, the changes to the trustlist.txt do not
appear to be picked up without refreshing the gpg-agent.  Use the 'all'
keyword to reload all of the gpg components.  The scdaemon is started as
a child of gpg-agent, for example.

We used to have a --kill at this spot, but I removed it in 2e285e7803
(t/lib-gpg: drop redundant killing of gpg-agent, 2019-02-07).  It seems
like it might be necessary (again) for 2.3.

Signed-off-by: Todd Zullinger <tmz@pobox.com>
---
 t/lib-gpg.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
index e997ce10ea..2bad35e61a 100644
--- a/t/lib-gpg.sh
+++ b/t/lib-gpg.sh
@@ -75,6 +75,7 @@ test_lazy_prereq GPGSM '
 	gpgsm --homedir "${GNUPGHOME}" -K --with-colons |
 	awk -F ":" "/^(fpr|fingerprint):/ {printf \"%s S relax\\n\", \$10}" \
 		>"${GNUPGHOME}/trustlist.txt" &&
+	(gpgconf --reload all || : ) &&
 
 	echo hello | gpgsm --homedir "${GNUPGHOME}" >/dev/null \
 	       -u committer@example.com -o /dev/null --sign -
-- 
2.35.1

[PATCH 3/3] t/lib-gpg: kill all gpg components, not just gpg-agent

[Fabian Stelzer]

From: Todd Zullinger <tmz@pobox.com>

The gpg-agent is one of several processes that newer releases of GnuPG
start automatically.  Issue a kill to each of them to ensure they do not
affect separate tests.  (Yes, the separate GNUPGHOME should do that
already. If we find that is case, we could drop the --kill entirely.)

In terms of compatibility, the 'all' keyword was added to the --kill &
--reload options in GnuPG 2.1.18.  Debian and RHEL are often used as
indicators of how a change might affect older systems we often try to
support.

    - Debian Strech (old old stable), which has limited security support
      until June 2022, has GnuPG 2.1.18 (or 2.2.x in backports).

    - CentOS/RHEL 7, which is supported until June 2024, has GnuPG
      2.0.22, which lacks the --kill option, so the change won't have
      any impact.

Signed-off-by: Todd Zullinger <tmz@pobox.com>
---
 t/lib-gpg.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
index 2bad35e61a..8b9fb6e932 100644
--- a/t/lib-gpg.sh
+++ b/t/lib-gpg.sh
@@ -40,7 +40,7 @@ test_lazy_prereq GPG '
 		#		> lib-gpg/ownertrust
 		mkdir "$GNUPGHOME" &&
 		chmod 0700 "$GNUPGHOME" &&
-		(gpgconf --kill gpg-agent || : ) &&
+		(gpgconf --kill all || : ) &&
 		gpg --homedir "${GNUPGHOME}" --import \
 			"$TEST_DIRECTORY"/lib-gpg/keyring.gpg &&
 		gpg --homedir "${GNUPGHOME}" --import-ownertrust \
-- 
2.35.1

Re: [PATCH 1/3] gpg-interface/gpgsm: fix for v2.3

[Todd Zullinger]

Hi,

Fabian Stelzer wrote:
> gpgsm v2.3 changed some details about its output:
>  - instead of displaying `fingerprint:` for keys it will print `sha1
>    fpr:` and `sha2 fpr:`
>  - some wording of errors has changed
>  - signing will omit an extra debug output line before the [GNUPG]: tag
> 
> This change adjusts the gpgsm test prerequisite to work with v2.3 as
> well by accepting `sha1 fpr:` as well as `fingerprint:`. To make this
> parsing more robust switch to gpg's `--with-colons` output format.
> Also allow both variants of errors for unknown certs.

I ran this series through the fedora buildsystem on releases
with gnupg 2.2 and 2.3.  All the tests pass, as expected.

I think we may be able to simplify the wording above and the
patch below regarding the fingerprint/shaN fpr output
change, I'll add a comment below the changed hunk.

> diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
> index 3e7ee1386a..e997ce10ea 100644
> --- a/t/lib-gpg.sh
> +++ b/t/lib-gpg.sh
> @@ -72,12 +72,10 @@ test_lazy_prereq GPGSM '
>  		--passphrase-fd 0 --pinentry-mode loopback \
>  		--import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 &&
>  
> -	gpgsm --homedir "${GNUPGHOME}" -K |
> -	grep fingerprint: |
> -	cut -d" " -f4 |
> -	tr -d "\\n" >"${GNUPGHOME}/trustlist.txt" &&
> +	gpgsm --homedir "${GNUPGHOME}" -K --with-colons |
> +	awk -F ":" "/^(fpr|fingerprint):/ {printf \"%s S relax\\n\", \$10}" \
> +		>"${GNUPGHOME}/trustlist.txt" &&

Using --with-colons to parse the output, we shouldn't be
affected by the changed output.  The pattern for awk can be
simplified to '^fpr:' as older and newer versions of gnupg
have used that string in the --with-colons output for many,
many years.

Perhaps that allows the commit message to say less about the
specific's the gnugp-2.3 output change and just mention that
it changed and using --with-colons is the preferred way to
parse the output (where we must parse output at all).

    Switch to gpg's `--with-colons` output format to make
    parsing more robust.  This avoids issues where the
    human-readable output from gpg commands changes.

or something?

Thanks,

-- 
Todd

[PATCH v3 1/3] gpg-interface/gpgsm: fix for v2.3

[Fabian Stelzer]

Checking if signing was successful will now accept '[GNUPG]:
SIG_CREATED' on any beginning of a line. Not just explictly the second
one anymore.

Switch to gpg's `--with-colons` output format to make
parsing more robust.  This avoids issues where the
human-readable output from gpg commands changes.

Adjust error messages checking in tests for v2.3 specific output changes.

Helped-By: Junio C Hamano <gitster@pobox.com>
Helped-By: Todd Zullinger <tmz@pobox.com>
---
 gpg-interface.c | 9 ++++++++-
 t/lib-gpg.sh    | 8 +++-----
 t/t4202-log.sh  | 2 +-
 3 files changed, 12 insertions(+), 7 deletions(-)

diff --git a/gpg-interface.c b/gpg-interface.c
index aa50224e67..280f1fa1a5 100644
--- a/gpg-interface.c
+++ b/gpg-interface.c
@@ -934,6 +934,7 @@ static int sign_buffer_gpg(struct strbuf *buffer, struct strbuf *signature,
 	struct child_process gpg = CHILD_PROCESS_INIT;
 	int ret;
 	size_t bottom;
+	const char *cp;
 	struct strbuf gpg_status = STRBUF_INIT;
 
 	strvec_pushl(&gpg.args,
@@ -953,7 +954,13 @@ static int sign_buffer_gpg(struct strbuf *buffer, struct strbuf *signature,
 			   signature, 1024, &gpg_status, 0);
 	sigchain_pop(SIGPIPE);
 
-	ret |= !strstr(gpg_status.buf, "\n[GNUPG:] SIG_CREATED ");
+	for (cp = gpg_status.buf;
+	     cp && (cp = strstr(cp, "[GNUPG:] SIG_CREATED "));
+	     cp++) {
+		if (cp == gpg_status.buf || cp[-1] == '\n')
+			break; /* found */
+	}
+	ret |= !cp;
 	strbuf_release(&gpg_status);
 	if (ret)
 		return error(_("gpg failed to sign the data"));
diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
index 3e7ee1386a..6bc083ca77 100644
--- a/t/lib-gpg.sh
+++ b/t/lib-gpg.sh
@@ -72,12 +72,10 @@ test_lazy_prereq GPGSM '
 		--passphrase-fd 0 --pinentry-mode loopback \
 		--import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 &&
 
-	gpgsm --homedir "${GNUPGHOME}" -K |
-	grep fingerprint: |
-	cut -d" " -f4 |
-	tr -d "\\n" >"${GNUPGHOME}/trustlist.txt" &&
+	gpgsm --homedir "${GNUPGHOME}" -K --with-colons |
+	awk -F ":" "/^fpr:/ {printf \"%s S relax\\n\", \$10}" \
+		>"${GNUPGHOME}/trustlist.txt" &&
 
-	echo " S relax" >>"${GNUPGHOME}/trustlist.txt" &&
 	echo hello | gpgsm --homedir "${GNUPGHOME}" >/dev/null \
 	       -u committer@example.com -o /dev/null --sign -
 '
diff --git a/t/t4202-log.sh b/t/t4202-log.sh
index 55fac64446..d599bf4b11 100755
--- a/t/t4202-log.sh
+++ b/t/t4202-log.sh
@@ -2037,7 +2037,7 @@ test_expect_success GPGSM 'log --graph --show-signature for merged tag x509 miss
 	git merge --no-ff -m msg signed_tag_x509_nokey &&
 	GNUPGHOME=. git log --graph --show-signature -n1 plain-x509-nokey >actual &&
 	grep "^|\\\  merged tag" actual &&
-	grep "^| | gpgsm: certificate not found" actual
+	grep -Ei "^| | gpgsm:( failed to find the)? certificate:? not found" actual
 '
 
 test_expect_success GPGSM 'log --graph --show-signature for merged tag x509 bad signature' '
-- 
2.35.1

[PATCH v3 2/3] t/lib-gpg: reload gpg components after updating trustlist

[Fabian Stelzer]

From: Todd Zullinger <tmz@pobox.com>

With gpgsm from gnupg-2.3, the changes to the trustlist.txt do not
appear to be picked up without refreshing the gpg-agent.  Use the 'all'
keyword to reload all of the gpg components.  The scdaemon is started as
a child of gpg-agent, for example.

We used to have a --kill at this spot, but I removed it in 2e285e7803
(t/lib-gpg: drop redundant killing of gpg-agent, 2019-02-07).  It seems
like it might be necessary (again) for 2.3.

Signed-off-by: Todd Zullinger <tmz@pobox.com>
---
 t/lib-gpg.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
index 6bc083ca77..38e2c0f4fb 100644
--- a/t/lib-gpg.sh
+++ b/t/lib-gpg.sh
@@ -75,6 +75,7 @@ test_lazy_prereq GPGSM '
 	gpgsm --homedir "${GNUPGHOME}" -K --with-colons |
 	awk -F ":" "/^fpr:/ {printf \"%s S relax\\n\", \$10}" \
 		>"${GNUPGHOME}/trustlist.txt" &&
+	(gpgconf --reload all || : ) &&
 
 	echo hello | gpgsm --homedir "${GNUPGHOME}" >/dev/null \
 	       -u committer@example.com -o /dev/null --sign -
-- 
2.35.1

[PATCH v3 3/3] t/lib-gpg: kill all gpg components, not just gpg-agent

[Fabian Stelzer]

From: Todd Zullinger <tmz@pobox.com>

The gpg-agent is one of several processes that newer releases of GnuPG
start automatically.  Issue a kill to each of them to ensure they do not
affect separate tests.  (Yes, the separate GNUPGHOME should do that
already. If we find that is case, we could drop the --kill entirely.)

In terms of compatibility, the 'all' keyword was added to the --kill &
--reload options in GnuPG 2.1.18.  Debian and RHEL are often used as
indicators of how a change might affect older systems we often try to
support.

    - Debian Strech (old old stable), which has limited security support
      until June 2022, has GnuPG 2.1.18 (or 2.2.x in backports).

    - CentOS/RHEL 7, which is supported until June 2024, has GnuPG
      2.0.22, which lacks the --kill option, so the change won't have
      any impact.

Signed-off-by: Todd Zullinger <tmz@pobox.com>
---
 t/lib-gpg.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
index 38e2c0f4fb..114785586a 100644
--- a/t/lib-gpg.sh
+++ b/t/lib-gpg.sh
@@ -40,7 +40,7 @@ test_lazy_prereq GPG '
 		#		> lib-gpg/ownertrust
 		mkdir "$GNUPGHOME" &&
 		chmod 0700 "$GNUPGHOME" &&
-		(gpgconf --kill gpg-agent || : ) &&
+		(gpgconf --kill all || : ) &&
 		gpg --homedir "${GNUPGHOME}" --import \
 			"$TEST_DIRECTORY"/lib-gpg/keyring.gpg &&
 		gpg --homedir "${GNUPGHOME}" --import-ownertrust \
-- 
2.35.1

Re: [PATCH v3 1/3] gpg-interface/gpgsm: fix for v2.3

[Junio C Hamano]

Fabian Stelzer <fs@gigacodes.de> writes:

> Checking if signing was successful will now accept '[GNUPG]:
> SIG_CREATED' on any beginning of a line. Not just explictly the second
> one anymore.

"the second or subsequent one", I would think, but the code change
looks correct anyway.

> Switch to gpg's `--with-colons` output format to make
> parsing more robust.  This avoids issues where the
> human-readable output from gpg commands changes.

Does this refer only to how parsing in tests is done?

> Adjust error messages checking in tests for v2.3 specific output changes.

Does this refer only to the change to 4202 where "failed to find
the" and the colon after "certificate" are made optional, so that
the regexp can read messages from both pre- and post-2.3 versions?

> diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
> index 3e7ee1386a..6bc083ca77 100644
> --- a/t/lib-gpg.sh
> +++ b/t/lib-gpg.sh
> @@ -72,12 +72,10 @@ test_lazy_prereq GPGSM '
>  		--passphrase-fd 0 --pinentry-mode loopback \
>  		--import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 &&
>  
> -	gpgsm --homedir "${GNUPGHOME}" -K |
> -	grep fingerprint: |
> -	cut -d" " -f4 |
> -	tr -d "\\n" >"${GNUPGHOME}/trustlist.txt" &&
> +	gpgsm --homedir "${GNUPGHOME}" -K --with-colons |
> +	awk -F ":" "/^fpr:/ {printf \"%s S relax\\n\", \$10}" \
> +		>"${GNUPGHOME}/trustlist.txt" &&

The old iteration had (fpr|fingerprint) which appeared as if it were
catering to both pre- and post-2.3 versions, but "with colons", all
versions we care about would say "fpr" and that is the reason why we
no longer have such an alternative here?  Just checking my
understanding.

> -	echo " S relax" >>"${GNUPGHOME}/trustlist.txt" &&

This removal is because...?  I do not recall seeing the explanation
in the proposed log message.

>  	echo hello | gpgsm --homedir "${GNUPGHOME}" >/dev/null \
>  	       -u committer@example.com -o /dev/null --sign -
>  '
> diff --git a/t/t4202-log.sh b/t/t4202-log.sh
> index 55fac64446..d599bf4b11 100755
> --- a/t/t4202-log.sh
> +++ b/t/t4202-log.sh
> @@ -2037,7 +2037,7 @@ test_expect_success GPGSM 'log --graph --show-signature for merged tag x509 miss
>  	git merge --no-ff -m msg signed_tag_x509_nokey &&
>  	GNUPGHOME=. git log --graph --show-signature -n1 plain-x509-nokey >actual &&
>  	grep "^|\\\  merged tag" actual &&
> -	grep "^| | gpgsm: certificate not found" actual
> +	grep -Ei "^| | gpgsm:( failed to find the)? certificate:? not found" actual
>  '

OK.  It might be easier to read if we give two expressions
separately and say "we can take either of these", i.e.

	# the former is from pre-2.3, the latter is from 2.3 and later
	grep -e "^| | gpgsm: certificate not found" \
	     -e "^| | gpgsm: failed to find the certificate: not found" \
	     actual

Thanks for working on this update.

Re: [PATCH v3 1/3] gpg-interface/gpgsm: fix for v2.3

[Fabian Stelzer]

On 02.03.2022 11:18, Junio C Hamano wrote:
>Fabian Stelzer <fs@gigacodes.de> writes:
>
>> Checking if signing was successful will now accept '[GNUPG]:
>> SIG_CREATED' on any beginning of a line. Not just explictly the second
>> one anymore.
>
>"the second or subsequent one", I would think, but the code change
>looks correct anyway.
>
>> Switch to gpg's `--with-colons` output format to make
>> parsing more robust.  This avoids issues where the
>> human-readable output from gpg commands changes.
>
>Does this refer only to how parsing in tests is done?

If only refers to the test prerequisite actually. I'll update the message.

>
>> Adjust error messages checking in tests for v2.3 specific output changes.
>
>Does this refer only to the change to 4202 where "failed to find
>the" and the colon after "certificate" are made optional, so that
>the regexp can read messages from both pre- and post-2.3 versions?
>
>> diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
>> index 3e7ee1386a..6bc083ca77 100644
>> --- a/t/lib-gpg.sh
>> +++ b/t/lib-gpg.sh
>> @@ -72,12 +72,10 @@ test_lazy_prereq GPGSM '
>>  		--passphrase-fd 0 --pinentry-mode loopback \
>>  		--import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 &&
>>
>> -	gpgsm --homedir "${GNUPGHOME}" -K |
>> -	grep fingerprint: |
>> -	cut -d" " -f4 |
>> -	tr -d "\\n" >"${GNUPGHOME}/trustlist.txt" &&
>> +	gpgsm --homedir "${GNUPGHOME}" -K --with-colons |
>> +	awk -F ":" "/^fpr:/ {printf \"%s S relax\\n\", \$10}" \
>> +		>"${GNUPGHOME}/trustlist.txt" &&
>
>The old iteration had (fpr|fingerprint) which appeared as if it were
>catering to both pre- and post-2.3 versions, but "with colons", all
>versions we care about would say "fpr" and that is the reason why we
>no longer have such an alternative here?  Just checking my
>understanding.

Correct. The `with-colons` always uses fpr pre and post 2.3

>
>> -	echo " S relax" >>"${GNUPGHOME}/trustlist.txt" &&
>
>This removal is because...?  I do not recall seeing the explanation
>in the proposed log message.

Switching to awk allows us to integrate this trailing info into the awk 
expression itself making this extra echo unnecessary.

>
>>  	echo hello | gpgsm --homedir "${GNUPGHOME}" >/dev/null \
>>  	       -u committer@example.com -o /dev/null --sign -
>>  '
>> diff --git a/t/t4202-log.sh b/t/t4202-log.sh
>> index 55fac64446..d599bf4b11 100755
>> --- a/t/t4202-log.sh
>> +++ b/t/t4202-log.sh
>> @@ -2037,7 +2037,7 @@ test_expect_success GPGSM 'log --graph --show-signature for merged tag x509 miss
>>  	git merge --no-ff -m msg signed_tag_x509_nokey &&
>>  	GNUPGHOME=. git log --graph --show-signature -n1 plain-x509-nokey >actual &&
>>  	grep "^|\\\  merged tag" actual &&
>> -	grep "^| | gpgsm: certificate not found" actual
>> +	grep -Ei "^| | gpgsm:( failed to find the)? certificate:? not found" actual
>>  '
>
>OK.  It might be easier to read if we give two expressions
>separately and say "we can take either of these", i.e.
>
>	# the former is from pre-2.3, the latter is from 2.3 and later
>	grep -e "^| | gpgsm: certificate not found" \
>	     -e "^| | gpgsm: failed to find the certificate: not found" \
>	     actual
>
>Thanks for working on this update.

Easy enough. Initially I used a subshell and 2 grep calls but this is 
obviously easier. I prefer the static strings over the regex as well.

I'll send a new patch probably tomorrow and try to improve the commit 
message.

Thanks

[PATCH v4 1/3] gpg-interface/gpgsm: fix for v2.3

[Fabian Stelzer]

Checking if signing was successful will now accept '[GNUPG]:
SIG_CREATED' on the beginning of the first or any subsequent line. Not
just explictly the second one anymore.

Gpgsm v2.3 changed its output when listing keys from `fingerprint` to
`sha1/2 fpr`. This leads to the gpgsm tests silently not being executed
because of a failed prerequisite.
Switch to gpg's `--with-colons` output format when evaluating test
prerequisites to make parsing more robust. This also allows us to
combine the existing grep/cut/tr/echo pipe for writing the trustlist.txt
into a single awk expression.

Adjust error message checking in test for v2.3 specific output changes.

Helped-By: Junio C Hamano <gitster@pobox.com>
Helped-By: Todd Zullinger <tmz@pobox.com>
---
 gpg-interface.c | 9 ++++++++-
 t/lib-gpg.sh    | 8 +++-----
 t/t4202-log.sh  | 3 ++-
 3 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/gpg-interface.c b/gpg-interface.c
index aa50224e67..280f1fa1a5 100644
--- a/gpg-interface.c
+++ b/gpg-interface.c
@@ -934,6 +934,7 @@ static int sign_buffer_gpg(struct strbuf *buffer, struct strbuf *signature,
 	struct child_process gpg = CHILD_PROCESS_INIT;
 	int ret;
 	size_t bottom;
+	const char *cp;
 	struct strbuf gpg_status = STRBUF_INIT;
 
 	strvec_pushl(&gpg.args,
@@ -953,7 +954,13 @@ static int sign_buffer_gpg(struct strbuf *buffer, struct strbuf *signature,
 			   signature, 1024, &gpg_status, 0);
 	sigchain_pop(SIGPIPE);
 
-	ret |= !strstr(gpg_status.buf, "\n[GNUPG:] SIG_CREATED ");
+	for (cp = gpg_status.buf;
+	     cp && (cp = strstr(cp, "[GNUPG:] SIG_CREATED "));
+	     cp++) {
+		if (cp == gpg_status.buf || cp[-1] == '\n')
+			break; /* found */
+	}
+	ret |= !cp;
 	strbuf_release(&gpg_status);
 	if (ret)
 		return error(_("gpg failed to sign the data"));
diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
index 3e7ee1386a..6bc083ca77 100644
--- a/t/lib-gpg.sh
+++ b/t/lib-gpg.sh
@@ -72,12 +72,10 @@ test_lazy_prereq GPGSM '
 		--passphrase-fd 0 --pinentry-mode loopback \
 		--import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 &&
 
-	gpgsm --homedir "${GNUPGHOME}" -K |
-	grep fingerprint: |
-	cut -d" " -f4 |
-	tr -d "\\n" >"${GNUPGHOME}/trustlist.txt" &&
+	gpgsm --homedir "${GNUPGHOME}" -K --with-colons |
+	awk -F ":" "/^fpr:/ {printf \"%s S relax\\n\", \$10}" \
+		>"${GNUPGHOME}/trustlist.txt" &&
 
-	echo " S relax" >>"${GNUPGHOME}/trustlist.txt" &&
 	echo hello | gpgsm --homedir "${GNUPGHOME}" >/dev/null \
 	       -u committer@example.com -o /dev/null --sign -
 '
diff --git a/t/t4202-log.sh b/t/t4202-log.sh
index 55fac64446..6306e2cbe5 100755
--- a/t/t4202-log.sh
+++ b/t/t4202-log.sh
@@ -2037,7 +2037,8 @@ test_expect_success GPGSM 'log --graph --show-signature for merged tag x509 miss
 	git merge --no-ff -m msg signed_tag_x509_nokey &&
 	GNUPGHOME=. git log --graph --show-signature -n1 plain-x509-nokey >actual &&
 	grep "^|\\\  merged tag" actual &&
-	grep "^| | gpgsm: certificate not found" actual
+	grep -e "^| | gpgsm: certificate not found" \
+	     -e "^| | gpgsm: failed to find the certificate: Not found" actual
 '
 
 test_expect_success GPGSM 'log --graph --show-signature for merged tag x509 bad signature' '
-- 
2.35.1

[PATCH v4 2/3] t/lib-gpg: reload gpg components after updating trustlist

[Fabian Stelzer]

From: Todd Zullinger <tmz@pobox.com>

With gpgsm from gnupg-2.3, the changes to the trustlist.txt do not
appear to be picked up without refreshing the gpg-agent.  Use the 'all'
keyword to reload all of the gpg components.  The scdaemon is started as
a child of gpg-agent, for example.

We used to have a --kill at this spot, but I removed it in 2e285e7803
(t/lib-gpg: drop redundant killing of gpg-agent, 2019-02-07).  It seems
like it might be necessary (again) for 2.3.

Signed-off-by: Todd Zullinger <tmz@pobox.com>
---
 t/lib-gpg.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
index 6bc083ca77..38e2c0f4fb 100644
--- a/t/lib-gpg.sh
+++ b/t/lib-gpg.sh
@@ -75,6 +75,7 @@ test_lazy_prereq GPGSM '
 	gpgsm --homedir "${GNUPGHOME}" -K --with-colons |
 	awk -F ":" "/^fpr:/ {printf \"%s S relax\\n\", \$10}" \
 		>"${GNUPGHOME}/trustlist.txt" &&
+	(gpgconf --reload all || : ) &&
 
 	echo hello | gpgsm --homedir "${GNUPGHOME}" >/dev/null \
 	       -u committer@example.com -o /dev/null --sign -
-- 
2.35.1

[PATCH v4 3/3] t/lib-gpg: kill all gpg components, not just gpg-agent

[Fabian Stelzer]

From: Todd Zullinger <tmz@pobox.com>

The gpg-agent is one of several processes that newer releases of GnuPG
start automatically.  Issue a kill to each of them to ensure they do not
affect separate tests.  (Yes, the separate GNUPGHOME should do that
already. If we find that is case, we could drop the --kill entirely.)

In terms of compatibility, the 'all' keyword was added to the --kill &
--reload options in GnuPG 2.1.18.  Debian and RHEL are often used as
indicators of how a change might affect older systems we often try to
support.

    - Debian Strech (old old stable), which has limited security support
      until June 2022, has GnuPG 2.1.18 (or 2.2.x in backports).

    - CentOS/RHEL 7, which is supported until June 2024, has GnuPG
      2.0.22, which lacks the --kill option, so the change won't have
      any impact.

Signed-off-by: Todd Zullinger <tmz@pobox.com>
---
 t/lib-gpg.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh
index 38e2c0f4fb..114785586a 100644
--- a/t/lib-gpg.sh
+++ b/t/lib-gpg.sh
@@ -40,7 +40,7 @@ test_lazy_prereq GPG '
 		#		> lib-gpg/ownertrust
 		mkdir "$GNUPGHOME" &&
 		chmod 0700 "$GNUPGHOME" &&
-		(gpgconf --kill gpg-agent || : ) &&
+		(gpgconf --kill all || : ) &&
 		gpg --homedir "${GNUPGHOME}" --import \
 			"$TEST_DIRECTORY"/lib-gpg/keyring.gpg &&
 		gpg --homedir "${GNUPGHOME}" --import-ownertrust \
-- 
2.35.1