[PATCH 1/2] t7510-signed-commit: use 'test_must_fail'

[PATCH 1/2] t7510-signed-commit: use 'test_must_fail'

[SZEDER Gábor]

The two tests 'detect fudged signature' and 'detect fudged signature
with NUL' in 't7510-signed-commit.sh' check that 'git verify-commit'
errors out when encountering a forged commit, but they do so by
running

  ! git verify-commit ...

Use 'test_must_fail' instead, because that would catch potential
unexpected errors like a segfault as well.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
---
 t/t7510-signed-commit.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/t/t7510-signed-commit.sh b/t/t7510-signed-commit.sh
index 762135adea..663bf68def 100755
--- a/t/t7510-signed-commit.sh
+++ b/t/t7510-signed-commit.sh
@@ -145,7 +145,7 @@ test_expect_success GPG 'detect fudged signature' '
 
 	sed -e "s/seventh/7th forged/" raw >forged1 &&
 	git hash-object -w -t commit forged1 >forged1.commit &&
-	! git verify-commit $(cat forged1.commit) &&
+	test_must_fail git verify-commit $(cat forged1.commit) &&
 	git show --pretty=short --show-signature $(cat forged1.commit) >actual1 &&
 	grep "BAD signature from" actual1 &&
 	! grep "Good signature from" actual1
@@ -156,7 +156,7 @@ test_expect_success GPG 'detect fudged signature with NUL' '
 	cat raw >forged2 &&
 	echo Qwik | tr "Q" "\000" >>forged2 &&
 	git hash-object -w -t commit forged2 >forged2.commit &&
-	! git verify-commit $(cat forged2.commit) &&
+	test_must_fail git verify-commit $(cat forged2.commit) &&
 	git show --pretty=short --show-signature $(cat forged2.commit) >actual2 &&
 	grep "BAD signature from" actual2 &&
 	! grep "Good signature from" actual2
-- 
2.18.0.rc0.207.ga6211da864

[PATCH 2/2] tests: make forging GPG signed commits and tags more robust

[SZEDER Gábor]

A couple of test scripts create forged GPG signed commits or tags to
check that such forgery can't fool various git commands' signature
verification.  All but one of those test scripts are prone to
occasional failures because the forgery creates a bogus GPG signature,
and git commands error out with an unexpected error message, e.g.
"Commit deadbeef does not have a GPG signature" instead of "...  has a
bad GPG signature".

't5573-pull-verify-signatures.sh', 't7510-signed-commit.sh' and
't7612-merge-verify-signatures.sh' create forged signed commits like
this:

  git commit -S -m "bad on side" &&
  git cat-file commit side-bad >raw &&
  sed -e "s/bad/forged bad/" raw >forged &&
  git hash-object -w -t commit forged >forged.commit

On rare occasions the given pattern occurs not only in the commit
message but in the GPG signature as well, and after it's replaced in
the signature the resulting signature becomes invalid, GPG will report
CRC error and that it couldn't find any signature, which will then
ultimately cause the test failure.

Since in all three cases the pattern to be replaced during the forgery
is the first word of the commit message's subject line, and since the
GPG signature in the commit object is indented by a space, let's just
anchor those patterns to the beginning of the line to prevent this
issue.

The test script 't7030-verify-tag.sh' creates a forged signed tag
object in a similar way by replacing the pattern "seventh", but the
GPG signature in tag objects is not indented by a space, so the above
solution is not applicable in this case.  However, in the tag object
in question the pattern "seventh" occurs not only in the tag message
but in the 'tag' header as well.  To create a forged tag object it's
sufficient to replace only one of the two occurences, so modify the
sed script to limit the pattern to the 'tag' header (i.e. a line
beginning with "tag ", which, because of the space character, can
never occur in the base64-encoded GPG signature).

Note that the forgery in 't7004-tag.sh' is not affected by this issue:
while 't7004' does create a forged signed tag kind of the same way,
it replaces "signed-tag" in the tag object, which, because of the '-'
character, can never occur in the base64-encoded GPG signarute.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
---
 t/t5573-pull-verify-signatures.sh  | 2 +-
 t/t7030-verify-tag.sh              | 2 +-
 t/t7510-signed-commit.sh           | 3 +--
 t/t7612-merge-verify-signatures.sh | 2 +-
 4 files changed, 4 insertions(+), 5 deletions(-)

diff --git a/t/t5573-pull-verify-signatures.sh b/t/t5573-pull-verify-signatures.sh
index 9594e891f4..747775c147 100755
--- a/t/t5573-pull-verify-signatures.sh
+++ b/t/t5573-pull-verify-signatures.sh
@@ -29,7 +29,7 @@ test_expect_success GPG 'create repositories with signed commits' '
 		echo 4 >d && git add d &&
 		test_tick && git commit -S -m "bad" &&
 		git cat-file commit HEAD >raw &&
-		sed -e "s/bad/forged bad/" raw >forged &&
+		sed -e "s/^bad/forged bad/" raw >forged &&
 		git hash-object -w -t commit forged >forged.commit &&
 		git checkout $(cat forged.commit)
 	) &&
diff --git a/t/t7030-verify-tag.sh b/t/t7030-verify-tag.sh
index b4b49eeb08..291a1e2b07 100755
--- a/t/t7030-verify-tag.sh
+++ b/t/t7030-verify-tag.sh
@@ -74,7 +74,7 @@ test_expect_success GPG 'verify and show signatures' '
 
 test_expect_success GPG 'detect fudged signature' '
 	git cat-file tag seventh-signed >raw &&
-	sed -e "s/seventh/7th forged/" raw >forged1 &&
+	sed -e "/^tag / s/seventh/7th forged/" raw >forged1 &&
 	git hash-object -w -t tag forged1 >forged1.tag &&
 	test_must_fail git verify-tag $(cat forged1.tag) 2>actual1 &&
 	grep "BAD signature from" actual1 &&
diff --git a/t/t7510-signed-commit.sh b/t/t7510-signed-commit.sh
index 663bf68def..6e2015ed9a 100755
--- a/t/t7510-signed-commit.sh
+++ b/t/t7510-signed-commit.sh
@@ -142,8 +142,7 @@ test_expect_success GPG 'show signed commit with signature' '
 
 test_expect_success GPG 'detect fudged signature' '
 	git cat-file commit seventh-signed >raw &&
-
-	sed -e "s/seventh/7th forged/" raw >forged1 &&
+	sed -e "s/^seventh/7th forged/" raw >forged1 &&
 	git hash-object -w -t commit forged1 >forged1.commit &&
 	test_must_fail git verify-commit $(cat forged1.commit) &&
 	git show --pretty=short --show-signature $(cat forged1.commit) >actual1 &&
diff --git a/t/t7612-merge-verify-signatures.sh b/t/t7612-merge-verify-signatures.sh
index e797c74112..e2b1df817a 100755
--- a/t/t7612-merge-verify-signatures.sh
+++ b/t/t7612-merge-verify-signatures.sh
@@ -23,7 +23,7 @@ test_expect_success GPG 'create signed commits' '
 	echo 3 >bar && git add bar &&
 	test_tick && git commit -S -m "bad on side" &&
 	git cat-file commit side-bad >raw &&
-	sed -e "s/bad/forged bad/" raw >forged &&
+	sed -e "s/^bad/forged bad/" raw >forged &&
 	git hash-object -w -t commit forged >forged.commit &&
 	git checkout initial &&
 
-- 
2.18.0.rc0.207.ga6211da864

Re: [PATCH 2/2] tests: make forging GPG signed commits and tags more robust

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1914 bytes --]

On Mon, Jun 04, 2018 at 03:39:26PM +0200, SZEDER Gábor wrote:
> On rare occasions the given pattern occurs not only in the commit
> message but in the GPG signature as well, and after it's replaced in
> the signature the resulting signature becomes invalid, GPG will report
> CRC error and that it couldn't find any signature, which will then
> ultimately cause the test failure.

Ooh, I hadn't seen these failures, but thanks for tracking this down.

> Since in all three cases the pattern to be replaced during the forgery
> is the first word of the commit message's subject line, and since the
> GPG signature in the commit object is indented by a space, let's just
> anchor those patterns to the beginning of the line to prevent this
> issue.
> 
> The test script 't7030-verify-tag.sh' creates a forged signed tag
> object in a similar way by replacing the pattern "seventh", but the
> GPG signature in tag objects is not indented by a space, so the above
> solution is not applicable in this case.  However, in the tag object
> in question the pattern "seventh" occurs not only in the tag message
> but in the 'tag' header as well.  To create a forged tag object it's
> sufficient to replace only one of the two occurences, so modify the
> sed script to limit the pattern to the 'tag' header (i.e. a line
> beginning with "tag ", which, because of the space character, can
> never occur in the base64-encoded GPG signature).
> 
> Note that the forgery in 't7004-tag.sh' is not affected by this issue:
> while 't7004' does create a forged signed tag kind of the same way,
> it replaces "signed-tag" in the tag object, which, because of the '-'
> character, can never occur in the base64-encoded GPG signarute.

This seems sane and obviously correct, and the other patch looked good,
too.  Thanks.
-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 867 bytes --]