git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Jeff King <peff@peff.net>
To: John Szakmeister <john@szakmeister.net>
Cc: Dennis Kaarsemaker <dennis@kaarsemaker.net>, git@vger.kernel.org
Subject: [PATCH 6/6] fsck: detect trailing garbage in all object types
Date: Fri, 13 Jan 2017 13:00:25 -0500	[thread overview]
Message-ID: <20170113180025.xkwyc6mzxl572jn7@sigill.intra.peff.net> (raw)
In-Reply-To: <20170113175258.e66taigy4wpokohk@sigill.intra.peff.net>

When a loose tree or commit is read by fsck (or any git
program), unpack_sha1_rest() checks whether there is extra
cruft at the end of the object file, after the zlib data.
Blobs that are streamed, however, do not have this check.

For normal git operations, it's not a big deal. We know the
sha1 and size checked out, so we have the object bytes we
wanted.  The trailing garbage doesn't affect what we're
trying to do.

But since the point of fsck is to find corruption or other
problems, it should be more thorough. This patch teaches its
loose-sha1 reader to detect extra bytes after the zlib
stream and complain.

Signed-off-by: Jeff King <peff@peff.net>
---
 sha1_file.c     |  5 +++++
 t/t1450-fsck.sh | 22 ++++++++++++++++++++++
 2 files changed, 27 insertions(+)

diff --git a/sha1_file.c b/sha1_file.c
index c0fccb73c..b77ab6d5c 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -3864,6 +3864,11 @@ static int check_stream_sha1(git_zstream *stream,
 		error("corrupt loose object '%s'", sha1_to_hex(expected_sha1));
 		return -1;
 	}
+	if (stream->avail_in) {
+		error("garbage at end of loose object '%s'",
+		      sha1_to_hex(expected_sha1));
+		return -1;
+	}
 
 	git_SHA1_Final(real_sha1, &c);
 	if (hashcmp(expected_sha1, real_sha1)) {
diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index 455c186fe..8975b4d1b 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -597,4 +597,26 @@ test_expect_success 'fsck finds problems in duplicate loose objects' '
 	)
 '
 
+test_expect_success 'fsck detects trailing loose garbage (commit)' '
+	git cat-file commit HEAD >basis &&
+	echo bump-commit-sha1 >>basis &&
+	commit=$(git hash-object -w -t commit basis) &&
+	file=$(sha1_file $commit) &&
+	test_when_finished "remove_object $commit" &&
+	chmod +w "$file" &&
+	echo garbage >>"$file" &&
+	test_must_fail git fsck 2>out &&
+	test_i18ngrep "garbage.*$commit" out
+'
+
+test_expect_success 'fsck detects trailing loose garbage (blob)' '
+	blob=$(echo trailing | git hash-object -w --stdin) &&
+	file=$(sha1_file $blob) &&
+	test_when_finished "remove_object $blob" &&
+	chmod +w "$file" &&
+	echo garbage >>"$file" &&
+	test_must_fail git fsck 2>out &&
+	test_i18ngrep "garbage.*$blob" out
+'
+
 test_done
-- 
2.11.0.629.g10075098c

  parent reply	other threads:[~2017-01-13 18:00 UTC|newest]

Thread overview: 39+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-01-07 12:50 "git fsck" not detecting garbage at the end of blob object files John Szakmeister
2017-01-07 21:47 ` Dennis Kaarsemaker
2017-01-08  5:26   ` Jeff King
2017-01-13  9:15     ` John Szakmeister
2017-01-13 17:52       ` [PATCH 0/6] loose-object fsck fixes/tightening Jeff King
2017-01-13 17:54         ` [PATCH 1/6] t1450: refactor loose-object removal Jeff King
2017-01-13 17:54         ` [PATCH 2/6] sha1_file: fix error message for alternate objects Jeff King
2017-01-13 17:55         ` [PATCH 3/6] t1450: test fsck of packed objects Jeff King
2017-01-13 17:58         ` [PATCH 4/6] sha1_file: add read_loose_object() function Jeff King
2017-01-13 17:59         ` [PATCH 5/6] fsck: parse loose object paths directly Jeff King
2018-10-30 20:03           ` Infinite loop regression in git-fsck in v2.12.0 Ævar Arnfjörð Bjarmason
2018-10-30 21:35             ` Jeff King
2018-10-30 22:28               ` Junio C Hamano
2018-10-30 22:56                 ` Jeff King
2018-10-30 23:12                   ` Jeff King
2018-10-30 23:18                     ` [PATCH 1/3] t1450: check large blob in trailing-garbage test Jeff King
2018-10-30 23:23                     ` [PATCH 2/3] check_stream_sha1(): handle input underflow Jeff King
2018-10-31  4:23                       ` Junio C Hamano
2018-10-31  4:30                         ` Jeff King
2018-10-31  4:44                           ` Junio C Hamano
2018-10-31  5:03                             ` Jeff King
2018-10-31  5:13                               ` Jeff King
2018-10-31  5:31                                 ` Junio C Hamano
2018-10-30 23:23                     ` [PATCH 3/3] cat-file: handle streaming failures consistently Jeff King
2018-10-31 12:42                       ` [PATCH 0/3] Add a GIT_TEST_FSCK test mode Ævar Arnfjörð Bjarmason
2018-10-31 12:42                       ` [PATCH 1/3] tests: add a "env-bool" helper to test-tool Ævar Arnfjörð Bjarmason
2018-10-31 12:42                       ` [PATCH 2/3] tests: mark those tests where "git fsck" fails at the end Ævar Arnfjörð Bjarmason
2018-11-01  3:37                         ` Junio C Hamano
2018-10-31 12:42                       ` [PATCH 3/3] tests: add a special test setup that runs "git fsck" before exiting Ævar Arnfjörð Bjarmason
2018-10-31 13:33                       ` [PATCH 3/3] cat-file: handle streaming failures consistently Torsten Bögershausen
2018-10-31 14:23                         ` Junio C Hamano
2018-10-31 14:37                           ` Jeff King
2018-10-31 17:38                       ` Eric Sunshine
2018-10-31 20:29                         ` Jeff King
2018-10-30 21:56             ` Infinite loop regression in git-fsck in v2.12.0 Ævar Arnfjörð Bjarmason
2018-10-30 23:08               ` Jeff King
2017-01-13 18:00         ` Jeff King [this message]
2017-01-19 11:18         ` [PATCH 0/6] loose-object fsck fixes/tightening John Szakmeister
2017-01-13  9:16   ` "git fsck" not detecting garbage at the end of blob object files John Szakmeister

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170113180025.xkwyc6mzxl572jn7@sigill.intra.peff.net \
    --to=peff@peff.net \
    --cc=dennis@kaarsemaker.net \
    --cc=git@vger.kernel.org \
    --cc=john@szakmeister.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).