git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Jeff King <peff@peff.net>
To: John Szakmeister <john@szakmeister.net>
Cc: Dennis Kaarsemaker <dennis@kaarsemaker.net>, git@vger.kernel.org
Subject: [PATCH 5/6] fsck: parse loose object paths directly
Date: Fri, 13 Jan 2017 12:59:44 -0500	[thread overview]
Message-ID: <20170113175944.tdbfqx3e4xhris7m@sigill.intra.peff.net> (raw)
In-Reply-To: <20170113175258.e66taigy4wpokohk@sigill.intra.peff.net>

When we iterate over the list of loose objects to check, we
get the actual path of each object. But we then throw it
away and pass just the sha1 to fsck_sha1(), which will do a
fresh lookup. Usually it would find the same object, but it
may not if an object exists both as a loose and a packed
object. We may end up checking the packed object twice, and
never look at the loose one.

In practice this isn't too terrible, because if fsck doesn't
complain, it means you have at least one good copy. But
since the point of fsck is to look for corruption, we should
be thorough.

The new read_loose_object() interface can help us get the
data from disk, and then we replace parse_object() with
parse_object_buffer(). As a bonus, our error messages now
mention the path to a corrupted object, which should make it
easier to track down errors when they do happen.

Signed-off-by: Jeff King <peff@peff.net>
---
 builtin/fsck.c  | 46 +++++++++++++++++++++++++++++++++-------------
 t/t1450-fsck.sh | 16 ++++++++++++++++
 2 files changed, 49 insertions(+), 13 deletions(-)

diff --git a/builtin/fsck.c b/builtin/fsck.c
index f01b81eeb..4b91ee95e 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -362,18 +362,6 @@ static int fsck_obj(struct object *obj)
 	return 0;
 }
 
-static int fsck_sha1(const unsigned char *sha1)
-{
-	struct object *obj = parse_object(sha1);
-	if (!obj) {
-		errors_found |= ERROR_OBJECT;
-		return error("%s: object corrupt or missing",
-			     sha1_to_hex(sha1));
-	}
-	obj->flags |= HAS_OBJ;
-	return fsck_obj(obj);
-}
-
 static int fsck_obj_buffer(const unsigned char *sha1, enum object_type type,
 			   unsigned long size, void *buffer, int *eaten)
 {
@@ -488,9 +476,41 @@ static void get_default_heads(void)
 	}
 }
 
+static struct object *parse_loose_object(const unsigned char *sha1,
+					 const char *path)
+{
+	struct object *obj;
+	void *contents;
+	enum object_type type;
+	unsigned long size;
+	int eaten;
+
+	if (read_loose_object(path, sha1, &type, &size, &contents) < 0)
+		return NULL;
+
+	if (!contents && type != OBJ_BLOB)
+		die("BUG: read_loose_object streamed a non-blob");
+
+	obj = parse_object_buffer(sha1, type, size, contents, &eaten);
+
+	if (!eaten)
+		free(contents);
+	return obj;
+}
+
 static int fsck_loose(const unsigned char *sha1, const char *path, void *data)
 {
-	if (fsck_sha1(sha1))
+	struct object *obj = parse_loose_object(sha1, path);
+
+	if (!obj) {
+		errors_found |= ERROR_OBJECT;
+		error("%s: object corrupt or missing: %s",
+		      sha1_to_hex(sha1), path);
+		return 0; /* keep checking other objects */
+	}
+
+	obj->flags = HAS_OBJ;
+	if (fsck_obj(obj))
 		errors_found |= ERROR_OBJECT;
 	return 0;
 }
diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index c39d42120..455c186fe 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -581,4 +581,20 @@ test_expect_success 'fsck errors in packed objects' '
 	! grep corrupt out
 '
 
+test_expect_success 'fsck finds problems in duplicate loose objects' '
+	rm -rf broken-duplicate &&
+	git init broken-duplicate &&
+	(
+		cd broken-duplicate &&
+		test_commit duplicate &&
+		# no "-d" here, so we end up with duplicates
+		git repack &&
+		# now corrupt the loose copy
+		file=$(sha1_file "$(git rev-parse HEAD)") &&
+		rm "$file" &&
+		echo broken >"$file" &&
+		test_must_fail git fsck
+	)
+'
+
 test_done
-- 
2.11.0.629.g10075098c


  parent reply	other threads:[~2017-01-13 17:59 UTC|newest]

Thread overview: 39+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-01-07 12:50 "git fsck" not detecting garbage at the end of blob object files John Szakmeister
2017-01-07 21:47 ` Dennis Kaarsemaker
2017-01-08  5:26   ` Jeff King
2017-01-13  9:15     ` John Szakmeister
2017-01-13 17:52       ` [PATCH 0/6] loose-object fsck fixes/tightening Jeff King
2017-01-13 17:54         ` [PATCH 1/6] t1450: refactor loose-object removal Jeff King
2017-01-13 17:54         ` [PATCH 2/6] sha1_file: fix error message for alternate objects Jeff King
2017-01-13 17:55         ` [PATCH 3/6] t1450: test fsck of packed objects Jeff King
2017-01-13 17:58         ` [PATCH 4/6] sha1_file: add read_loose_object() function Jeff King
2017-01-13 17:59         ` Jeff King [this message]
2018-10-30 20:03           ` Infinite loop regression in git-fsck in v2.12.0 Ævar Arnfjörð Bjarmason
2018-10-30 21:35             ` Jeff King
2018-10-30 22:28               ` Junio C Hamano
2018-10-30 22:56                 ` Jeff King
2018-10-30 23:12                   ` Jeff King
2018-10-30 23:18                     ` [PATCH 1/3] t1450: check large blob in trailing-garbage test Jeff King
2018-10-30 23:23                     ` [PATCH 2/3] check_stream_sha1(): handle input underflow Jeff King
2018-10-31  4:23                       ` Junio C Hamano
2018-10-31  4:30                         ` Jeff King
2018-10-31  4:44                           ` Junio C Hamano
2018-10-31  5:03                             ` Jeff King
2018-10-31  5:13                               ` Jeff King
2018-10-31  5:31                                 ` Junio C Hamano
2018-10-30 23:23                     ` [PATCH 3/3] cat-file: handle streaming failures consistently Jeff King
2018-10-31 12:42                       ` [PATCH 0/3] Add a GIT_TEST_FSCK test mode Ævar Arnfjörð Bjarmason
2018-10-31 12:42                       ` [PATCH 1/3] tests: add a "env-bool" helper to test-tool Ævar Arnfjörð Bjarmason
2018-10-31 12:42                       ` [PATCH 2/3] tests: mark those tests where "git fsck" fails at the end Ævar Arnfjörð Bjarmason
2018-11-01  3:37                         ` Junio C Hamano
2018-10-31 12:42                       ` [PATCH 3/3] tests: add a special test setup that runs "git fsck" before exiting Ævar Arnfjörð Bjarmason
2018-10-31 13:33                       ` [PATCH 3/3] cat-file: handle streaming failures consistently Torsten Bögershausen
2018-10-31 14:23                         ` Junio C Hamano
2018-10-31 14:37                           ` Jeff King
2018-10-31 17:38                       ` Eric Sunshine
2018-10-31 20:29                         ` Jeff King
2018-10-30 21:56             ` Infinite loop regression in git-fsck in v2.12.0 Ævar Arnfjörð Bjarmason
2018-10-30 23:08               ` Jeff King
2017-01-13 18:00         ` [PATCH 6/6] fsck: detect trailing garbage in all object types Jeff King
2017-01-19 11:18         ` [PATCH 0/6] loose-object fsck fixes/tightening John Szakmeister
2017-01-13  9:16   ` "git fsck" not detecting garbage at the end of blob object files John Szakmeister

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170113175944.tdbfqx3e4xhris7m@sigill.intra.peff.net \
    --to=peff@peff.net \
    --cc=dennis@kaarsemaker.net \
    --cc=git@vger.kernel.org \
    --cc=john@szakmeister.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).