From: "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: avarab@gmail.com, Junio C Hamano <gitster@pobox.com>,
Derrick Stolee <dstolee@microsoft.com>
Subject: [PATCH v2 09/11] multi-pack-index: verify object offsets
Date: Thu, 13 Sep 2018 11:02:25 -0700 (PDT) [thread overview]
Message-ID: <ef20193d59bd33a4c56e8c059ea7ddefbb6a8e72.1536861730.git.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.34.v2.git.gitgitgadget@gmail.com>
From: Derrick Stolee <dstolee@microsoft.com>
The 'git multi-pack-index verify' command must verify the object
offsets stored in the multi-pack-index are correct. There are two
ways the offset chunk can be incorrect: the pack-int-id and the
object offset.
Replace the BUG() statement with a die() statement, now that we
may hit a bad pack-int-id during a 'verify' command on a corrupt
multi-pack-index, and it is covered by a test.
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
midx.c | 29 ++++++++++++++++++++++++++++-
t/t5319-multi-pack-index.sh | 27 +++++++++++++++++++++++++++
2 files changed, 55 insertions(+), 1 deletion(-)
diff --git a/midx.c b/midx.c
index 80094c02a7..47e7e6113a 100644
--- a/midx.c
+++ b/midx.c
@@ -197,7 +197,8 @@ int prepare_midx_pack(struct multi_pack_index *m, uint32_t pack_int_id)
struct strbuf pack_name = STRBUF_INIT;
if (pack_int_id >= m->num_packs)
- BUG("bad pack-int-id");
+ die(_("bad pack-int-id: %u (%u total packs"),
+ pack_int_id, m->num_packs);
if (m->packs[pack_int_id])
return 0;
@@ -970,5 +971,31 @@ int verify_midx_file(const char *object_dir)
i, oid_to_hex(&oid1), oid_to_hex(&oid2), i + 1);
}
+ for (i = 0; i < m->num_objects; i++) {
+ struct object_id oid;
+ struct pack_entry e;
+ off_t m_offset, p_offset;
+
+ nth_midxed_object_oid(&oid, m, i);
+ if (!fill_midx_entry(&oid, &e, m)) {
+ midx_report(_("failed to load pack entry for oid[%d] = %s"),
+ i, oid_to_hex(&oid));
+ continue;
+ }
+
+ if (open_pack_index(e.p)) {
+ midx_report(_("failed to load pack-index for packfile %s"),
+ e.p->pack_name);
+ break;
+ }
+
+ m_offset = e.offset;
+ p_offset = find_pack_entry_one(oid.hash, e.p);
+
+ if (m_offset != p_offset)
+ midx_report(_("incorrect object offset for oid[%d] = %s: %"PRIx64" != %"PRIx64),
+ i, oid_to_hex(&oid), m_offset, p_offset);
+ }
+
return verify_midx_error;
}
diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh
index a968b9a959..828c240389 100755
--- a/t/t5319-multi-pack-index.sh
+++ b/t/t5319-multi-pack-index.sh
@@ -176,6 +176,7 @@ test_expect_success 'verify bad signature' '
'
HASH_LEN=20
+NUM_OBJECTS=74
MIDX_BYTE_VERSION=4
MIDX_BYTE_OID_VERSION=5
MIDX_BYTE_CHUNK_COUNT=6
@@ -192,6 +193,10 @@ MIDX_OID_FANOUT_WIDTH=4
MIDX_BYTE_OID_FANOUT_ORDER=$((MIDX_OFFSET_OID_FANOUT + 250 * $MIDX_OID_FANOUT_WIDTH + 1))
MIDX_OFFSET_OID_LOOKUP=$(($MIDX_OFFSET_OID_FANOUT + 256 * $MIDX_OID_FANOUT_WIDTH))
MIDX_BYTE_OID_LOOKUP=$(($MIDX_OFFSET_OID_LOOKUP + 16 * $HASH_LEN))
+MIDX_OFFSET_OBJECT_OFFSETS=$(($MIDX_OFFSET_OID_LOOKUP + $NUM_OBJECTS * $HASH_LEN))
+MIDX_OFFSET_WIDTH=8
+MIDX_BYTE_PACK_INT_ID=$(($MIDX_OFFSET_OBJECT_OFFSETS + 16 * $MIDX_OFFSET_WIDTH + 2))
+MIDX_BYTE_OFFSET=$(($MIDX_OFFSET_OBJECT_OFFSETS + 16 * $MIDX_OFFSET_WIDTH + 6))
test_expect_success 'verify bad version' '
corrupt_midx_and_verify $MIDX_BYTE_VERSION "\00" $objdir \
@@ -243,6 +248,16 @@ test_expect_success 'verify oid lookup out of order' '
"oid lookup out of order"
'
+test_expect_success 'verify incorrect pack-int-id' '
+ corrupt_midx_and_verify $MIDX_BYTE_PACK_INT_ID "\07" $objdir \
+ "bad pack-int-id"
+'
+
+test_expect_success 'verify incorrect offset' '
+ corrupt_midx_and_verify $MIDX_BYTE_OFFSET "\07" $objdir \
+ "incorrect object offset"
+'
+
test_expect_success 'repack removes multi-pack-index' '
test_path_is_file $objdir/pack/multi-pack-index &&
git repack -adf &&
@@ -310,4 +325,16 @@ test_expect_success 'verify multi-pack-index with 64-bit offsets' '
git multi-pack-index verify --object-dir=objects64
'
+NUM_OBJECTS=63
+MIDX_OFFSET_OID_FANOUT=$((MIDX_OFFSET_PACKNAMES + 54))
+MIDX_OFFSET_OID_LOOKUP=$((MIDX_OFFSET_OID_FANOUT + 256 * $MIDX_OID_FANOUT_WIDTH))
+MIDX_OFFSET_OBJECT_OFFSETS=$(($MIDX_OFFSET_OID_LOOKUP + $NUM_OBJECTS * $HASH_LEN))
+MIDX_OFFSET_LARGE_OFFSETS=$(($MIDX_OFFSET_OBJECT_OFFSETS + $NUM_OBJECTS * $MIDX_OFFSET_WIDTH))
+MIDX_BYTE_LARGE_OFFSET=$(($MIDX_OFFSET_LARGE_OFFSETS + 3))
+
+test_expect_success 'verify incorrect 64-bit offset' '
+ corrupt_midx_and_verify $MIDX_BYTE_LARGE_OFFSET "\07" objects64 \
+ "incorrect object offset"
+'
+
test_done
--
gitgitgadget
next prev parent reply other threads:[~2018-09-13 18:02 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-09-05 14:46 [PATCH 00/11] Add 'git multi-pack-index verify' command Derrick Stolee via GitGitGadget
2018-09-05 14:46 ` [PATCH 01/11] multi-pack-index: add 'verify' verb Derrick Stolee via GitGitGadget
2018-09-05 18:59 ` Eric Sunshine
2018-09-05 19:37 ` Derrick Stolee
2018-09-05 14:46 ` [PATCH 02/11] multi-pack-index: verify bad header Derrick Stolee via GitGitGadget
2018-09-07 0:27 ` Eric Sunshine
2018-09-05 14:46 ` [PATCH 03/11] multi-pack-index: verify corrupt chunk lookup table Derrick Stolee via GitGitGadget
2018-09-05 14:46 ` [PATCH 04/11] multi-pack-index: verify packname order Derrick Stolee via GitGitGadget
2018-09-05 18:15 ` Stefan Beller
2018-09-05 19:11 ` Derrick Stolee
2018-09-05 19:14 ` Stefan Beller
2018-09-05 19:28 ` Derrick Stolee
2018-09-05 14:46 ` [PATCH 05/11] multi-pack-index: verify missing pack Derrick Stolee via GitGitGadget
2018-09-05 14:46 ` [PATCH 06/11] multi-pack-index: verify oid fanout order Derrick Stolee via GitGitGadget
2018-09-05 14:46 ` [PATCH 07/11] multi-pack-index: verify oid lookup order Derrick Stolee via GitGitGadget
2018-09-05 14:46 ` [PATCH 08/11] multi-pack-index: fix 32-bit vs 64-bit size check Derrick Stolee via GitGitGadget
2018-09-05 14:46 ` [PATCH 09/11] multi-pack-index: verify object offsets Derrick Stolee via GitGitGadget
2018-09-07 0:34 ` Eric Sunshine
2018-09-07 13:10 ` Derrick Stolee
2018-09-05 14:46 ` [PATCH 10/11] multi-pack-index: report progress during 'verify' Derrick Stolee via GitGitGadget
2018-09-05 14:46 ` [PATCH 11/11] fsck: verify multi-pack-index Derrick Stolee via GitGitGadget
2018-09-13 18:02 ` [PATCH v2 00/11] Add 'git multi-pack-index verify' command Derrick Stolee via GitGitGadget
2018-09-13 18:02 ` [PATCH v2 01/11] multi-pack-index: add 'verify' verb Derrick Stolee via GitGitGadget
2018-09-13 18:02 ` [PATCH v2 02/11] multi-pack-index: verify bad header Derrick Stolee via GitGitGadget
2018-09-13 18:02 ` [PATCH v2 03/11] multi-pack-index: verify corrupt chunk lookup table Derrick Stolee via GitGitGadget
2018-09-13 18:02 ` [PATCH v2 04/11] multi-pack-index: verify packname order Derrick Stolee via GitGitGadget
2018-09-13 18:02 ` [PATCH v2 05/11] multi-pack-index: verify missing pack Derrick Stolee via GitGitGadget
2018-09-13 18:02 ` [PATCH v2 06/11] multi-pack-index: verify oid fanout order Derrick Stolee via GitGitGadget
2018-09-13 18:02 ` [PATCH v2 07/11] multi-pack-index: verify oid lookup order Derrick Stolee via GitGitGadget
2018-09-13 18:02 ` [PATCH v2 08/11] multi-pack-index: fix 32-bit vs 64-bit size check Derrick Stolee via GitGitGadget
2018-09-13 18:02 ` Derrick Stolee via GitGitGadget [this message]
2018-09-13 18:02 ` [PATCH v2 10/11] multi-pack-index: report progress during 'verify' Derrick Stolee via GitGitGadget
2018-09-13 19:02 ` Ævar Arnfjörð Bjarmason
2018-09-13 18:02 ` [PATCH v2 11/11] fsck: verify multi-pack-index Derrick Stolee via GitGitGadget
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ef20193d59bd33a4c56e8c059ea7ddefbb6a8e72.1536861730.git.gitgitgadget@gmail.com \
--to=gitgitgadget@gmail.com \
--cc=avarab@gmail.com \
--cc=dstolee@microsoft.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).