From: Derrick Stolee <stolee@gmail.com> To: git@vger.kernel.org, dstolee@microsoft.com Cc: gitster@pobox.com, sbeller@google.com, pclouds@gmail.com, avarab@gmail.com, sunshine@sunshineco.com, szeder.dev@gmail.com Subject: [PATCH v4 15/23] midx: write object offsets Date: Thu, 12 Jul 2018 15:39:32 -0400 Message-ID: <20180712193940.21065-16-dstolee@microsoft.com> (raw) In-Reply-To: <20180712193940.21065-1-dstolee@microsoft.com> The final pair of chunks for the multi-pack-index file stores the object offsets. We default to using 32-bit offsets as in the pack-index version 1 format, but if there exists an offset larger than 32-bits, we use a trick similar to the pack-index version 2 format by storing all offsets at least 2^31 in a 64-bit table; we use the 32-bit table to point into that 64-bit table as necessary. We only store these 64-bit offsets if necessary, so create a test that manipulates a version 2 pack-index to fake a large offset. This allows us to test that the large offset table is created, but the data does not match the actual packfile offsets. The multi-pack-index offset does match the (corrupted) pack-index offset, so a future feature will compare these offsets during a 'verify' step. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> --- Documentation/technical/pack-format.txt | 15 +++- midx.c | 100 +++++++++++++++++++++++- midx.h | 2 + t/helper/test-read-midx.c | 4 + t/t5319-multi-pack-index.sh | 49 +++++++++--- 5 files changed, 155 insertions(+), 15 deletions(-) diff --git a/Documentation/technical/pack-format.txt b/Documentation/technical/pack-format.txt index 3215f7bfcd..cab5bdd2ff 100644 --- a/Documentation/technical/pack-format.txt +++ b/Documentation/technical/pack-format.txt @@ -311,7 +311,20 @@ CHUNK DATA: The OIDs for all objects in the MIDX are stored in lexicographic order in this chunk. - (This section intentionally left incomplete.) + Object Offsets (ID: {'O', 'O', 'F', 'F'}) + Stores two 4-byte values for every object. + 1: The pack-int-id for the pack storing this object. + 2: The offset within the pack. + If all offsets are less than 2^31, then the large offset chunk + will not exist and offsets are stored as in IDX v1. + If there is at least one offset value larger than 2^32-1, then + the large offset chunk must exist. If the large offset chunk + exists and the 31st bit is on, then removing that bit reveals + the row in the large offsets containing the 8-byte offset of + this object. + + [Optional] Object Large Offsets (ID: {'L', 'O', 'F', 'F'}) + 8-byte offsets into large packfiles. TRAILER: diff --git a/midx.c b/midx.c index 7a954eb0cd..e83110ae92 100644 --- a/midx.c +++ b/midx.c @@ -18,13 +18,18 @@ #define MIDX_HASH_LEN 20 #define MIDX_MIN_SIZE (MIDX_HEADER_SIZE + MIDX_HASH_LEN) -#define MIDX_MAX_CHUNKS 3 +#define MIDX_MAX_CHUNKS 5 #define MIDX_CHUNK_ALIGNMENT 4 #define MIDX_CHUNKID_PACKNAMES 0x504e414d /* "PNAM" */ #define MIDX_CHUNKID_OIDFANOUT 0x4f494446 /* "OIDF" */ #define MIDX_CHUNKID_OIDLOOKUP 0x4f49444c /* "OIDL" */ +#define MIDX_CHUNKID_OBJECTOFFSETS 0x4f4f4646 /* "OOFF" */ +#define MIDX_CHUNKID_LARGEOFFSETS 0x4c4f4646 /* "LOFF" */ #define MIDX_CHUNKLOOKUP_WIDTH (sizeof(uint32_t) + sizeof(uint64_t)) #define MIDX_CHUNK_FANOUT_SIZE (sizeof(uint32_t) * 256) +#define MIDX_CHUNK_OFFSET_WIDTH (2 * sizeof(uint32_t)) +#define MIDX_CHUNK_LARGE_OFFSET_WIDTH (sizeof(uint64_t)) +#define MIDX_LARGE_OFFSET_NEEDED 0x80000000 static char *get_midx_filename(const char *object_dir) { @@ -112,6 +117,14 @@ struct multi_pack_index *load_multi_pack_index(const char *object_dir) m->chunk_oid_lookup = m->data + chunk_offset; break; + case MIDX_CHUNKID_OBJECTOFFSETS: + m->chunk_object_offsets = m->data + chunk_offset; + break; + + case MIDX_CHUNKID_LARGEOFFSETS: + m->chunk_large_offsets = m->data + chunk_offset; + break; + case 0: die(_("terminating multi-pack-index chunk id appears earlier than expected")); break; @@ -131,6 +144,8 @@ struct multi_pack_index *load_multi_pack_index(const char *object_dir) die(_("multi-pack-index missing required OID fanout chunk")); if (!m->chunk_oid_lookup) die(_("multi-pack-index missing required OID lookup chunk")); + if (!m->chunk_object_offsets) + die(_("multi-pack-index missing required object offsets chunk")); m->num_objects = ntohl(m->chunk_oid_fanout[255]); @@ -454,6 +469,56 @@ static size_t write_midx_oid_lookup(struct hashfile *f, unsigned char hash_len, return written; } +static size_t write_midx_object_offsets(struct hashfile *f, int large_offset_needed, + struct pack_midx_entry *objects, uint32_t nr_objects) +{ + struct pack_midx_entry *list = objects; + uint32_t i, nr_large_offset = 0; + size_t written = 0; + + for (i = 0; i < nr_objects; i++) { + struct pack_midx_entry *obj = list++; + + hashwrite_be32(f, obj->pack_int_id); + + if (large_offset_needed && obj->offset >> 31) + hashwrite_be32(f, MIDX_LARGE_OFFSET_NEEDED | nr_large_offset++); + else if (!large_offset_needed && obj->offset >> 32) + BUG("object %s requires a large offset (%"PRIx64") but the MIDX is not writing large offsets!", + oid_to_hex(&obj->oid), + obj->offset); + else + hashwrite_be32(f, (uint32_t)obj->offset); + + written += MIDX_CHUNK_OFFSET_WIDTH; + } + + return written; +} + +static size_t write_midx_large_offsets(struct hashfile *f, uint32_t nr_large_offset, + struct pack_midx_entry *objects, uint32_t nr_objects) +{ + struct pack_midx_entry *list = objects; + size_t written = 0; + + while (nr_large_offset) { + struct pack_midx_entry *obj = list++; + uint64_t offset = obj->offset; + + if (!(offset >> 31)) + continue; + + hashwrite_be32(f, offset >> 32); + hashwrite_be32(f, offset & 0xffffffffUL); + written += 2 * sizeof(uint32_t); + + nr_large_offset--; + } + + return written; +} + int write_midx_file(const char *object_dir) { unsigned char cur_chunk, num_chunks = 0; @@ -466,8 +531,9 @@ int write_midx_file(const char *object_dir) uint64_t written = 0; uint32_t chunk_ids[MIDX_MAX_CHUNKS + 1]; uint64_t chunk_offsets[MIDX_MAX_CHUNKS + 1]; - uint32_t nr_entries; + uint32_t nr_entries, num_large_offsets = 0; struct pack_midx_entry *entries = NULL; + int large_offsets_needed = 0; midx_name = get_midx_filename(object_dir); if (safe_create_leading_directories(midx_name)) { @@ -494,13 +560,19 @@ int write_midx_file(const char *object_dir) sort_packs_by_name(packs.names, packs.nr, pack_perm); entries = get_sorted_entries(packs.list, pack_perm, packs.nr, &nr_entries); + for (i = 0; i < nr_entries; i++) { + if (entries[i].offset > 0x7fffffff) + num_large_offsets++; + if (entries[i].offset > 0xffffffff) + large_offsets_needed = 1; + } hold_lock_file_for_update(&lk, midx_name, LOCK_DIE_ON_ERROR); f = hashfd(lk.tempfile->fd, lk.tempfile->filename.buf); FREE_AND_NULL(midx_name); cur_chunk = 0; - num_chunks = 3; + num_chunks = large_offsets_needed ? 5 : 4; written = write_midx_header(f, num_chunks, packs.nr); @@ -516,9 +588,21 @@ int write_midx_file(const char *object_dir) chunk_offsets[cur_chunk] = chunk_offsets[cur_chunk - 1] + MIDX_CHUNK_FANOUT_SIZE; cur_chunk++; - chunk_ids[cur_chunk] = 0; + chunk_ids[cur_chunk] = MIDX_CHUNKID_OBJECTOFFSETS; chunk_offsets[cur_chunk] = chunk_offsets[cur_chunk - 1] + nr_entries * MIDX_HASH_LEN; + cur_chunk++; + chunk_offsets[cur_chunk] = chunk_offsets[cur_chunk - 1] + nr_entries * MIDX_CHUNK_OFFSET_WIDTH; + if (large_offsets_needed) { + chunk_ids[cur_chunk] = MIDX_CHUNKID_LARGEOFFSETS; + + cur_chunk++; + chunk_offsets[cur_chunk] = chunk_offsets[cur_chunk - 1] + + num_large_offsets * MIDX_CHUNK_LARGE_OFFSET_WIDTH; + } + + chunk_ids[cur_chunk] = 0; + for (i = 0; i <= num_chunks; i++) { if (i && chunk_offsets[i] < chunk_offsets[i - 1]) BUG("incorrect chunk offsets: %"PRIu64" before %"PRIu64, @@ -556,6 +640,14 @@ int write_midx_file(const char *object_dir) written += write_midx_oid_lookup(f, MIDX_HASH_LEN, entries, nr_entries); break; + case MIDX_CHUNKID_OBJECTOFFSETS: + written += write_midx_object_offsets(f, large_offsets_needed, entries, nr_entries); + break; + + case MIDX_CHUNKID_LARGEOFFSETS: + written += write_midx_large_offsets(f, num_large_offsets, entries, nr_entries); + break; + default: BUG("trying to write unknown chunk id %"PRIx32, chunk_ids[i]); diff --git a/midx.h b/midx.h index 8572cf0f4b..e15966272f 100644 --- a/midx.h +++ b/midx.h @@ -17,6 +17,8 @@ struct multi_pack_index { const unsigned char *chunk_pack_names; const uint32_t *chunk_oid_fanout; const unsigned char *chunk_oid_lookup; + const unsigned char *chunk_object_offsets; + const unsigned char *chunk_large_offsets; const char **pack_names; char object_dir[FLEX_ARRAY]; diff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c index f7c17b0940..8e19972e89 100644 --- a/t/helper/test-read-midx.c +++ b/t/helper/test-read-midx.c @@ -26,6 +26,10 @@ static int read_midx_file(const char *object_dir) printf(" oid-fanout"); if (m->chunk_oid_lookup) printf(" oid-lookup"); + if (m->chunk_object_offsets) + printf(" object-offsets"); + if (m->chunk_large_offsets) + printf(" large-offsets"); printf("\nnum_objects: %d\n", m->num_objects); diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh index 1cf609958f..4a4fa26f7a 100755 --- a/t/t5319-multi-pack-index.sh +++ b/t/t5319-multi-pack-index.sh @@ -6,27 +6,30 @@ test_description='multi-pack-indexes' midx_read_expect () { NUM_PACKS=$1 NUM_OBJECTS=$2 + NUM_CHUNKS=$3 + OBJECT_DIR=$4 + EXTRA_CHUNKS="$5" { cat <<-EOF && - header: 4d494458 1 3 $NUM_PACKS - chunks: pack-names oid-fanout oid-lookup + header: 4d494458 1 $NUM_CHUNKS $NUM_PACKS + chunks: pack-names oid-fanout oid-lookup object-offsets$EXTRA_CHUNKS num_objects: $NUM_OBJECTS packs: EOF if test $NUM_PACKS -ge 1 then - ls pack/ | grep idx | sort + ls $OBJECT_DIR/pack/ | grep idx | sort fi && - printf "object-dir: .\n" + printf "object-dir: $OBJECT_DIR\n" } >expect && - test-tool read-midx . >actual && + test-tool read-midx $OBJECT_DIR >actual && test_cmp expect actual } test_expect_success 'write midx with no packs' ' test_when_finished rm -f pack/multi-pack-index && git multi-pack-index --object-dir=. write && - midx_read_expect 0 0 + midx_read_expect 0 0 4 . ' generate_objects () { @@ -76,13 +79,13 @@ test_expect_success 'write midx with one v1 pack' ' pack=$(git pack-objects --index-version=1 pack/test <obj-list) && test_when_finished rm pack/test-$pack.pack pack/test-$pack.idx pack/multi-pack-index && git multi-pack-index --object-dir=. write && - midx_read_expect 1 18 + midx_read_expect 1 18 4 . ' test_expect_success 'write midx with one v2 pack' ' git pack-objects --index-version=2,0x40 pack/test <obj-list && git multi-pack-index --object-dir=. write && - midx_read_expect 1 18 + midx_read_expect 1 18 4 . ' test_expect_success 'add more objects' ' @@ -96,7 +99,7 @@ test_expect_success 'add more objects' ' test_expect_success 'write midx with two packs' ' git pack-objects --index-version=1 pack/test-2 <obj-list && git multi-pack-index --object-dir=. write && - midx_read_expect 2 34 + midx_read_expect 2 34 4 . ' test_expect_success 'add more packs' ' @@ -110,7 +113,33 @@ test_expect_success 'add more packs' ' test_expect_success 'write midx with twelve packs' ' git multi-pack-index --object-dir=. write && - midx_read_expect 12 74 + midx_read_expect 12 74 4 . +' + +# usage: corrupt_data <file> <pos> [<data>] +corrupt_data () { + file=$1 + pos=$2 + data="${3:-\0}" + printf "$data" | dd of="$file" bs=1 seek="$pos" conv=notrunc +} + +# Force 64-bit offsets by manipulating the idx file. +# This makes the IDX file _incorrect_ so be careful to clean up after! +test_expect_success 'force some 64-bit offsets with pack-objects' ' + mkdir objects64 && + mkdir objects64/pack && + for i in $(test_seq 1 11) + do + generate_objects 11 + done && + commit_and_list_objects && + pack64=$(git pack-objects --index-version=2,0x40 objects64/pack/test-64 <obj-list) && + idx64=objects64/pack/test-64-$pack64.idx && + chmod u+w $idx64 && + corrupt_data $idx64 2999 "\02" && + midx64=$(git multi-pack-index --object-dir=objects64 write) && + midx_read_expect 1 63 5 objects64 " large-offsets" ' test_done -- 2.18.0.118.gd4f65b8d14
next prev parent reply other threads:[~2018-07-12 19:40 UTC|newest] Thread overview: 192+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-06-07 14:03 [PATCH 00/23] Multi-pack-index (MIDX) Derrick Stolee 2018-06-07 14:03 ` [PATCH 01/23] midx: add design document Derrick Stolee 2018-06-11 19:04 ` Stefan Beller 2018-06-18 18:48 ` Derrick Stolee 2018-06-07 14:03 ` [PATCH 02/23] midx: add midx format details to pack-format.txt Derrick Stolee 2018-06-11 19:19 ` Stefan Beller 2018-06-18 19:01 ` Derrick Stolee 2018-06-18 19:41 ` Stefan Beller 2018-06-07 14:03 ` [PATCH 03/23] midx: add midx builtin Derrick Stolee 2018-06-07 17:20 ` Duy Nguyen 2018-06-18 19:23 ` Derrick Stolee 2018-06-11 21:02 ` Stefan Beller 2018-06-18 19:40 ` Derrick Stolee 2018-06-18 19:55 ` Stefan Beller 2018-06-18 19:58 ` Derrick Stolee 2018-06-07 14:03 ` [PATCH 04/23] midx: add 'write' subcommand and basic wiring Derrick Stolee 2018-06-07 17:27 ` Duy Nguyen 2018-06-07 14:03 ` [PATCH 05/23] midx: write header information to lockfile Derrick Stolee 2018-06-07 17:35 ` Duy Nguyen 2018-06-12 15:00 ` Duy Nguyen 2018-06-19 12:54 ` Derrick Stolee 2018-06-19 14:59 ` Duy Nguyen 2018-06-19 15:24 ` Derrick Stolee 2018-06-07 14:03 ` [PATCH 06/23] midx: struct midxed_git and 'read' subcommand Derrick Stolee 2018-06-07 17:54 ` Duy Nguyen 2018-06-20 13:13 ` Derrick Stolee 2018-06-07 18:31 ` Duy Nguyen 2018-06-20 13:33 ` Derrick Stolee 2018-06-20 15:07 ` Duy Nguyen 2018-06-20 16:39 ` Derrick Stolee 2018-06-07 14:03 ` [PATCH 07/23] midx: expand test data Derrick Stolee 2018-06-07 14:03 ` [PATCH 08/23] midx: read packfiles from pack directory Derrick Stolee 2018-06-07 18:03 ` Duy Nguyen 2018-06-20 16:33 ` [PATCH] packfile: generalize pack directory list Derrick Stolee 2018-06-07 14:03 ` [PATCH 09/23] midx: write pack names in chunk Derrick Stolee 2018-06-07 18:26 ` Duy Nguyen 2018-06-21 15:25 ` Derrick Stolee 2018-06-21 17:38 ` Junio C Hamano 2018-06-22 18:25 ` Derrick Stolee 2018-06-22 18:31 ` Junio C Hamano 2018-06-22 18:32 ` Derrick Stolee 2018-06-07 14:03 ` [PATCH 10/23] midx: write a lookup into the pack names chunk Derrick Stolee 2018-06-09 16:43 ` Duy Nguyen 2018-06-21 17:23 ` Derrick Stolee 2018-06-07 14:03 ` [PATCH 11/23] midx: sort and deduplicate objects from packfiles Derrick Stolee 2018-06-09 17:07 ` Duy Nguyen 2018-06-21 17:54 ` Derrick Stolee 2018-06-07 14:03 ` [PATCH 12/23] midx: write object ids in a chunk Derrick Stolee 2018-06-09 17:25 ` Duy Nguyen 2018-06-07 14:03 ` [PATCH 13/23] midx: write object id fanout chunk Derrick Stolee 2018-06-09 17:28 ` Duy Nguyen 2018-06-21 19:49 ` Derrick Stolee 2018-06-07 14:03 ` [PATCH 14/23] midx: write object offsets Derrick Stolee 2018-06-09 17:41 ` Duy Nguyen 2018-06-07 14:03 ` [PATCH 15/23] midx: create core.midx config setting Derrick Stolee 2018-06-07 14:03 ` [PATCH 16/23] midx: prepare midxed_git struct Derrick Stolee 2018-06-09 17:47 ` Duy Nguyen 2018-06-07 14:03 ` [PATCH 17/23] midx: read objects from multi-pack-index Derrick Stolee 2018-06-09 17:56 ` Duy Nguyen 2018-06-21 20:03 ` Derrick Stolee 2018-06-07 14:03 ` [PATCH 18/23] midx: use midx in abbreviation calculations Derrick Stolee 2018-06-09 18:01 ` Duy Nguyen 2018-06-22 18:38 ` Derrick Stolee 2018-06-07 14:03 ` [PATCH 19/23] midx: use existing midx when writing new one Derrick Stolee 2018-06-07 14:03 ` [PATCH 20/23] midx: use midx in approximate_object_count Derrick Stolee 2018-06-09 18:03 ` Duy Nguyen 2018-06-22 18:39 ` Derrick Stolee 2018-06-07 14:03 ` [PATCH 21/23] midx: prevent duplicate packfile loads Derrick Stolee 2018-06-09 18:05 ` Duy Nguyen 2018-06-07 14:03 ` [PATCH 22/23] midx: use midx to find ref-deltas Derrick Stolee 2018-06-07 14:03 ` [PATCH 23/23] midx: clear midx on repack Derrick Stolee 2018-06-09 18:13 ` Duy Nguyen 2018-06-22 18:44 ` Derrick Stolee 2018-06-07 14:06 ` [PATCH 00/23] Multi-pack-index (MIDX) Derrick Stolee 2018-06-07 14:45 ` Ævar Arnfjörð Bjarmason 2018-06-07 14:54 ` Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 00/24] " Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 01/24] multi-pack-index: add design document Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 02/24] multi-pack-index: add format details Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 03/24] multi-pack-index: add builtin Derrick Stolee 2018-06-25 19:15 ` Junio C Hamano 2018-06-25 14:34 ` [PATCH v2 04/24] multi-pack-index: add 'write' verb Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 05/24] midx: write header information to lockfile Derrick Stolee 2018-06-25 19:19 ` Junio C Hamano 2018-07-05 19:13 ` Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 06/24] multi-pack-index: load into memory Derrick Stolee 2018-06-25 19:38 ` Junio C Hamano 2018-07-05 14:19 ` Derrick Stolee 2018-07-05 18:58 ` Eric Sunshine 2018-07-06 19:20 ` Junio C Hamano 2018-06-25 14:34 ` [PATCH v2 07/24] multi-pack-index: expand test data Derrick Stolee 2018-06-25 19:45 ` Junio C Hamano 2018-06-25 14:34 ` [PATCH v2 08/24] packfile: generalize pack directory list Derrick Stolee 2018-06-25 19:57 ` Junio C Hamano 2018-06-25 14:34 ` [PATCH v2 09/24] multi-pack-index: read packfile list Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 10/24] multi-pack-index: write pack names in chunk Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 11/24] midx: read pack names into array Derrick Stolee 2018-06-25 23:52 ` Eric Sunshine 2018-06-25 14:34 ` [PATCH v2 12/24] midx: sort and deduplicate objects from packfiles Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 13/24] midx: write object ids in a chunk Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 14/24] midx: write object id fanout chunk Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 15/24] midx: write object offsets Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 16/24] config: create core.multiPackIndex setting Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 17/24] midx: prepare midxed_git struct Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 18/24] midx: read objects from multi-pack-index Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 19/24] midx: use midx in abbreviation calculations Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 20/24] midx: use existing midx when writing new one Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 21/24] midx: use midx in approximate_object_count Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 22/24] midx: prevent duplicate packfile loads Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 23/24] packfile: skip loading index if in multi-pack-index Derrick Stolee 2018-06-25 14:34 ` [PATCH v2 24/24] midx: clear midx on repack Derrick Stolee 2018-07-06 0:52 ` [PATCH v3 00/24] Multi-pack-index (MIDX) Derrick Stolee 2018-07-06 0:52 ` [PATCH v3 01/24] multi-pack-index: add design document Derrick Stolee 2018-07-06 0:52 ` [PATCH v3 02/24] multi-pack-index: add format details Derrick Stolee 2018-07-06 0:53 ` [PATCH v3 03/24] multi-pack-index: add builtin Derrick Stolee 2018-07-06 3:54 ` Eric Sunshine 2018-07-06 0:53 ` [PATCH v3 04/24] multi-pack-index: add 'write' verb Derrick Stolee 2018-07-06 4:07 ` Eric Sunshine 2018-07-06 0:53 ` [PATCH v3 05/24] midx: write header information to lockfile Derrick Stolee 2018-07-06 0:53 ` [PATCH v3 06/24] multi-pack-index: load into memory Derrick Stolee 2018-07-06 4:19 ` Eric Sunshine 2018-07-06 5:18 ` Eric Sunshine 2018-07-09 19:08 ` Junio C Hamano 2018-07-12 16:06 ` Derrick Stolee 2018-07-06 0:53 ` [PATCH v3 07/24] multi-pack-index: expand test data Derrick Stolee 2018-07-06 4:36 ` Eric Sunshine 2018-07-06 5:20 ` Eric Sunshine 2018-07-12 14:10 ` Derrick Stolee 2018-07-12 18:02 ` Eric Sunshine 2018-07-12 18:06 ` Derrick Stolee 2018-07-06 0:53 ` [PATCH v3 08/24] packfile: generalize pack directory list Derrick Stolee 2018-07-06 0:53 ` [PATCH v3 09/24] multi-pack-index: read packfile list Derrick Stolee 2018-07-06 0:53 ` [PATCH v3 10/24] multi-pack-index: write pack names in chunk Derrick Stolee 2018-07-06 0:53 ` [PATCH v3 11/24] midx: read pack names into array Derrick Stolee 2018-07-06 4:58 ` Eric Sunshine 2018-07-06 0:53 ` [PATCH v3 12/24] midx: sort and deduplicate objects from packfiles Derrick Stolee 2018-07-06 0:53 ` [PATCH v3 13/24] midx: write object ids in a chunk Derrick Stolee 2018-07-06 5:04 ` Eric Sunshine 2018-07-06 0:53 ` [PATCH v3 14/24] midx: write object id fanout chunk Derrick Stolee 2018-07-06 0:53 ` [PATCH v3 15/24] midx: write object offsets Derrick Stolee 2018-07-06 5:27 ` Eric Sunshine 2018-07-12 16:33 ` Derrick Stolee 2018-07-06 0:53 ` [PATCH v3 16/24] config: create core.multiPackIndex setting Derrick Stolee 2018-07-06 5:39 ` Eric Sunshine 2018-07-12 13:19 ` Derrick Stolee 2018-07-12 16:30 ` Derrick Stolee 2018-07-11 9:48 ` SZEDER Gábor 2018-07-12 13:01 ` Derrick Stolee 2018-07-12 13:31 ` SZEDER Gábor 2018-07-12 15:40 ` Derrick Stolee 2018-07-12 17:29 ` Junio C Hamano 2018-07-06 0:53 ` [PATCH v3 17/24] midx: prepare midxed_git struct Derrick Stolee 2018-07-06 5:41 ` Eric Sunshine 2018-07-06 0:53 ` [PATCH v3 18/24] midx: read objects from multi-pack-index Derrick Stolee 2018-07-06 0:53 ` [PATCH v3 19/24] midx: use midx in abbreviation calculations Derrick Stolee 2018-07-06 0:53 ` [PATCH v3 20/24] midx: use existing midx when writing new one Derrick Stolee 2018-07-06 0:53 ` [PATCH v3 21/24] midx: use midx in approximate_object_count Derrick Stolee 2018-07-06 0:53 ` [PATCH v3 22/24] midx: prevent duplicate packfile loads Derrick Stolee 2018-07-06 0:53 ` [PATCH v3 23/24] packfile: skip loading index if in multi-pack-index Derrick Stolee 2018-07-06 0:53 ` [PATCH v3 24/24] midx: clear midx on repack Derrick Stolee 2018-07-06 5:52 ` Eric Sunshine 2018-07-12 19:39 ` [PATCH v4 00/23] Multi-pack-index (MIDX) Derrick Stolee 2018-07-12 19:39 ` [PATCH v4 01/23] multi-pack-index: add design document Derrick Stolee 2018-07-12 19:39 ` [PATCH v4 02/23] multi-pack-index: add format details Derrick Stolee 2018-07-12 19:39 ` [PATCH v4 03/23] multi-pack-index: add builtin Derrick Stolee 2018-07-20 18:22 ` Junio C Hamano 2018-07-20 22:15 ` brian m. carlson 2018-07-20 22:28 ` Junio C Hamano 2018-07-12 19:39 ` [PATCH v4 04/23] multi-pack-index: add 'write' verb Derrick Stolee 2018-07-12 22:56 ` Eric Sunshine 2018-07-12 19:39 ` [PATCH v4 05/23] midx: write header information to lockfile Derrick Stolee 2018-07-12 19:39 ` [PATCH v4 06/23] multi-pack-index: load into memory Derrick Stolee 2018-07-12 19:39 ` [PATCH v4 07/23] t5319: expand test data Derrick Stolee 2018-07-12 19:39 ` [PATCH v4 08/23] packfile: generalize pack directory list Derrick Stolee 2018-07-12 19:39 ` [PATCH v4 09/23] multi-pack-index: read packfile list Derrick Stolee 2018-07-12 19:39 ` [PATCH v4 10/23] multi-pack-index: write pack names in chunk Derrick Stolee 2018-07-12 19:39 ` [PATCH v4 11/23] midx: read pack names into array Derrick Stolee 2018-07-12 19:39 ` [PATCH v4 12/23] midx: sort and deduplicate objects from packfiles Derrick Stolee 2018-07-12 19:39 ` [PATCH v4 13/23] midx: write object ids in a chunk Derrick Stolee 2018-07-12 19:39 ` [PATCH v4 14/23] midx: write object id fanout chunk Derrick Stolee 2018-07-12 19:39 ` Derrick Stolee [this message] 2018-07-12 19:39 ` [PATCH v4 16/23] config: create core.multiPackIndex setting Derrick Stolee 2018-07-12 21:05 ` Junio C Hamano 2018-07-13 0:50 ` Derrick Stolee 2018-07-12 19:39 ` [PATCH v4 17/23] midx: read objects from multi-pack-index Derrick Stolee 2018-07-12 19:39 ` [PATCH v4 18/23] midx: use midx in abbreviation calculations Derrick Stolee 2018-07-12 19:39 ` [PATCH v4 19/23] midx: use existing midx when writing new one Derrick Stolee 2018-07-12 19:39 ` [PATCH v4 20/23] midx: use midx in approximate_object_count Derrick Stolee 2018-07-12 19:39 ` [PATCH v4 21/23] midx: prevent duplicate packfile loads Derrick Stolee 2018-07-12 19:39 ` [PATCH v4 22/23] packfile: skip loading index if in multi-pack-index Derrick Stolee 2018-07-12 19:39 ` [PATCH v4 23/23] midx: clear midx on repack Derrick Stolee 2018-07-12 21:11 ` [PATCH v4 00/23] Multi-pack-index (MIDX) Junio C Hamano
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: http://vger.kernel.org/majordomo-info.html * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20180712193940.21065-16-dstolee@microsoft.com \ --to=stolee@gmail.com \ --cc=avarab@gmail.com \ --cc=dstolee@microsoft.com \ --cc=git@vger.kernel.org \ --cc=gitster@pobox.com \ --cc=pclouds@gmail.com \ --cc=sbeller@google.com \ --cc=sunshine@sunshineco.com \ --cc=szeder.dev@gmail.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
git@vger.kernel.org list mirror (unofficial, one of many) This inbox may be cloned and mirrored by anyone: git clone --mirror https://public-inbox.org/git git clone --mirror http://ou63pmih66umazou.onion/git git clone --mirror http://czquwvybam4bgbro.onion/git git clone --mirror http://hjrcffqmbrq6wope.onion/git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V1 git git/ https://public-inbox.org/git \ git@vger.kernel.org public-inbox-index git Example config snippet for mirrors. Newsgroups are available over NNTP: nntp://news.public-inbox.org/inbox.comp.version-control.git nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git nntp://news.gmane.io/gmane.comp.version-control.git note: .onion URLs require Tor: https://www.torproject.org/ code repositories for the project(s) associated with this inbox: https://80x24.org/mirrors/git.git AGPL code for this site: git clone https://public-inbox.org/public-inbox.git