git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / Atom feed
From: Derrick Stolee <stolee@gmail.com>
To: Stefan Beller <sbeller@google.com>
Cc: git@vger.kernel.org, dstolee@microsoft.com, avarab@gmail.com,
	jrnieder@gmail.com, jonathantanmy@google.com,
	mfick@codeaurora.org
Subject: Re: [PATCH 01/23] midx: add design document
Date: Mon, 18 Jun 2018 14:48:25 -0400
Message-ID: <e251bcff-8638-19d8-2937-0e913b19215a@gmail.com> (raw)
In-Reply-To: <CAGZ79ka4RD_E5NpEf8-7BUCcoqEc+hDrHNR+YAqT=6m5TYg7mg@mail.gmail.com>

On 6/11/2018 3:04 PM, Stefan Beller wrote:
> On Thu, Jun 7, 2018 at 7:03 AM Derrick Stolee <stolee@gmail.com> wrote:
>> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
>> ---
>>   Documentation/technical/midx.txt | 109 +++++++++++++++++++++++++++++++
>>   1 file changed, 109 insertions(+)
>>   create mode 100644 Documentation/technical/midx.txt
>>
>> diff --git a/Documentation/technical/midx.txt b/Documentation/technical/midx.txt
>> new file mode 100644
>> index 0000000000..789f410d71
>> --- /dev/null
>> +++ b/Documentation/technical/midx.txt
>> @@ -0,0 +1,109 @@
>> +Multi-Pack-Index (MIDX) Design Notes
>> +====================================
>> +
>> +The Git object directory contains a 'pack' directory containing
>> +packfiles (with suffix ".pack") and pack-indexes (with suffix
>> +".idx"). The pack-indexes provide a way to lookup objects and
>> +navigate to their offset within the pack, but these must come
>> +in pairs with the packfiles. This pairing depends on the file
>> +names, as the pack-index differs only in suffix with its pack-
>> +file. While the pack-indexes provide fast lookup per packfile,
>> +this performance degrades as the number of packfiles increases,
>> +because abbreviations need to inspect every packfile and we are
>> +more likely to have a miss on our most-recently-used packfile.
>> +For some large repositories, repacking into a single packfile
>> +is not feasible due to storage space or excessive repack times.
> This leads to the question how MIDX will cope with large repos or
> a large number of packs. As it is just an index and not a pack itself,
> I guess it is smaller by some orders of magnitude, such that it
> is ok for now.

The MIDX file is only slightly larger than the union of the IDX files 
for those packfiles.

>> +The multi-pack-index (MIDX for short) stores a list of objects
>> +and their offsets into multiple packfiles. It contains:
>> +
>> +- A list of packfile names.
>> +- A sorted list of object IDs.
>> +- A list of metadata for the ith object ID including:
>> +  - A value j referring to the jth packfile.
>> +  - An offset within the jth packfile for the object.
>> +- If large offsets are required, we use another list of large
>> +  offsets similar to version 2 pack-indexes.
>> +
>> +Thus, we can provide O(log N) lookup time for any number
>> +of packfiles.
> This sounds great for the lookup case!
> Though that is for the repo-read case.
> Let's read on how the dynamics of a repository are dealt with,
> e.g. integrating new packs into the MIDX, or how we deal with
> objects in multiple packs.
>
>> +
>> +Design Details
>> +--------------
>> +
>> +- The MIDX is stored in a file named 'multi-pack-index' in the
>> +  .git/objects/pack directory. This could be stored in the pack
>> +  directory of an alternate. It refers only to packfiles in that
>> +  same directory.
> So there is one and only one multi pack index?
> That makes the case of preparing the next MIDX that contains more
> pack references more interesting, as then we have to atomically update
> that file.

There is only one, but we can make the file incremental without changing 
this name similar to how the split index works.

>
>> +- The core.midx config setting must be on to consume MIDX files.
> Looking through current config options, I would rename this to a more
> suggestive name. I searched for the core.idx counterpart that enables
> idx files -- it turns out that is named pack.indexVersion.
>
> So maybe pack.MultiIndex ? That could start out as a boolean as in this
> series and then evolve into a version number or such later.

I'll use that name and rename this file to 
Documentation/technical/multi-pack-index.txt

>
>> +- The file format includes parameters for the object ID hash
>> +  function, so a future change of hash algorithm does not require
>> +  a change in format.
>> +
>> +- The MIDX keeps only one record per object ID. If an object appears
>> +  in multiple packfiles, then the MIDX selects the copy in the most-
>> +  recently modified packfile.
> Okay. That answers the question from above. Though this is just the tie
> breaking decision and not a hard limitation? (i.e. we could change this
> this later to that pack that has e.g. shortest delta chain for that object or
> such)

This is a soft requirement. It is an easy thing to track at the moment. 
We can compute the MIDX without opening a packfile, for instance.

>
>> +- If there exist packfiles in the pack directory not registered in
>> +  the MIDX, then those packfiles are loaded into the `packed_git`
>> +  list and `packed_git_mru` cache.
> Not sure I understand the implications of this?
> Does that mean we first look at the multi index and if an object is not
> found, we'll search linearly through all packs that are not part of the
> MIDX? That would require the MIDX to be kepot up to date reasonably
> to be useful.

If you add a packfile to the pack directory, you can immediately start 
consuming it. You do not need to wait for the MIDX to be updated. The 
more asynchronous these auxiliary data structures (MIDX, commit-graph) 
can be, the better. This is in direct contrast to the reachability 
bitmap which is useless without its corresponding packfile.

>
>> +- The pack-indexes (.idx files) remain in the pack directory so we
>> +  can delete the MIDX file, set core.midx to false, or downgrade
>> +  without any loss of information.
> In the future will it be possible to have no .idx files and just have the .midx?
> (I guess that depends on the strategy of how to integrate new packs into
> the MIDX?)

This may be part of a future plan, but we need to know a user will never 
set pack.multiIndex to false if they deleted their IDX files.

>> +- The MIDX file format uses a chunk-based approach (similar to the
>> +  commit-graph file) that allows optional data to be added.
> ... or the index files v2 (or reftable files)? Sure, you are most familiar with
> commit-graph files, but others may find it easier to have some older
> file formats to relate to.

I specifically mean that we have a "table of contents" describing the 
list of chunks. IDX v2 relies on a fixed ordering of the tables, and the 
offsets are computed by consuming the last fanout value (number of 
objects). Also, I'm not sure how to add optional data (data that can 
safely be ignored by an earlier version of Git) to an IDX without 
incrementing the version.

>> +Future Work
>> +-----------
>> +
>> +- Add a 'verify' subcommand to the 'git midx' builtin to verify the
>> +  contents of the multi-pack-index file match the offsets listed in
>> +  the corresponding pack-indexes.
>> +
>> +- The multi-pack-index allows many packfiles, especially in a context
>> +  where repacking is expensive (such as a very large repo), or
>> +  unexpected maintenance time is unacceptable (such as a high-demand
>> +  build machine).
> Supposedly maintenance (git gc) can be run in the background without
> interfering with day-to-day life, how is the regeneration of commit graph
> or MIDX files impacting the work here?

Assuming infinite RAM and disk, then yes we could not interfere with 
daily life. A big problem we see is that users frequently don't have the 
disk space to store a second copy of their packfiles on disk, even if we 
could organize a new packfile in reasonable time.

The MIDX is only invalid when a packfile it references is deleted.

The commit-graph is never invalid, except if a commit is deleted by GC. 
But even in that case, how did we "see" the commit ID? As long as we 
don't access these commits, the commit-graph feature doesn't violate 
expectations and can be generated asynchronously with a GC and repack.

>
>>      However, the multi-pack-index needs to be rewritten
>> +  in full every time. We can extend the format to be incremental, so
>> +  writes are fast. By storing a small "tip" multi-pack-index that
>> +  points to large "base" MIDX files, we can keep writes fast while
>> +  still reducing the number of binary searches required for object
>> +  lookups.
> So we can have multiple MIDX files? How would that work? Would there
> be a chunk that refers to other MIDX files?

We can have an optional chunk that refers to a list of "base" MIDX 
files. We then load that full list into multiple 'midxed_git' structs, 
and iterate through the list. VSTS keeps this list to a maximum length 
of 3 (LARGE, Medium, tiny) and merging files as necessary.

>
>> +- The reachability bitmap is currently paired directly with a single
>> +  packfile, using the pack-order as the object order to hopefully
>> +  compress the bitmaps well using run-length encoding. This could be
>> +  extended to pair a reachability bitmap with a multi-pack-index. If
>> +  the multi-pack-index is extended to store a "stable object order"
>> +  (a function Order(hash) = integer that is constant for a given hash,
> This stable object order doesn't fly well with integrating new packs?

When you integrate new packs, the lexicographic order changes as the new 
objects are inserted into the list. However, if we track a separate 
integer value (order[obj]) associated with the object, and keep that 
constant, we can track a stable order for an object across multiple 
generations of MIDX files. New objects are assigned order values larger 
than the previous order values. We can select a "good" ordering of the 
new objects as we extend the list.

This requires a new chunk in the file format. It also helps to store the 
reverse-lookup lex[i] which provides the lexicographic position of the 
object 'obj' with stable-order order[obj] == i.

I'm being intentionally vague in this document to hint towards a 
valuable feature without giving robust details of something that may 
never get built. But, I do think this is one feature of the MIDX that 
would be of the most value for services that use Git as a server 
process, as it allows the reachability bitmap to be set to this stable 
order instead of a single pack order. This is speculation on my part, as 
I don't know how such servers are maintained in the background.

>
>> +  even as the multi-pack-index is updated) then a reachability bitmap
>> +  could point to a multi-pack-index and be updated independently.
>> +
>> +- Packfiles can be marked as "special" using empty files that share
>> +  the initial name but replace ".pack" with ".keep" or ".promisor".
>> +  We can add an optional chunk of data to the multi-pack-index that
>> +  records flags of information about the packfiles. This allows new
>> +  states, such as 'repacked' or 'redeltified', that can help with
>> +  pack maintenance in a multi-pack environment. It may also be
>> +  helpful to organize packfiles by object type (commit, tree, blob,
>> +  etc.) and use this metadata to help that maintenance.
>> +
>> +- The partial clone feature records special "promisor" packs that
>> +  may point to objects that are not stored locally, but available
>> +  on request to a server. The multi-pack-index does not currently
>> +  track these promisor packs.


  reply index

Thread overview: 192+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-06-07 14:03 [PATCH 00/23] Multi-pack-index (MIDX) Derrick Stolee
2018-06-07 14:03 ` [PATCH 01/23] midx: add design document Derrick Stolee
2018-06-11 19:04   ` Stefan Beller
2018-06-18 18:48     ` Derrick Stolee [this message]
2018-06-07 14:03 ` [PATCH 02/23] midx: add midx format details to pack-format.txt Derrick Stolee
2018-06-11 19:19   ` Stefan Beller
2018-06-18 19:01     ` Derrick Stolee
2018-06-18 19:41       ` Stefan Beller
2018-06-07 14:03 ` [PATCH 03/23] midx: add midx builtin Derrick Stolee
2018-06-07 17:20   ` Duy Nguyen
2018-06-18 19:23     ` Derrick Stolee
2018-06-11 21:02   ` Stefan Beller
2018-06-18 19:40     ` Derrick Stolee
2018-06-18 19:55       ` Stefan Beller
2018-06-18 19:58         ` Derrick Stolee
2018-06-07 14:03 ` [PATCH 04/23] midx: add 'write' subcommand and basic wiring Derrick Stolee
2018-06-07 17:27   ` Duy Nguyen
2018-06-07 14:03 ` [PATCH 05/23] midx: write header information to lockfile Derrick Stolee
2018-06-07 17:35   ` Duy Nguyen
2018-06-12 15:00   ` Duy Nguyen
2018-06-19 12:54     ` Derrick Stolee
2018-06-19 14:59       ` Duy Nguyen
2018-06-19 15:24         ` Derrick Stolee
2018-06-07 14:03 ` [PATCH 06/23] midx: struct midxed_git and 'read' subcommand Derrick Stolee
2018-06-07 17:54   ` Duy Nguyen
2018-06-20 13:13     ` Derrick Stolee
2018-06-07 18:31   ` Duy Nguyen
2018-06-20 13:33     ` Derrick Stolee
2018-06-20 15:07       ` Duy Nguyen
2018-06-20 16:39         ` Derrick Stolee
2018-06-07 14:03 ` [PATCH 07/23] midx: expand test data Derrick Stolee
2018-06-07 14:03 ` [PATCH 08/23] midx: read packfiles from pack directory Derrick Stolee
2018-06-07 18:03   ` Duy Nguyen
2018-06-20 16:33     ` [PATCH] packfile: generalize pack directory list Derrick Stolee
2018-06-07 14:03 ` [PATCH 09/23] midx: write pack names in chunk Derrick Stolee
2018-06-07 18:26   ` Duy Nguyen
2018-06-21 15:25     ` Derrick Stolee
2018-06-21 17:38       ` Junio C Hamano
2018-06-22 18:25         ` Derrick Stolee
2018-06-22 18:31           ` Junio C Hamano
2018-06-22 18:32             ` Derrick Stolee
2018-06-07 14:03 ` [PATCH 10/23] midx: write a lookup into the pack names chunk Derrick Stolee
2018-06-09 16:43   ` Duy Nguyen
2018-06-21 17:23     ` Derrick Stolee
2018-06-07 14:03 ` [PATCH 11/23] midx: sort and deduplicate objects from packfiles Derrick Stolee
2018-06-09 17:07   ` Duy Nguyen
2018-06-21 17:54     ` Derrick Stolee
2018-06-07 14:03 ` [PATCH 12/23] midx: write object ids in a chunk Derrick Stolee
2018-06-09 17:25   ` Duy Nguyen
2018-06-07 14:03 ` [PATCH 13/23] midx: write object id fanout chunk Derrick Stolee
2018-06-09 17:28   ` Duy Nguyen
2018-06-21 19:49     ` Derrick Stolee
2018-06-07 14:03 ` [PATCH 14/23] midx: write object offsets Derrick Stolee
2018-06-09 17:41   ` Duy Nguyen
2018-06-07 14:03 ` [PATCH 15/23] midx: create core.midx config setting Derrick Stolee
2018-06-07 14:03 ` [PATCH 16/23] midx: prepare midxed_git struct Derrick Stolee
2018-06-09 17:47   ` Duy Nguyen
2018-06-07 14:03 ` [PATCH 17/23] midx: read objects from multi-pack-index Derrick Stolee
2018-06-09 17:56   ` Duy Nguyen
2018-06-21 20:03     ` Derrick Stolee
2018-06-07 14:03 ` [PATCH 18/23] midx: use midx in abbreviation calculations Derrick Stolee
2018-06-09 18:01   ` Duy Nguyen
2018-06-22 18:38     ` Derrick Stolee
2018-06-07 14:03 ` [PATCH 19/23] midx: use existing midx when writing new one Derrick Stolee
2018-06-07 14:03 ` [PATCH 20/23] midx: use midx in approximate_object_count Derrick Stolee
2018-06-09 18:03   ` Duy Nguyen
2018-06-22 18:39     ` Derrick Stolee
2018-06-07 14:03 ` [PATCH 21/23] midx: prevent duplicate packfile loads Derrick Stolee
2018-06-09 18:05   ` Duy Nguyen
2018-06-07 14:03 ` [PATCH 22/23] midx: use midx to find ref-deltas Derrick Stolee
2018-06-07 14:03 ` [PATCH 23/23] midx: clear midx on repack Derrick Stolee
2018-06-09 18:13   ` Duy Nguyen
2018-06-22 18:44     ` Derrick Stolee
2018-06-07 14:06 ` [PATCH 00/23] Multi-pack-index (MIDX) Derrick Stolee
2018-06-07 14:45 ` Ævar Arnfjörð Bjarmason
2018-06-07 14:54   ` Derrick Stolee
2018-06-25 14:34 ` [PATCH v2 00/24] " Derrick Stolee
2018-06-25 14:34   ` [PATCH v2 01/24] multi-pack-index: add design document Derrick Stolee
2018-06-25 14:34   ` [PATCH v2 02/24] multi-pack-index: add format details Derrick Stolee
2018-06-25 14:34   ` [PATCH v2 03/24] multi-pack-index: add builtin Derrick Stolee
2018-06-25 19:15     ` Junio C Hamano
2018-06-25 14:34   ` [PATCH v2 04/24] multi-pack-index: add 'write' verb Derrick Stolee
2018-06-25 14:34   ` [PATCH v2 05/24] midx: write header information to lockfile Derrick Stolee
2018-06-25 19:19     ` Junio C Hamano
2018-07-05 19:13       ` Derrick Stolee
2018-06-25 14:34   ` [PATCH v2 06/24] multi-pack-index: load into memory Derrick Stolee
2018-06-25 19:38     ` Junio C Hamano
2018-07-05 14:19       ` Derrick Stolee
2018-07-05 18:58         ` Eric Sunshine
2018-07-06 19:20           ` Junio C Hamano
2018-06-25 14:34   ` [PATCH v2 07/24] multi-pack-index: expand test data Derrick Stolee
2018-06-25 19:45     ` Junio C Hamano
2018-06-25 14:34   ` [PATCH v2 08/24] packfile: generalize pack directory list Derrick Stolee
2018-06-25 19:57     ` Junio C Hamano
2018-06-25 14:34   ` [PATCH v2 09/24] multi-pack-index: read packfile list Derrick Stolee
2018-06-25 14:34   ` [PATCH v2 10/24] multi-pack-index: write pack names in chunk Derrick Stolee
2018-06-25 14:34   ` [PATCH v2 11/24] midx: read pack names into array Derrick Stolee
2018-06-25 23:52     ` Eric Sunshine
2018-06-25 14:34   ` [PATCH v2 12/24] midx: sort and deduplicate objects from packfiles Derrick Stolee
2018-06-25 14:34   ` [PATCH v2 13/24] midx: write object ids in a chunk Derrick Stolee
2018-06-25 14:34   ` [PATCH v2 14/24] midx: write object id fanout chunk Derrick Stolee
2018-06-25 14:34   ` [PATCH v2 15/24] midx: write object offsets Derrick Stolee
2018-06-25 14:34   ` [PATCH v2 16/24] config: create core.multiPackIndex setting Derrick Stolee
2018-06-25 14:34   ` [PATCH v2 17/24] midx: prepare midxed_git struct Derrick Stolee
2018-06-25 14:34   ` [PATCH v2 18/24] midx: read objects from multi-pack-index Derrick Stolee
2018-06-25 14:34   ` [PATCH v2 19/24] midx: use midx in abbreviation calculations Derrick Stolee
2018-06-25 14:34   ` [PATCH v2 20/24] midx: use existing midx when writing new one Derrick Stolee
2018-06-25 14:34   ` [PATCH v2 21/24] midx: use midx in approximate_object_count Derrick Stolee
2018-06-25 14:34   ` [PATCH v2 22/24] midx: prevent duplicate packfile loads Derrick Stolee
2018-06-25 14:34   ` [PATCH v2 23/24] packfile: skip loading index if in multi-pack-index Derrick Stolee
2018-06-25 14:34   ` [PATCH v2 24/24] midx: clear midx on repack Derrick Stolee
2018-07-06  0:52   ` [PATCH v3 00/24] Multi-pack-index (MIDX) Derrick Stolee
2018-07-06  0:52     ` [PATCH v3 01/24] multi-pack-index: add design document Derrick Stolee
2018-07-06  0:52     ` [PATCH v3 02/24] multi-pack-index: add format details Derrick Stolee
2018-07-06  0:53     ` [PATCH v3 03/24] multi-pack-index: add builtin Derrick Stolee
2018-07-06  3:54       ` Eric Sunshine
2018-07-06  0:53     ` [PATCH v3 04/24] multi-pack-index: add 'write' verb Derrick Stolee
2018-07-06  4:07       ` Eric Sunshine
2018-07-06  0:53     ` [PATCH v3 05/24] midx: write header information to lockfile Derrick Stolee
2018-07-06  0:53     ` [PATCH v3 06/24] multi-pack-index: load into memory Derrick Stolee
2018-07-06  4:19       ` Eric Sunshine
2018-07-06  5:18         ` Eric Sunshine
2018-07-09 19:08       ` Junio C Hamano
2018-07-12 16:06         ` Derrick Stolee
2018-07-06  0:53     ` [PATCH v3 07/24] multi-pack-index: expand test data Derrick Stolee
2018-07-06  4:36       ` Eric Sunshine
2018-07-06  5:20         ` Eric Sunshine
2018-07-12 14:10         ` Derrick Stolee
2018-07-12 18:02           ` Eric Sunshine
2018-07-12 18:06             ` Derrick Stolee
2018-07-06  0:53     ` [PATCH v3 08/24] packfile: generalize pack directory list Derrick Stolee
2018-07-06  0:53     ` [PATCH v3 09/24] multi-pack-index: read packfile list Derrick Stolee
2018-07-06  0:53     ` [PATCH v3 10/24] multi-pack-index: write pack names in chunk Derrick Stolee
2018-07-06  0:53     ` [PATCH v3 11/24] midx: read pack names into array Derrick Stolee
2018-07-06  4:58       ` Eric Sunshine
2018-07-06  0:53     ` [PATCH v3 12/24] midx: sort and deduplicate objects from packfiles Derrick Stolee
2018-07-06  0:53     ` [PATCH v3 13/24] midx: write object ids in a chunk Derrick Stolee
2018-07-06  5:04       ` Eric Sunshine
2018-07-06  0:53     ` [PATCH v3 14/24] midx: write object id fanout chunk Derrick Stolee
2018-07-06  0:53     ` [PATCH v3 15/24] midx: write object offsets Derrick Stolee
2018-07-06  5:27       ` Eric Sunshine
2018-07-12 16:33         ` Derrick Stolee
2018-07-06  0:53     ` [PATCH v3 16/24] config: create core.multiPackIndex setting Derrick Stolee
2018-07-06  5:39       ` Eric Sunshine
2018-07-12 13:19         ` Derrick Stolee
2018-07-12 16:30           ` Derrick Stolee
2018-07-11  9:48       ` SZEDER Gábor
2018-07-12 13:01         ` Derrick Stolee
2018-07-12 13:31           ` SZEDER Gábor
2018-07-12 15:40             ` Derrick Stolee
2018-07-12 17:29             ` Junio C Hamano
2018-07-06  0:53     ` [PATCH v3 17/24] midx: prepare midxed_git struct Derrick Stolee
2018-07-06  5:41       ` Eric Sunshine
2018-07-06  0:53     ` [PATCH v3 18/24] midx: read objects from multi-pack-index Derrick Stolee
2018-07-06  0:53     ` [PATCH v3 19/24] midx: use midx in abbreviation calculations Derrick Stolee
2018-07-06  0:53     ` [PATCH v3 20/24] midx: use existing midx when writing new one Derrick Stolee
2018-07-06  0:53     ` [PATCH v3 21/24] midx: use midx in approximate_object_count Derrick Stolee
2018-07-06  0:53     ` [PATCH v3 22/24] midx: prevent duplicate packfile loads Derrick Stolee
2018-07-06  0:53     ` [PATCH v3 23/24] packfile: skip loading index if in multi-pack-index Derrick Stolee
2018-07-06  0:53     ` [PATCH v3 24/24] midx: clear midx on repack Derrick Stolee
2018-07-06  5:52       ` Eric Sunshine
2018-07-12 19:39     ` [PATCH v4 00/23] Multi-pack-index (MIDX) Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 01/23] multi-pack-index: add design document Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 02/23] multi-pack-index: add format details Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 03/23] multi-pack-index: add builtin Derrick Stolee
2018-07-20 18:22         ` Junio C Hamano
2018-07-20 22:15           ` brian m. carlson
2018-07-20 22:28             ` Junio C Hamano
2018-07-12 19:39       ` [PATCH v4 04/23] multi-pack-index: add 'write' verb Derrick Stolee
2018-07-12 22:56         ` Eric Sunshine
2018-07-12 19:39       ` [PATCH v4 05/23] midx: write header information to lockfile Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 06/23] multi-pack-index: load into memory Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 07/23] t5319: expand test data Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 08/23] packfile: generalize pack directory list Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 09/23] multi-pack-index: read packfile list Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 10/23] multi-pack-index: write pack names in chunk Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 11/23] midx: read pack names into array Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 12/23] midx: sort and deduplicate objects from packfiles Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 13/23] midx: write object ids in a chunk Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 14/23] midx: write object id fanout chunk Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 15/23] midx: write object offsets Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 16/23] config: create core.multiPackIndex setting Derrick Stolee
2018-07-12 21:05         ` Junio C Hamano
2018-07-13  0:50           ` Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 17/23] midx: read objects from multi-pack-index Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 18/23] midx: use midx in abbreviation calculations Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 19/23] midx: use existing midx when writing new one Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 20/23] midx: use midx in approximate_object_count Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 21/23] midx: prevent duplicate packfile loads Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 22/23] packfile: skip loading index if in multi-pack-index Derrick Stolee
2018-07-12 19:39       ` [PATCH v4 23/23] midx: clear midx on repack Derrick Stolee
2018-07-12 21:11       ` [PATCH v4 00/23] Multi-pack-index (MIDX) Junio C Hamano

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=e251bcff-8638-19d8-2937-0e913b19215a@gmail.com \
    --to=stolee@gmail.com \
    --cc=avarab@gmail.com \
    --cc=dstolee@microsoft.com \
    --cc=git@vger.kernel.org \
    --cc=jonathantanmy@google.com \
    --cc=jrnieder@gmail.com \
    --cc=mfick@codeaurora.org \
    --cc=sbeller@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org list mirror (unofficial, one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox