Re: [PATCH] pack-format: correct multi-pack-index description

[Johannes Berg <johannes@sipsolutions.net>]

On Mon, 2020-02-10 at 09:46 -0500, Derrick Stolee wrote:

> Part of my initial plan was to have this incremental file format.
> The commit-graph uses a very similar mechanism. The difference may
> be that you likely allow multiple .midx files found by scanning the
> pack directory, 

Right, just scan and use any midx that exist, then compare the packs in
there against all the packs found, and then remove any packs that
actually *are* in an midx from the search list. That leaves you with all
information, but optimised by midx where possible.

> but I would expect something like the
> "commit-graph-chain" file that provides an ordered list of the
> incremental files. This can be important for deciding when to merge
> layers or delete old files, and would be critical to the possibility
> of converting reachability bitmaps to rely on a stable object order
> stored in the multi-pack-index instead of pack-order.

Right, if we delete then we have to also remove any midx covering the
deleted pack, that's pretty rare in bup as a backup tool though.

> The reason the multi-pack-index has not become incremental is that
> VFS for Git no longer needs to write it very often. We write the
> entire multi-pack-index during a background job that triggers once
> per day. If we needed to write it more frequently, then the incremental
> format would be more important to us.

So, wait, what if a new pack is created? Does it just get used in
addition to the multi-pack-index, if it's not covered by it, like I
described above?

If so, I guess it wouldn't actually really matter here. I was afraid
(but didn't check yet) that git would always use only the single multi-
pack-index file, and not also search additional packs, so that it always
has to be maintained in "perfect order" ...

> That said: if someone wanted to contribute an incremental format,
> then I would be happy to review it!

I might still get motivated to do so :-)

johannes