Re: reftable [v7]: new ref storage format

[Stefan Beller <sbeller@google.com>]

On Tue, Aug 15, 2017 at 7:48 PM, Shawn Pearce <spearce@spearce.org> wrote:
> 7th iteration of the reftable storage format.
>
> You can read a rendered version of this here:
> https://googlers.googlesource.com/sop/jgit/+/reftable/Documentation/technical/reftable.md
>
> Changes from v6:
> - Blocks are variable sized, and alignment is optional.
> - ref index is required on variable sized multi-block files.
>
> - restart_count/offsets are again at the end of the block.
> - value_type = 0x3 is only for symbolic references.
> - "other" files cannot be stored in reftable.
>
> - object blocks are explicitly optional.
> - object blocks use position (offset in bytes), not block id.
> - removed complex log_chained format for log blocks
>
> - Layout uses log, ref file extensions
> - Described reader algorithm to obtain a snapshot

- back to the old "intra-block index is last"
  for all block types. ok.
- changed (only ref?) indexes to start char + 3 byte size:
  Which starting char do object/log indexes have?

"Unaligned files must include the ref index to support fast lookup."

Why this? I would imagine the client (which has ~5 branches),
would not need this, but only a ref block, that's it.

Ctrl-F for 'block_size' reveals nothing is computed
relative to the block_size in this format, yet we can
set it to an arbitrary number. If following the spec,
the reader at $DAY_JOB needs to be able to read
both aligned and unaligned reftables, despite our plan
to ever write aligned ref tables, what would the reader
use the block_size for? (I think we can omit that field
from the header/footer now, no?)