Re: reftable & jgit compatibility

[Patrick Steinhardt <ps@pks.im>]

[-- Attachment #1: Type: text/plain, Size: 2139 bytes --]

On Thu, Apr 04, 2024 at 08:44:44AM +0200, Han-Wen Nienhuys wrote:
> On Thu, Apr 4, 2024 at 8:23 AM Patrick Steinhardt <ps@pks.im> wrote:
> 
> > > I think the easiest way to make this happen is if CGit would ship a
> > > command to dump a raw reftable in a release soonish. Then JGit could
> > > use that command to cross-check that a JGit-written reftable can be
> > > read correctly by the CGit code.  By shipping just the dumper you
> > > avoid having to wait for proper reftable support to land in git.
> >
> > You do realize that "proper reftable support" has already landed, right?
> 
> I had not realized this, and that's great news!
> 
> > So you can just use Git to create a reftable-enabled repository, write
> > commits and then use JGit to access the whole repository instead of only
> > checking a single table.
> 
> For testing, it's probably easier if you can work in terms of
> individual tables (because that is where the complexity lies:
> different blocksizes, restart frequencies, with index, without index,
> with reflog, without reflog etc.), but one can create controlled
> individual tables by creating a whole repo and then compacting it.
> OTOH, this would necessitate exposing all writer options to the git
> CLI, which is maybe a bit much.

Potentially, yeah. But as you say, it's likely quite some complexity to
expose this via the CLI directly. So for now, I'm going to focus on some
basic interoperability tests in Git that act on the repository level. We
can build on that and expand them as required when the need arises.

Different blocksizes is definitely a bit of a sore spot right now. I do
plan to expose write options via Git config options in the future, e.g.
something like "reftable.blockSize" or "reftable.restartCount". But for
all I know the CGit reftable library doesn't yet play nice with block
sizes other than 4k.

I didn't yet want to introduce configs which are specific to reftables
in the first release of Git with the reftable backend, so I pushed this
issue further down. I do plan to work on that in the next release cycle
though.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]