Re: [RFC/WIP] Pluggable reference backends

[Jeff King <peff@peff.net>]

On Mon, Mar 10, 2014 at 07:30:45AM -0700, Shawn Pearce wrote:

> > * Store references in a SQLite database, to get correct transaction
> >   handling.
> 
> No to SQLLite in git-core. Using it from JGit requires building
> SQLLite and a JNI wrapper, which makes JGit significantly less
> portable. I know SQLLite is pretty amazing, but implementing
> compatibility with it from JGit will be a big nightmare for us.

That seems like a poor reason not to implement a pluggable feature for
git-core. If we implement it, then a site using only git-core can take
advantage of it. Sites with JGit cannot, and would use a different
pluggable storage mechanism that's supported by both. But if we don't
implement, it hurts people using only git-core, and it does not help
sites using JGit at all.

That's assuming that attention spent on implementing the feature does
not take away from implementing some other parallel scheme that does the
same thing but does not use SQLite. I don't know what that would be
offhand; mapping the ref and reflog into a relational database is pretty
simple, and we get a lot of robustness and efficiency benefits for free.
We could perhaps have some kind of "relational" backend could use an
ODBC-like abstraction to point to a database. I have no idea if people
would want to ever store refs in a "real" server-backend RDBMS, but I
suspect Java has native support for such things.

Certainly I think we should aim for compatibility where we can, but if
there's not a compatible way to do something, I don't think the
limitations of one platform should drag other ones down. And that goes
both ways; we had to reimplement disk-compatible EWAH from scratch in C
for git-core to have bitmaps, whereas JGit just got to use a ready-made
library. I don't think that was a bad thing.  People in
mixed-implementation environments couldn't use it, but people with
JGit-only environments were free to take advantage of it.

At any rate, the repository needs to advertise "this is the ref storage
mechanism I use" in the config. We're going to need to bump
core.repositoryformatversion for such cases (because an old version of
git should not blindly lock and write to a refs/ directory that nobody
else is ever going to look at). And I'd suggest with that bump adding in
something like core.refstorage, so that an implementation can say
"foobar ref storage? Never heard of it" and barf. Whether it's because
that implementation doesn't support "foobar", because it's an old
version that doesn't understand "foobar" yet, or because it was simply
built without "foobar" support.

-Peff