Re: [RFC/WIP] Pluggable reference backends

[David Kastrup <dak@gnu.org>]

Jeff King <peff@peff.net> writes:

> On Mon, Mar 10, 2014 at 05:14:02PM +0100, David Kastrup wrote:
>
>> [storing refs in sqlite]
>>
>> Of course, the basic premise for this feature is "let's assume that our
>> file and/or operating system suck at providing file system functionality
>> at file name granularity".  There have been two historically approaches
>> to that problem that are not independent: a) use Linux b) kick Linus.
>
> You didn't define "suck" here, but there are a number of issues with the
> current ref storage system. Here is a sampling:
>
>   1. The filesystem does not present an atomic view of the data (e.g.,
>      you read "a", then while you are reading "b", somebody else updates
>      "a"; your view is one that never existed at any point in time).

If there are no system calls suitable for addressing this problem that
fundamentally concerns the use of the file system as a file-name
addressed data store, I don't see why "kick Linus" would not apply here.

>   2. Using the filesystem creates D/F conflicts between branches "foo"
>      and "foo/bar". Because this name is a primary key even for the
>      reflogs, we cannot easily persist reflogs after the ref is
>      removed.

That actually sounds more like "kick Junio" territory (the wonderful
times when "kick Linus" could achieve almost anything are over).  To
wit: this sounds like a design shortcoming in Git's use of filesystems,
not something that is actually inherent in the use of files.

>   3. We use packed-refs in conjunction with loose ones to achieve
>      reasonable performance when there are a large number of refs. The
>      scheme for determining the current value of a ref is complicated
>      and error-prone (we had several race conditions that caused real
>      data loss).

Again, that sounds like we are talking about a scenario that is not a
problem of files inherently but rather of Git's ways of managing them.

> Those things can be solved through better support from the filesystem.
> But they were also solved decades ago by relational databases.

Relational databases that are not implemented on raw storage managed by
database servers will still map their operations to file operations.

> But they are also a proven technology for solving exactly the sorts of
> problems that some people are having with git. I do not see a reason
> not to consider them as an option for a pluggable refs system.

But I think it would be wrong to try solving "2." above at the database
level when its actual problem lies with the reference->filename mapping
scheme.

-- 
David Kastrup