Re: [PATCH 00/38] Virtualization of the refs API

[Michael Haggerty <mhagger@alum.mit.edu>]

On 07/10/2016 05:09 PM, Duy Nguyen wrote:
> On Fri, Jun 3, 2016 at 11:03 PM, Michael Haggerty <mhagger@alum.mit.edu> wrote:
>> Since the that ref-iterator [1] changes seem to have gotten a positive
>> reception, let's try to keep up the momentum. I hope I'm not
>> overloading the review pipeline...
>>
>> I think all of the groundwork is in place now to virtualize the refs
>> API. This will open the way to storing refs in ways other than the
>> familiar loose refs / packed refs format, such as David Turner's
>> proposed LMDB-based storage [2].
>>
>> This is a long patch series, but most of the patches are pretty simple
>> and formulaic. The goal is to implement a `ref_store`. In the language
>> of object-oriented programming, `ref_store` is an abstract base class
>> representing a reference storage backend. It provides methods to read,
>> write, and delete references and symrefs, and to iterate over
>> references, reflogs, and reflog entries, plus a number of other
>> things—19 methods in all.
> 
> I probably don't know what I'm talking about because I don't follow
> your work closely enough. Please ignore if this is nonsense. But if we
> extend/change API, we might need to update git-for-each-ref too, to
> expose it to shell scripts and external commands. I guess for
> iteration there's nothing else more needed, but we may need to
> introduction new options for the storage thing, e.g. to select
> storage...

This patch series doesn't change the external API in any significant
way. It only wraps it up on a virtualization layer so that a different
reference storage backends can be plugged in.

So as long as people are using plumbing commands to work with references
(rather than reading/writing files under $GIT_DIR directly), they should
notice no difference.

There are only two exceptions that I know of:

1. Users will need to be able to request that a repository use a
non-default reference backend, and (less importantly) inquire about
which reference backend a particular repository is using. Those
facilities will be added when the first non-files reference backend is
added.

2. At least one command (`git pack-refs`) is particular to the files
backend, and won't be needed (at least not in its current form) for
other backends. Conversely, it is conceivable that future reference
backends will require their own "maintenance" commands. Such commands
would be added as they are needed.

If there were operations that other reference backends could do much
more efficiently than the files backend (like, hypothetically, return
all references matching a regular expression without having to iterate
through all references), then it might make sense for performance
reasons to provide commands to access that functionality. But at the
moment I don't know of any such cases.

Michael