Re: Proposal/Discussion: Turning parts of Git into libraries

[Elijah Newren <newren@gmail.com>]

On Tue, Feb 21, 2023 at 2:07 PM Emily Shaffer <nasamuffin@google.com> wrote:
>
> On Fri, Feb 17, 2023 at 8:05 PM Elijah Newren <newren@gmail.com> wrote:
> >
> > On Fri, Feb 17, 2023 at 1:45 PM Emily Shaffer <nasamuffin@google.com> wrote:
[...]
> > > So what's next? Naturally, I'm looking forward to a spirited
> > > discussion about this topic - I'd like to know which concerns haven't
> > > been addressed and figure out whether we can find a way around them,
> > > and generally build awareness of this effort with the community.
> >
> > I'm curious whether clarifying scope/layering and cleaning up
> > interfaces might mean you'd be interested in things like:
> >   * https://github.com/newren/git/commits/header-cleanups (which was
> > still WIP; I paused working on it because I figured people would see
> > it as big "cleanup" patches with no practical benefit)
> >   * https://github.com/gitgitgadget/git/pull/1149 (which has been
> > ready to submit for a _long_ time, but I just haven't yet)
> > or if these two things are orthogonal to what you have in mind.
>
> Extremely yes. :) Even "small" stuff like the need for header cleanups
> have already come up for Glen and Calvin working on config and strbuf.

Ok, I'll clean up what I've got and submit.

[...]
> > > Much, much later on, I'm expecting us to form a plan around allowing
> > > "plugins" - that is, replacing library functionality we use today with
> > > an alternative library, such as an object store relying on a
> > > distributed file store like S3. Making that work well will also likely
> > > involve us coming up with a solution for dependency injection, and to
> > > begin using vtables for some libraries. I'm hoping that we can figure
> > > out a way to do that that won't make the Git source ugly. Around this
> > > time, I think it will make sense to buy into unit tests even more and
> > > start using an approach like mocking to test various edge cases. And
> > > at some point, it's likely that we'll want to make the interfaces to
> > > various Git libraries consistent with each other, which would involve
> > > some large-scale but hopefully-mechanical refactors.
> >
> > Would these plugins resemble the pluggable merge backends that was
> > added to builtin/merge.c?  Would it replace that mechanism with a
> > different one?  Would it be more like the refs backends?
>
> I suspect that it's likely to be most similar to the refs backend
> replacement, although I investigated it only a little bit just now.
>
> The pluggable merge backends are an interesting thought - right now
> all those alternatives are built in and we decide based on config,
> right?

While we have several built in merge backends (recursive, resolve,
ort, octopus, ours, subtree), it is not limited to built-in
alternatives.  If someone creates a "git-merge-$STRATEGY" executable
and runs `git merge -s $STRATEGY` then builtin/merge.c will fork their
subcommand to try to resolve the merge.  (Users can even specify
something like `-s $STRATEGY1 -s $STRATEGY2` to have git find the best
strategy to use.)  The subcommand is then expected to update the
working tree and index and return an exit status signalling whether
the merge was clean.  It's very much built around assuming you
currently have a branch checked out and you want to merge into that
branch.

We do not know how widely used this feature is, but it's kept us from
fixing some API mistakes.  For example, a user strategy can return an
exit status of 2 signalling that it cannot even consider the merge in
question (e.g. most strategies cannot handle octopus merges).
However, the merge strategies are allowed to muck with the working
directory and index and leave them in a totally messed up state prior
to returning the "2" signalling that the given merge strategy is
inappropriate and another should be selected instead.  That means
builtin/merge.c is required after calling any given strategy and if it
does not succeed, go and "clean everything up".