Re: [Discussion] The architecture of Scalar (and others) within Git

[Junio C Hamano <gitster@pobox.com>]

Derrick Stolee <stolee@gmail.com> writes:

> Optimization Factors
> --------------------
>
> There are multiple factors involved that we wish to optimize. Here is my
> understanding of the most-important factors and how they interact:
>
> ### Value
>
> The main reason to include Scalar within the Git codebase is to make it
> available to more users. This should have downstream effects based on user
> feedback about what they like or dislike about the Scalar CLI that can
> inform future features in core Git that simplify repository management at
> scale.

In other words, Scalar wants to piggyback on Git's popularity, yet
wants to retain total control of how its end-user experience should
look like, eating the cake and having it at the same time?

I do not think it is fair to proceed with this discussion without
having "Scalar lives in its own tree, uses this mailing list to
discuss how it should evolve, drawing expertise from the
contributors found here, but otherwise would stay to be a spearate
project" as another option and debating its pros-and-cons, next to
the spectrum among contrib/scalar, scalar/, and scalar.c approaches.
IIUC, cgit has one-directional coupling to depend on libgit.a just
like the Scalar wants to link with it---I am not saying Scalar should
do the same, but just raising it as an example that you do *not* have
to live in my tree anywhere to be able to link with libgit.a; that
would probably count as an "**Option 0:** A separate project that
borrows from Git by having it as a submodule".

I can think of only one obvious advantage going that route, and that
is why I said "I am not saying Scalar should do the same", which is
that the Scalar project will then retain total control over how it
is built and how its end-user experience should stay the same.

I do not think it is an option to refuse to take any input from
others on this list on parts of what you have already done, if you
want to be in my tree, anywhere, not even in contrib/.

Just like we say "the plumbing command X has behaved this way
forever, and we cannot break the backward compatibility" to some
possible changes to any of our tools, however, "Scalar has been used
with that subcommand behaving this particular way even before the
code was donated to Git project, so we cannot lightly change it"
would equally be a good input when we evaluate when some things in
it may need to be changed, so I do not think it would pose a great
backward compatibility issue (after all, we do not change things
lightly just for the sake of changing it).

But approaching our developers with a new piece of code, saying "I
want to piggyback on your popularity.  Among the stuff given to you,
some are out of limit and you may not even discuss about modifying
it" upfront, is a problematic attitude.

Now, the goal from purely Git project's point of view would be to
improve Git's user experience, not necessarily Scalar's, by exposing
more of our users to the way its opinionated (which is *not*
necessarily a bad thing) decisions work to help its users, and our
developers to the scalar codebase.  Some of our developers may not
even agree with the design decisions Scalar made and that is
expected and is fine; based on such a reaction, the lesson Git
learns from Scalar to solve a similar issue may turn out to be solve
it differently.

I do not know how much Options 1-3 would help such cross pollination
more over the Option 0, offhand, though.


> Possible approaches to land in our final target
> -----------------------------------------------

Here I sense the "final target" is somehow Scalar is delivered to
end-users through my tree and being maintained there, but from my
point of view, that is merely an early step.  Wasn't the final
target for "git" to learn from Scalar's successes and mistakes and
give native support to itches Scalar wanted to scratch?

And if that is indeed the goal, when it is achieved, Scalar would
become a historical curiosity just like Cogito is today.  With
Cogito, we cross-pollinated just fine without having both in the
same tree at all, and when Cogito outlived its usefulness, we did
not have to remove anything from our tree.  With cgit, we did not
cross-pollinate at all, unfortunately, and gitweb more or less stays
the same way as it was years ago, and cgit is still a project with a
useful product.  We can reach both extremes even if we take Option
0.  But it would have been messy for something like Cogito, which
was an early UI and workflow experiment, if we took a more
integrated approach.

> **Phase 1.** Keep code in contrib/ while we contribute the Scalar features.
>
> Since the current patches on the mailing list are not feature complete, it
> can be beneficial to move forward with the code in `contrib/scalar/` until
> we reach feature parity. At that point, we could move the source into its
> final resting place.
>
> This leans into the fact that `contrib/` contains "interesting tools...
> maybe even experimental ones". While Scalar is not feature complete, it
> can be isolated as experimental here until it is ready for promotion to
> non-experimental.

I personally have no problem with this approach, but with less
scrutiny, being in contrib/ longer, with "some stuff are not subject
to discussion" attitude, would inevitably lower the code quality of
the product, and it is dubious it would become "ready for promotion"
ever.  The contrib/ hierarchy, especially in early days when we
wanted a way to grow the ecosystem, as a place to host "these are
done externally, and we only carry them for convenience", was a
successful model, but I do not think of anything that successfully
used contrib/ as a nursery to later graduate to the core and to be
seen as the same quality as other things that started their life in
core.  I am pessimistic to the model that uses contrib/ as such a
place.

> Depending on the final goal, we could drop some of the work that is currently
> built within the `contrib/scalar/` directory, such as `-c`/`-C` parsing or
> documentation builds. These features would be reimplemented in the new
> location, so we can prevent that duplicate effort if we have a different
> final location in mind.

> **Phase 2.** Establish community standards on optional components
>
> While the work in `contrib/scalar/` continues to reach feature parity, we
> can work as a community to set some ground rules about these kinds of
> optional features (depending on how "optional" we land). This can include
> standards such as how the files are laid out in the repository and how
> they interact with the `Makefile`. Whatever we do for Scalar is likely to
> form an example for a future contribution that we can't predict today.
> Many of the questions we are asking today can be written for posterity:
>
> * What is an appropriate level of coupling with the Git codebase?
> * Where should code custom to the component live?
> * How should documentation and tests be organized?
> * How do we initialize builds of a component?
> * Who is responsible for supporting the component? Who fixes the bugs?
> * Who is responsible for reviewing changes to the component?

I am afraid that the answer to the first four are "it depends" (but
the remaining two would fall out naturally out of them).

Thanks.  My brain is fried with the -rc work, rereading the material
that went to -rc0 since the last release, so I may have more to add
later, but for now, the above is what came to my mind.