Re: [PATCH 0/2] [RFC] scalar: prepare documentation for future work

[Derrick Stolee <derrickstolee@github.com>]

On 6/29/2022 12:58 PM, Victoria Dye via GitGitGadget wrote:

> Beginning with the "Philosophy of Scalar" [2], much of the existing
> documentation refers to Scalar as "opinionated" - not unlike git itself.
> However, with scalar now part of Git, continuing to describe it as
> "opinionated" could unintentionally portray it as a divergence from "the
> rest of Git" and its development community.

I appreciate this perspective.
 
> Rather than think of Scalar as an opinionated take on Git, I prefer to think
> of it as a manifestation of Git's own "opinions" for a specific use case,

This is a good reframing.

> All that to say, Scalar is essentially a collection of porcelain commands
> [3] that fill a substantial UX niche in Git. Now, let's get into Scalar's
> "future state".

(I was going to comment here about Scalar as proving ground, but...)
  
> A plan for Scalar
> =================
> 
> Given the slightly tweaked "philosophy" above, my ultimate goal for Scalar
> is to have it contain only what is too experimental or too large
> repo-focused to be a default option or behavior in Git. Over time, some
> features may be moved out of Scalar and into Git defaults as they are proven
> stable and beneficial to the vast majority of users [4].

Here you point out that the new goal is for Scalar to be a proving ground
for new features. We can more readily update the recommended Scalar config
than we can update Git defaults.

For example, Scalar's first version included setting up cone-mode sparse-
checkout when that was a brand-new feature. This new default was adopted
in core Git two years later.

> So what do we need to get there?
> 
> At a high level, the remaining work to "finalize" Scalar (past this RFC) can
> be broken into three parts:
> 
>  1. Complete a few more features and subcommands of Scalar (integrate with
>     the built-in FSMonitor & implement scalar help).

The builtin FS Monitor setting is one that has also been part of the Scalar
implementation for a while, but I also doubt that we will make it a default
setting for _all_ Git repositories. Here, we are a good proving ground for
"recommended config for large repos" until we create an alternative under
the 'git' executable.

>  2. Move stable, general-purpose parts of 'scalar.c' into other Git
>     builtins/libraries (mainly scalar diagnose, either as part of git
>     bug-report [5] or a new git built-in).
> 
>  3. Move Scalar out of contrib/ and into the "top-level" of Git. Includes
>     expanded testing, especially performance testing.

This is an interesting order change from the initial plan. I like that
we are minimizing the code footprint of Scalar before doing the move out
of contrib/. But also, this changes the complex nature of it being in
contrib/ but also wanting to change APIs in libgit.a for the sake of
Scalar.

Your step 2 is focused on changing APIs in libgit.a _for the sake of
Git commands_ and in the meantime removing code from contrib/ and calling
those APIs in libgit.a instead. I like the idea.

This does delay the conversion of Scalar out of contrib, which is worth
thinking about.

> What's Scalar's future?
> =======================
> 
> With the completion of the tasks listed in the previous section, there's
> nothing left from the original Scalar CLI (in the Microsoft fork of Git) to
> upstream. I don't consider that the "end" of supporting Scalar because, for
> it to remain an effective tool, it'll need to stay up-to-date with the
> latest performance features introduced to Git.
> 
> For example, one possible future extension might be to have Scalar enable
> the sparse index by default, especially when more built-ins are updated with
> sparse index compatibility. I'm interested in hearing what other features
> might fit well there!

I did a double-take, "We're not doing that already?" Of course, we enabled
the sparse-index by default across all of microsoft/git. We did that when
microsoft/git had early versions of the sparse index integrations that are
now in shipped versions of Git. It might be worth considering updating the
default to enable sparse index automatically for repos using cone-mode
sparse-checkout. (This is an alternative to using Scalar as that vector,
only because the sparse index might be stable enough to recommend it for
all of Git. It's also possible that we don't want to enable the sparse index
by default in Git proper until more integrations are complete.)
 
> What I'm looking for
> ====================
> 
> The two patches in this series revise existing documentation to match the
> description above. Please let me know if 1) those revisions match your
> understanding of the above, 2) they convey the intent clearly, and (most
> importantly) 3) they reflect a reasonable direction to take Scalar. And, of
> course, if there's something I missed, I welcome any and all feedback &
> ideas!

Thanks for the clear cover letter. I'll review the patches with this in
mind.

Thanks,
-Stolee