Re: [PATCH v4] sparse-checkout.txt: new document with sparse-checkout directions

[ZheNing Hu <adlternative@gmail.com>]

ZheNing Hu <adlternative@gmail.com> 于2022年11月15日周二 12:03写道：
>
> Hi,
>
> Elijah Newren via GitGitGadget <gitgitgadget@gmail.com> 于2022年11月6日周日 14:04写道：
> >
> > From: Elijah Newren <newren@gmail.com>
> >
> > Once upon a time, Matheus wrote some patches to make
> >    git grep [--cached | <REVISION>] ...
> > restrict its output to the sparsity specification when working in a
> > sparse checkout[1].  That effort got derailed by two things:
> >
> >   (1) The --sparse-index work just beginning which we wanted to avoid
> >       creating conflicts for
> >   (2) Never deciding on flag and config names and planned high level
> >       behavior for all commands.
> >
> > More recently, Shaoxuan implemented a more limited form of Matheus'
> > patches that only affected --cached, using a different flag name,
> > but also changing the default behavior in line with what Matheus did.
> > This again highlighted the fact that we never decided on command line
> > flag names, config option names, and the big picture path forward.
> >
> > The --sparse-index work has been mostly complete (or at least released
> > into production even if some small edges remain) for quite some time
> > now.  We have also had several discussions on flag and config names,
> > though we never came to solid conclusions.  Stolee once upon a time
> > suggested putting all these into some document in
> > Documentation/technical[3], which Victoria recently also requested[4].
> > I'm behind the times, but here's a patch attempting to finally do that.
> >
> > [1] https://lore.kernel.org/git/5f3f7ac77039d41d1692ceae4b0c5df3bb45b74a.1612901326.git.matheus.bernardino@usp.br/
> >     (See his second link in that email in particular)
> > [2] https://lore.kernel.org/git/20220908001854.206789-2-shaoxuan.yuan02@gmail.com/
> > [3] https://lore.kernel.org/git/CABPp-BHwNoVnooqDFPAsZxBT9aR5Dwk5D9sDRCvYSb8akxAJgA@mail.gmail.com/
> >     (Scroll to the very end for the final few paragraphs)
> > [4] https://lore.kernel.org/git/cafcedba-96a2-cb85-d593-ef47c8c8397c@github.com/
> >
> > Signed-off-by: Elijah Newren <newren@gmail.com>
> > ---
> >     sparse-checkout.txt: new document with sparse-checkout directions
> >
> >     v2 and v3 didn't get any reviews (I know, I know, this document is
> >     really long), but it's been nearly a month and this patch is still
> >     marked as "Needs Review", so I'm hoping sending a v4 will encourage
> >     feedback. I think it's good enough to accept and start iterating, but
> >     want to be sure others agree.
> >
> >     As before, I think we're starting to converge on actual proposals;
> >     there's some areas we've agreed on, others we've compromised on, and
> >     some we've just figured out what the others were saying. The discussion
> >     has been very illuminating; thanks to everyone who has chimed in. I've
> >     tried to take my best stab at cleaning up and culling things that don't
> >     need to remain as open questions, but if I've mis-represented anyone or
> >     missed something, don't hesitate to speak up. Everything is still open
> >     for debate, even if not marked as a currently open question.
> >
> >     Changes since v3:
> >
> >      * A few minor wording cleanups here and there, and one paragraph moved
> >        to keep similar things together.
> >
> >     Changes since v2:
> >
> >      * Compromised with Stollee on log -- Behavior A only affects
> >        patch-related operations, not revision walking
> >      * Incorporated Junio's suggestions about untracked file handling
> >      * Added new usecases, one brought up by Martin, one by Stolee
> >      * Added new sections:
> >        * Usecases of primary concern
> >        * Oversimplified mental models ("Cliff Notes" for this document!)
> >      * Recategorization of a few commands based on discussion
> >      * Greater details on how index operations work under Behavior A, to
> >        avoid weird edge cases
> >      * Extended explanation of the sparse specification, particularly when
> >        index differs from HEAD
> >      * Switched proposed flag names to --scope={sparse,all} to avoid binary
> >        flags that are hard to extend
> >      * Switched proposed config option name (still need good values and
> >        descriptions for it, though)
> >      * Removed questions we seemed to have agreement on. Modified/extended
> >        some existing questions.
> >      * Added Stolee's sparse-backfill ideas to the plans
> >      * Additional Known bugs
> >      * Various wording improvements
> >      * Possibly other things I've missed.
> >
> >     Changes since v1:
> >
> >      * Added new sections:
> >        * "Terminology"
> >        * "Behavior classes"
> >        * "Sparse specification vs. sparsity patterns"
> >      * Tried to shuffle commands from unknown into appropriate sections
> >        based on feedback, but I got some conflicting feedback, so...who
> >        knows if thing are in the right place
> >      * More consistency in using "sparse specification" over other terms
> >      * Extra comments about how add/rm/mv operate on moving files across the
> >        tracked/untracked boundary
> >      * --restrict-but-warn should have been "restrict or error", but
> >        reworded even more heavily as part of "Behavior classes" section
> >      * Added extra questions based on feedback (--no-expand, update-index
> >        stuff, apply --index)
> >      * More details on apply/am bugs
> >      * Documented read-tree issue
> >      * A few cases of fixing line wrapping at <=80 chars
> >      * Added more alternate name suggestions for options instead of
> >        --[no-]restrict
> >
> > Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1367%2Fnewren%2Fsparse-checkout-directions-v4
> > Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1367/newren/sparse-checkout-directions-v4
> > Pull-Request: https://github.com/gitgitgadget/git/pull/1367
> >
> >  Documentation/technical/sparse-checkout.txt | 1103 +++++++++++++++++++
> >  1 file changed, 1103 insertions(+)
> >  create mode 100644 Documentation/technical/sparse-checkout.txt
> >
> > diff --git a/Documentation/technical/sparse-checkout.txt b/Documentation/technical/sparse-checkout.txt
> > new file mode 100644
> > +=== Terminology ===
> > +
> > +sparse directory: An entry in the index corresponding to a directory, which
> > +       appears in the index instead of all the files under that directory
> > +       that would normally appear.  See also sparse-index.  Something that
> > +       can cause confusion is that the "sparse directory" does NOT match
> > +       the sparse specification, i.e. the directory is NOT present in the
> > +       working tree.  May be renamed in the future (e.g. to "skipped
> > +       directory").
> > +
> > +sparse index: A special mode for sparse-checkout that also makes the
> > +       index sparse by recording a directory entry in lieu of all the
> > +       files underneath that directory (thus making that a "skipped
> > +       directory" which unfortunately has also been called a "sparse
> > +       directory"), and does this for potentially multiple
> > +       directories.  Controlled by the --[no-]sparse-index option to
> > +       init|set|reapply.
> > +
> > +sparsity patterns: patterns from $GIT_DIR/info/sparse-checkout used to
> > +       define the set of files of interest.  A warning: It is easy to
> > +       over-use this term (or the shortened "patterns" term), for two
> > +       reasons: (1) users in cone mode specify directories rather than
> > +       patterns (their directories are transformed into patterns, but
> > +       users may think you are talking about non-cone mode if you use the
> > +       word "patterns"), and (b) the sparse specification might
>
> nit: s/(b)/(2)/g
>
> > +       transiently differ in the working tree or index from the sparsity
> > +       patterns (see "Sparse specification vs. sparsity patterns").
> > +
> > +sparse specification: The set of paths in the user's area of focus.  This
> > +       is typically just the tracked files that match the sparsity
> > +       patterns, but the sparse specification can temporarily differ and
> > +       include additional files.  (See also "Sparse specification
> > +       vs. sparsity patterns")
> > +
> > +       * When working with history, the sparse specification is exactly
> > +         the set of files matching the sparsity patterns.
> > +       * When interacting with the working tree, the sparse specification
> > +         is the set of tracked files with a clear SKIP_WORKTREE bit or
> > +         tracked files present in the working copy.
>

I found af6a518 (repo_read_index: clear SKIP_WORKTREE bit from files
present in worktree) which maybe a good place to learn about "sparse
specification",
it has a long commit message though.

> I'm guessing what you mean here is:
> Some files are stored with a flag bit of !SKIP_WORKTREE in its index entry.
> But files are "vivifying" (restore to worktree) or new files added to
> index (tracked files),
> they also belong to the sparse specification.
>
> I think we can add some examples to describe these terms.
>
> #!/bin/sh
>
> set -x
>
> rm -rf mono-repo
> git init mono-repo -b main
> (
>   cd mono-repo &&
>   mkdir p1 p2 &&
>   echo a >p1/a &&
>   echo b >p1/b &&
>   echo a >p2/a &&
>   echo b >p2/b &&
>   git add . &&
>   git commit -m ok &&
>   git sparse-checkout set p1 &&
>   git ls-files -t &&
>   echo a >>p1/a &&
>   echo b >>p1/b &&
>   mkdir p2 p3 &&
>   echo next >>p2/a &&
>   echo next >>p3/c &&
>   git add p3/c &&
>   # p2/a and p3/c vivify
>   git ls-files -t &&
>   # compare wortree/commit
>   git --no-pager diff HEAD --name-only
> )
>
> > +       * When modifying or showing results from the index, the sparse
> > +         specification is the set of files with a clear SKIP_WORKTREE bit
> > +         or that differ in the index from HEAD.
>
> #!/bin/sh
>
> set -x
>
> rm -rf mono-repo
> git init mono-repo -b main
> (
>   cd mono-repo &&
>   mkdir p1 p2 &&
>   echo a >p1/a &&
>   echo b >p1/b &&
>   echo a >p2/a &&
>   echo b >p2/b &&
>   git add . &&
>   git commit -m ok &&
>   git sparse-checkout set p1 &&
>   git update-index --chmod=+x p2/a &&
>   # compare commit/index
>   git --no-pager diff --cached --name-only
> )
>