git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / code / Atom feed
From: "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: Victoria Dye <vdye@github.com>, Derrick Stolee <stolee@gmail.com>,
	Lessley Dennington <lessleydennington@gmail.com>,
	Elijah Newren <newren@gmail.com>
Subject: [PATCH 0/5] Remove the present-despite-SKIP_WORKTREE class of bugs (for sparse-checkouts)
Date: Thu, 13 Jan 2022 16:43:45 +0000	[thread overview]
Message-ID: <pull.1114.git.1642092230.gitgitgadget@gmail.com> (raw)

(Maintainer note: This series builds on (v2 of) vd/sparse-clean-etc, because
it tweaks one of the testcases added there.)

(Note 2: There was a previous RFC round of this series at
https://lore.kernel.org/git/20220109045732.2497526-1-newren@gmail.com/.)

Files in the present-despite-SKIP_WORKTREE state have caused no ends of
discussions and bugs[1,2,3,4,5,6,...and lots of others]. Trying to address
the big issue of discovering & recovering from this state has befuddled me
for over a year because I was worried we'd need additional code at every
skip_worktree-checking path in the code (and they are all over the place),
and that we'd make the code significantly slower unless we plumbed a bunch
of additional information all over the place to allow some reasonable
optimizations.

This series tries to solve the problem a bit differently by automatic early
discovery and recovery; as it result, it greatly simplifies the landscape,
reduces our testing matrix burden, and fixes a large swath of bugs. And I
figured out how to get the perf cost down to essentially negligible.

Changes since RFC version:

 * updated the commit messages as per suggestions from Victoria, including
   adding performance measurements
 * renamed the new function to use a clearer name
 * replaced the final patch with a different optimization, which is both
   simpler and performs quite a bit better (the cost for my previous patch 5
   was already decent in many cases, but had a few cases where the cost was
   significant).

Quick overview:

 * Patches 1 & 2 add a test to demonstrate accidental deletion of
   possibly-modified files, and then fix the bug.
 * Patch 3 is the crux of this series; a small amount of code with a huge
   commit message
 * Patch 4 updates the documentation
 * Patch 5 adds an optimization to reduce the performance impact of patch 3

[1]
https://lore.kernel.org/git/CABPp-BFnFpzwGC11TLoLs8YK5yiisA5D5-fFjXnJsbESVDwZsA@mail.gmail.com/
[2]
https://lore.kernel.org/git/CABPp-BGJ_Nvi5TmgriD9Bh6eNXE2EDq2f8e8QKXAeYG3BxZafA@mail.gmail.com/
[3]
https://lore.kernel.org/git/pull.809.git.git.1592356884310.gitgitgadget@gmail.com/
[4] commit 66b209b ("merge-ort: implement CE_SKIP_WORKTREE handling with
conflicted entries", 2021-03-20) [5] commit ba359fd ("stash: fix stash
application in sparse-checkouts", 2020-12-01) [6]
https://lore.kernel.org/git/CABPp-BH9tju7WVm=QZDOvaMDdZbpNXrVWQdN-jmfN8wC6YVhmw@mail.gmail.com/

Elijah Newren (5):
  t1011: add testcase demonstrating accidental loss of user
    modifications
  unpack-trees: fix accidental loss of user changes
  repo_read_index: clear SKIP_WORKTREE bit from files present in
    worktree
  Update documentation related to sparsity and the skip-worktree bit
  Accelerate clear_skip_worktree_from_present_files() by caching

 Documentation/git-read-tree.txt          | 12 +++-
 Documentation/git-sparse-checkout.txt    | 76 ++++++++++++++----------
 Documentation/git-update-index.txt       | 57 +++++++++++++-----
 repository.c                             |  7 +++
 sparse-index.c                           | 73 +++++++++++++++++++++++
 sparse-index.h                           |  1 +
 t/t1011-read-tree-sparse-checkout.sh     | 23 ++++++-
 t/t1092-sparse-checkout-compatibility.sh | 16 ++---
 t/t3705-add-sparse-checkout.sh           |  2 +
 t/t6428-merge-conflicts-sparse.sh        | 23 ++-----
 t/t7012-skip-worktree-writing.sh         | 44 +++-----------
 t/t7817-grep-sparse-checkout.sh          | 11 +++-
 unpack-trees.c                           |  4 +-
 13 files changed, 235 insertions(+), 114 deletions(-)


base-commit: 80697e9259e4a2c12fd0546f5f1895fb3068a66c
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1114%2Fnewren%2Ffix-present-despite-skip-worktree-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1114/newren/fix-present-despite-skip-worktree-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1114
-- 
gitgitgadget

             reply	other threads:[~2022-01-13 16:44 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-01-13 16:43 Elijah Newren via GitGitGadget [this message]
2022-01-13 16:43 ` [PATCH 1/5] t1011: add testcase demonstrating accidental loss of user modifications Elijah Newren via GitGitGadget
2022-01-13 16:43 ` [PATCH 2/5] unpack-trees: fix accidental loss of user changes Elijah Newren via GitGitGadget
2022-01-13 16:43 ` [PATCH 3/5] repo_read_index: clear SKIP_WORKTREE bit from files present in worktree Elijah Newren via GitGitGadget
2022-01-13 16:43 ` [PATCH 4/5] Update documentation related to sparsity and the skip-worktree bit Elijah Newren via GitGitGadget
2022-01-13 16:43 ` [PATCH 5/5] Accelerate clear_skip_worktree_from_present_files() by caching Elijah Newren via GitGitGadget
2022-01-13 23:35   ` Elijah Newren
2022-01-14 15:59 ` [PATCH v2 0/5] Remove the present-despite-SKIP_WORKTREE class of bugs (for sparse-checkouts) Elijah Newren via GitGitGadget
2022-01-14 15:59   ` [PATCH v2 1/5] t1011: add testcase demonstrating accidental loss of user modifications Elijah Newren via GitGitGadget
2022-02-16  8:51     ` Ævar Arnfjörð Bjarmason
2022-02-16 16:02       ` Elijah Newren
2022-01-14 15:59   ` [PATCH v2 2/5] unpack-trees: fix accidental loss of user changes Elijah Newren via GitGitGadget
2022-01-14 15:59   ` [PATCH v2 3/5] repo_read_index: clear SKIP_WORKTREE bit from files present in worktree Elijah Newren via GitGitGadget
2022-02-16  8:57     ` Ævar Arnfjörð Bjarmason
2022-02-16 16:08       ` Elijah Newren
2022-02-19  1:06     ` Jonathan Nieder
2022-02-19 16:42       ` Elijah Newren
2022-02-19 18:14         ` Jonathan Nieder
2022-02-20  5:28           ` Elijah Newren
2022-02-20 16:56       ` Derrick Stolee
2022-02-22 23:17         ` Jonathan Nieder
2022-01-14 15:59   ` [PATCH v2 4/5] Update documentation related to sparsity and the skip-worktree bit Elijah Newren via GitGitGadget
2022-02-16  9:15     ` Ævar Arnfjörð Bjarmason
2022-02-16 16:21       ` Elijah Newren
2022-01-14 15:59   ` [PATCH v2 5/5] Accelerate clear_skip_worktree_from_present_files() by caching Elijah Newren via GitGitGadget
2022-01-15  1:39     ` Victoria Dye
2022-02-16  9:32     ` Ævar Arnfjörð Bjarmason
2022-02-16 16:30       ` Elijah Newren
2022-02-17  4:40         ` Elijah Newren
2022-01-15  1:51   ` [PATCH v2 0/5] Remove the present-despite-SKIP_WORKTREE class of bugs (for sparse-checkouts) Victoria Dye

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=pull.1114.git.1642092230.gitgitgadget@gmail.com \
    --to=gitgitgadget@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=lessleydennington@gmail.com \
    --cc=newren@gmail.com \
    --cc=stolee@gmail.com \
    --cc=vdye@github.com \
    --subject='Re: [PATCH 0/5] Remove the present-despite-SKIP_WORKTREE class of bugs (for sparse-checkouts)' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Code repositories for project(s) associated with this inbox:

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).