git@vger.kernel.org mailing list mirror (one of many)
 help / Atom feed
* [PATCH 0/8] introduce no-overlay and cached mode in git checkout
@ 2018-12-09 20:04 Thomas Gummerer
  2018-12-09 20:04 ` [PATCH 1/8] move worktree tests to t24* Thomas Gummerer
                   ` (11 more replies)
  0 siblings, 12 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-09 20:04 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Thomas Gummerer

Here's the series I mentioned a couple of times on the list already,
introducing a no-overlay mode in 'git checkout'.  The inspiration for
this came from Junios message in [*1*].

Basically the idea is to also delete files when the match <pathspec>
in 'git checkout <tree-ish> -- <pathspec>' in the current tree, but
don't match <pathspec> in <tree-ish>.  The rest of the cases are
already properly taken care of by 'git checkout'.

The final step in the series is to actually make use of this in 'git
stash', which simplifies the code there a bit.  I am however happy to
hold off on this step until the stash-in-C series is merged, so we
don't delay that further.

In addition to the no-overlay mode, we also add a --cached mode, which
works only on the index, thus similar to 'git reset <tree-ish> -- <pathspec>'.

Actually deprecating 'git reset <tree-ish> -- <pathspec>' should come
later, probably not before Duy's restore-files command lands, as 'git
checkout --no-overlay <tree-ish> -- <pathspec>' is a bit cumbersome to
type compared to 'git reset <tree-ish> -- <pathspec>'.

My hope is also that the no-overlay mode could become the new default
in the restore-files command Duy is currently working on.

No documentation yet, as I wanted to get this out for review first.
I'm not familiar with most of the code I touched here, so there may
well be much better ways to implement some of this, that I wasn't able
to figure out.  I'd be very happy with some feedback around that.

Another thing I'm not sure about is how to deal with conflicts.  In
the cached mode this patch series is not dealing with it at all, as
'git checkout -- <pathspec>' when pathspec matches a file with
conflicts doesn't update the index.  For the no-overlay mode, the file
is removed if the corresponding stage is not found in the index.  I'm
however not sure this is the right thing to do in all cases?

*1*: <xmqq4loqplou.fsf@gitster.mtv.corp.google.com>

Thomas Gummerer (8):
  move worktree tests to t24*
  entry: factor out unlink_entry function
  entry: support CE_WT_REMOVE flag in checkout_entry
  read-cache: add invalidate parameter to remove_marked_cache_entries
  checkout: introduce --{,no-}overlay option
  checkout: add --cached option
  checkout: add allow ignoring unmatched pathspec
  stash: use git checkout --index

 builtin/checkout.c                            |  66 +++++++++--
 cache.h                                       |   7 +-
 entry.c                                       |  22 ++++
 git-stash.sh                                  |  12 +-
 read-cache.c                                  |   8 +-
 split-index.c                                 |   2 +-
 t/t2016-checkout-patch.sh                     |   8 ++
 t/t2022-checkout-paths.sh                     |   9 ++
 t/t2025-checkout-no-overlay.sh                |  31 ++++++
 t/t2026-checkout-cached.sh                    | 103 ++++++++++++++++++
 ...-worktree-add.sh => t2400-worktree-add.sh} |   0
 ...ktree-prune.sh => t2401-worktree-prune.sh} |   0
 ...orktree-list.sh => t2402-worktree-list.sh} |   0
 t/t9902-completion.sh                         |   3 +
 unpack-trees.c                                |  21 +---
 15 files changed, 251 insertions(+), 41 deletions(-)
 create mode 100755 t/t2025-checkout-no-overlay.sh
 create mode 100755 t/t2026-checkout-cached.sh
 rename t/{t2025-worktree-add.sh => t2400-worktree-add.sh} (100%)
 rename t/{t2026-worktree-prune.sh => t2401-worktree-prune.sh} (100%)
 rename t/{t2027-worktree-list.sh => t2402-worktree-list.sh} (100%)

-- 
2.20.0.rc2.411.g8f28e744c2


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH 1/8] move worktree tests to t24*
  2018-12-09 20:04 [PATCH 0/8] introduce no-overlay and cached mode in git checkout Thomas Gummerer
@ 2018-12-09 20:04 ` Thomas Gummerer
  2018-12-10  3:48   ` Junio C Hamano
  2018-12-10 15:32   ` Duy Nguyen
  2018-12-09 20:04 ` [PATCH 2/8] entry: factor out unlink_entry function Thomas Gummerer
                   ` (10 subsequent siblings)
  11 siblings, 2 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-09 20:04 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Thomas Gummerer

The 'git worktree' command used to be just another mode in 'git
checkout', namely 'git checkout --to'.  When the tests for the latter
were retrofitted for the former, the test name was adjusted, but the
test number was kept, even though the test is testing a different
command now.  t/README states: "Second digit tells the particular
command we are testing.", so 'git worktree' should have a separate
number just for itself.

Move the worktree tests to t24* to adhere to that guideline. We're
going to make use of the free'd up numbers in a subsequent commit.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 t/{t2025-worktree-add.sh => t2400-worktree-add.sh}     | 0
 t/{t2026-worktree-prune.sh => t2401-worktree-prune.sh} | 0
 t/{t2027-worktree-list.sh => t2402-worktree-list.sh}   | 0
 3 files changed, 0 insertions(+), 0 deletions(-)
 rename t/{t2025-worktree-add.sh => t2400-worktree-add.sh} (100%)
 rename t/{t2026-worktree-prune.sh => t2401-worktree-prune.sh} (100%)
 rename t/{t2027-worktree-list.sh => t2402-worktree-list.sh} (100%)

diff --git a/t/t2025-worktree-add.sh b/t/t2400-worktree-add.sh
similarity index 100%
rename from t/t2025-worktree-add.sh
rename to t/t2400-worktree-add.sh
diff --git a/t/t2026-worktree-prune.sh b/t/t2401-worktree-prune.sh
similarity index 100%
rename from t/t2026-worktree-prune.sh
rename to t/t2401-worktree-prune.sh
diff --git a/t/t2027-worktree-list.sh b/t/t2402-worktree-list.sh
similarity index 100%
rename from t/t2027-worktree-list.sh
rename to t/t2402-worktree-list.sh
-- 
2.20.0.405.gbc1bbc6f85


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH 2/8] entry: factor out unlink_entry function
  2018-12-09 20:04 [PATCH 0/8] introduce no-overlay and cached mode in git checkout Thomas Gummerer
  2018-12-09 20:04 ` [PATCH 1/8] move worktree tests to t24* Thomas Gummerer
@ 2018-12-09 20:04 ` Thomas Gummerer
  2018-12-10 15:49   ` Duy Nguyen
  2018-12-09 20:04 ` [PATCH 3/8] entry: support CE_WT_REMOVE flag in checkout_entry Thomas Gummerer
                   ` (9 subsequent siblings)
  11 siblings, 1 reply; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-09 20:04 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Thomas Gummerer

Factor out the 'unlink_entry()' function from unpack-trees.c to
entry.c.  It will be used in other places as well in subsequent
steps.

As it's no longer a static function, also move the documentation to
the header file to make it more discoverable.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 cache.h        |  5 +++++
 entry.c        | 15 +++++++++++++++
 unpack-trees.c | 19 -------------------
 3 files changed, 20 insertions(+), 19 deletions(-)

diff --git a/cache.h b/cache.h
index ca36b44ee0..c1c953e810 100644
--- a/cache.h
+++ b/cache.h
@@ -1542,6 +1542,11 @@ struct checkout {
 extern int checkout_entry(struct cache_entry *ce, const struct checkout *state, char *topath);
 extern void enable_delayed_checkout(struct checkout *state);
 extern int finish_delayed_checkout(struct checkout *state);
+/*
+ * Unlink the last component and schedule the leading directories for
+ * removal, such that empty directories get removed.
+ */
+extern void unlink_entry(const struct cache_entry *ce);
 
 struct cache_def {
 	struct strbuf path;
diff --git a/entry.c b/entry.c
index 5d136c5d55..3ec148ceee 100644
--- a/entry.c
+++ b/entry.c
@@ -508,3 +508,18 @@ int checkout_entry(struct cache_entry *ce,
 	create_directories(path.buf, path.len, state);
 	return write_entry(ce, path.buf, state, 0);
 }
+
+void unlink_entry(const struct cache_entry *ce)
+{
+	const struct submodule *sub = submodule_from_ce(ce);
+	if (sub) {
+		/* state.force is set at the caller. */
+		submodule_move_head(ce->name, "HEAD", NULL,
+				    SUBMODULE_MOVE_HEAD_FORCE);
+	}
+	if (!check_leading_path(ce->name, ce_namelen(ce)))
+		return;
+	if (remove_or_warn(ce->ce_mode, ce->name))
+		return;
+	schedule_dir_for_removal(ce->name, ce_namelen(ce));
+}
diff --git a/unpack-trees.c b/unpack-trees.c
index 7570df481b..e8d1a6ac50 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -300,25 +300,6 @@ static void load_gitmodules_file(struct index_state *index,
 	}
 }
 
-/*
- * Unlink the last component and schedule the leading directories for
- * removal, such that empty directories get removed.
- */
-static void unlink_entry(const struct cache_entry *ce)
-{
-	const struct submodule *sub = submodule_from_ce(ce);
-	if (sub) {
-		/* state.force is set at the caller. */
-		submodule_move_head(ce->name, "HEAD", NULL,
-				    SUBMODULE_MOVE_HEAD_FORCE);
-	}
-	if (!check_leading_path(ce->name, ce_namelen(ce)))
-		return;
-	if (remove_or_warn(ce->ce_mode, ce->name))
-		return;
-	schedule_dir_for_removal(ce->name, ce_namelen(ce));
-}
-
 static struct progress *get_progress(struct unpack_trees_options *o)
 {
 	unsigned cnt = 0, total = 0;
-- 
2.20.0.405.gbc1bbc6f85


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH 3/8] entry: support CE_WT_REMOVE flag in checkout_entry
  2018-12-09 20:04 [PATCH 0/8] introduce no-overlay and cached mode in git checkout Thomas Gummerer
  2018-12-09 20:04 ` [PATCH 1/8] move worktree tests to t24* Thomas Gummerer
  2018-12-09 20:04 ` [PATCH 2/8] entry: factor out unlink_entry function Thomas Gummerer
@ 2018-12-09 20:04 ` Thomas Gummerer
  2018-12-10 15:58   ` Duy Nguyen
  2018-12-10 17:49   ` Elijah Newren
  2018-12-09 20:04 ` [PATCH 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries Thomas Gummerer
                   ` (8 subsequent siblings)
  11 siblings, 2 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-09 20:04 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Thomas Gummerer

'checkout_entry()' currently only supports creating new entries in the
working tree, but not deleting them.  Add the ability to remove
entries at the same time if the entry is marked with the CE_WT_REMOVE
flag.

Currently this doesn't have any effect, as the CE_WT_REMOVE flag is
only used in unpack-tree, however we will make use of this in a
subsequent step in the series.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 entry.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/entry.c b/entry.c
index 3ec148ceee..cd1c6601b6 100644
--- a/entry.c
+++ b/entry.c
@@ -441,6 +441,13 @@ int checkout_entry(struct cache_entry *ce,
 	static struct strbuf path = STRBUF_INIT;
 	struct stat st;
 
+	if (ce->ce_flags & CE_WT_REMOVE) {
+		if (topath)
+			BUG("Can't remove entry to a path");
+		unlink_entry(ce);
+		return 0;
+	}
+
 	if (topath)
 		return write_entry(ce, topath, state, 1);
 
-- 
2.20.0.405.gbc1bbc6f85


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries
  2018-12-09 20:04 [PATCH 0/8] introduce no-overlay and cached mode in git checkout Thomas Gummerer
                   ` (2 preceding siblings ...)
  2018-12-09 20:04 ` [PATCH 3/8] entry: support CE_WT_REMOVE flag in checkout_entry Thomas Gummerer
@ 2018-12-09 20:04 ` Thomas Gummerer
  2018-12-10 16:08   ` Duy Nguyen
  2018-12-09 20:04 ` [PATCH 5/8] checkout: introduce --{,no-}overlay option Thomas Gummerer
                   ` (7 subsequent siblings)
  11 siblings, 1 reply; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-09 20:04 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Thomas Gummerer

When marking cache entries for removal, and later removing them all at
once using 'remove_marked_cache_entries()', cache entries currently
have to be invalidated manually in the cache tree and in the untracked
cache.

Add an invalidate flag to the function.  With the flag set, the
function will take care of invalidating the path in the cache tree and
in the untracked cache.

This will be useful in a subsequent commit.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---

For the two current callsites, unpack-trees seems to do this
invalidation itself internally.  I don't quite understand why we don't
need it in split-index mode though.  I assume it's because the cache
tree in the main index would already have been invalidated?  I didn't
have much time to dig, but couldn't produce any failures with it
either, so I assume not invalidating paths is the right thing to do
here.

 cache.h        | 2 +-
 read-cache.c   | 8 +++++++-
 split-index.c  | 2 +-
 unpack-trees.c | 2 +-
 4 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/cache.h b/cache.h
index c1c953e810..1deee48f5b 100644
--- a/cache.h
+++ b/cache.h
@@ -751,7 +751,7 @@ extern void rename_index_entry_at(struct index_state *, int pos, const char *new
 /* Remove entry, return true if there are more entries to go. */
 extern int remove_index_entry_at(struct index_state *, int pos);
 
-extern void remove_marked_cache_entries(struct index_state *istate);
+extern void remove_marked_cache_entries(struct index_state *istate, int invalidate);
 extern int remove_file_from_index(struct index_state *, const char *path);
 #define ADD_CACHE_VERBOSE 1
 #define ADD_CACHE_PRETEND 2
diff --git a/read-cache.c b/read-cache.c
index 4ca81286c0..d86a06acba 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -590,13 +590,19 @@ int remove_index_entry_at(struct index_state *istate, int pos)
  * CE_REMOVE is set in ce_flags.  This is much more effective than
  * calling remove_index_entry_at() for each entry to be removed.
  */
-void remove_marked_cache_entries(struct index_state *istate)
+void remove_marked_cache_entries(struct index_state *istate, int invalidate)
 {
 	struct cache_entry **ce_array = istate->cache;
 	unsigned int i, j;
 
 	for (i = j = 0; i < istate->cache_nr; i++) {
 		if (ce_array[i]->ce_flags & CE_REMOVE) {
+			if (invalidate) {
+				cache_tree_invalidate_path(istate,
+							   ce_array[i]->name);
+				untracked_cache_remove_from_index(istate,
+								  ce_array[i]->name);
+			}
 			remove_name_hash(istate, ce_array[i]);
 			save_or_free_index_entry(istate, ce_array[i]);
 		}
diff --git a/split-index.c b/split-index.c
index 5820412dc5..8aebc3661b 100644
--- a/split-index.c
+++ b/split-index.c
@@ -162,7 +162,7 @@ void merge_base_index(struct index_state *istate)
 	ewah_each_bit(si->replace_bitmap, replace_entry, istate);
 	ewah_each_bit(si->delete_bitmap, mark_entry_for_delete, istate);
 	if (si->nr_deletions)
-		remove_marked_cache_entries(istate);
+		remove_marked_cache_entries(istate, 0);
 
 	for (i = si->nr_replacements; i < si->saved_cache_nr; i++) {
 		if (!ce_namelen(si->saved_cache[i]))
diff --git a/unpack-trees.c b/unpack-trees.c
index e8d1a6ac50..8e6afa924d 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -392,7 +392,7 @@ static int check_updates(struct unpack_trees_options *o)
 				unlink_entry(ce);
 		}
 	}
-	remove_marked_cache_entries(index);
+	remove_marked_cache_entries(index, 0);
 	remove_scheduled_dirs();
 
 	if (should_update_submodules() && o->update && !o->dry_run)
-- 
2.20.0.405.gbc1bbc6f85


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH 5/8] checkout: introduce --{,no-}overlay option
  2018-12-09 20:04 [PATCH 0/8] introduce no-overlay and cached mode in git checkout Thomas Gummerer
                   ` (3 preceding siblings ...)
  2018-12-09 20:04 ` [PATCH 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries Thomas Gummerer
@ 2018-12-09 20:04 ` Thomas Gummerer
  2018-12-10 16:42   ` Duy Nguyen
  2018-12-10 18:19   ` Elijah Newren
  2018-12-09 20:04 ` [PATCH 6/8] checkout: add --cached option Thomas Gummerer
                   ` (6 subsequent siblings)
  11 siblings, 2 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-09 20:04 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Thomas Gummerer

Currently 'git checkout' is defined as an overlay operation, which
means that if in 'git checkout <tree-ish> -- [<pathspec>]' we have an
entry in the index that matches <pathspec>, but that doesn't exist in
<tree-ish>, that entry will not be removed from the index or the
working tree.

Introduce a new --{,no-}overlay option, which allows using 'git
checkout' in non-overlay mode, thus removing files from the working
tree if they do not exist in <tree-ish> but match <pathspec>.

Note that 'git checkout -p <tree-ish> -- [<pathspec>]' already works
this way, so no changes are needed for the patch mode.  We disallow
'git checkout --overlay -p' to avoid confusing users who would expect
to be able to force overlay mode in 'git checkout -p' this way.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 builtin/checkout.c             | 64 +++++++++++++++++++++++++++-------
 t/t2025-checkout-no-overlay.sh | 47 +++++++++++++++++++++++++
 t/t9902-completion.sh          |  1 +
 3 files changed, 99 insertions(+), 13 deletions(-)
 create mode 100755 t/t2025-checkout-no-overlay.sh

diff --git a/builtin/checkout.c b/builtin/checkout.c
index acdafc6e4c..0aef35bbc4 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -44,6 +44,7 @@ struct checkout_opts {
 	int ignore_skipworktree;
 	int ignore_other_worktrees;
 	int show_progress;
+	int overlay_mode;
 	/*
 	 * If new checkout options are added, skip_merge_working_tree
 	 * should be updated accordingly.
@@ -132,7 +133,8 @@ static int skip_same_name(const struct cache_entry *ce, int pos)
 	return pos;
 }
 
-static int check_stage(int stage, const struct cache_entry *ce, int pos)
+static int check_stage(int stage, const struct cache_entry *ce, int pos,
+		       int overlay_mode)
 {
 	while (pos < active_nr &&
 	       !strcmp(active_cache[pos]->name, ce->name)) {
@@ -140,6 +142,8 @@ static int check_stage(int stage, const struct cache_entry *ce, int pos)
 			return 0;
 		pos++;
 	}
+	if (!overlay_mode)
+		return 0;
 	if (stage == 2)
 		return error(_("path '%s' does not have our version"), ce->name);
 	else
@@ -165,7 +169,7 @@ static int check_stages(unsigned stages, const struct cache_entry *ce, int pos)
 }
 
 static int checkout_stage(int stage, const struct cache_entry *ce, int pos,
-			  const struct checkout *state)
+			  const struct checkout *state, int overlay_mode)
 {
 	while (pos < active_nr &&
 	       !strcmp(active_cache[pos]->name, ce->name)) {
@@ -173,6 +177,10 @@ static int checkout_stage(int stage, const struct cache_entry *ce, int pos,
 			return checkout_entry(active_cache[pos], state, NULL);
 		pos++;
 	}
+	if (!overlay_mode) {
+		unlink_entry(ce);
+		return 0;
+	}
 	if (stage == 2)
 		return error(_("path '%s' does not have our version"), ce->name);
 	else
@@ -302,15 +310,29 @@ static int checkout_paths(const struct checkout_opts *opts,
 		ce->ce_flags &= ~CE_MATCHED;
 		if (!opts->ignore_skipworktree && ce_skip_worktree(ce))
 			continue;
-		if (opts->source_tree && !(ce->ce_flags & CE_UPDATE))
-			/*
-			 * "git checkout tree-ish -- path", but this entry
-			 * is in the original index; it will not be checked
-			 * out to the working tree and it does not matter
-			 * if pathspec matched this entry.  We will not do
-			 * anything to this entry at all.
-			 */
-			continue;
+		if (opts->source_tree && !(ce->ce_flags & CE_UPDATE)) {
+			if (!opts->overlay_mode &&
+			    ce_path_match(&the_index, ce, &opts->pathspec, ps_matched)) {
+				/*
+				 * "git checkout --no-overlay <tree-ish> -- path",
+				 * and the path is not in tree-ish, but is in
+				 * the current index, which means that it should 
+				 * be removed.
+				 */
+				ce->ce_flags |= CE_MATCHED | CE_REMOVE | CE_WT_REMOVE;
+				continue;
+			} else {
+				/*
+				 * "git checkout tree-ish -- path", but this
+				 * entry is in the original index; it will not
+				 * be checked out to the working tree and it
+				 * does not matter if pathspec matched this
+				 * entry.  We will not do anything to this entry
+				 * at all.
+				 */
+				continue;
+			}
+		}
 		/*
 		 * Either this entry came from the tree-ish we are
 		 * checking the paths out of, or we are checking out
@@ -348,7 +370,7 @@ static int checkout_paths(const struct checkout_opts *opts,
 			if (opts->force) {
 				warning(_("path '%s' is unmerged"), ce->name);
 			} else if (opts->writeout_stage) {
-				errs |= check_stage(opts->writeout_stage, ce, pos);
+				errs |= check_stage(opts->writeout_stage, ce, pos, opts->overlay_mode);
 			} else if (opts->merge) {
 				errs |= check_stages((1<<2) | (1<<3), ce, pos);
 			} else {
@@ -375,12 +397,14 @@ static int checkout_paths(const struct checkout_opts *opts,
 				continue;
 			}
 			if (opts->writeout_stage)
-				errs |= checkout_stage(opts->writeout_stage, ce, pos, &state);
+				errs |= checkout_stage(opts->writeout_stage, ce, pos, &state, opts->overlay_mode);
 			else if (opts->merge)
 				errs |= checkout_merged(pos, &state);
 			pos = skip_same_name(ce, pos) - 1;
 		}
 	}
+	remove_marked_cache_entries(&the_index, 1);
+	remove_scheduled_dirs();
 	errs |= finish_delayed_checkout(&state);
 
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
@@ -542,6 +566,11 @@ static int skip_merge_working_tree(const struct checkout_opts *opts,
 	 * opts->show_progress only impacts output so doesn't require a merge
 	 */
 
+	/*
+	 * opts->overlay_mode cannot be used with switching branches so is
+	 * not tested here
+	 */
+
 	/*
 	 * If we aren't creating a new branch any changes or updates will
 	 * happen in the existing branch.  Since that could only be updating
@@ -1178,6 +1207,10 @@ static int checkout_branch(struct checkout_opts *opts,
 		die(_("'%s' cannot be used with switching branches"),
 		    "--patch");
 
+	if (!opts->overlay_mode)
+		die(_("'%s' cannot be used with switching branches"),
+		    "--no-overlay");
+
 	if (opts->writeout_stage)
 		die(_("'%s' cannot be used with switching branches"),
 		    "--ours/--theirs");
@@ -1266,6 +1299,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 			    "checkout", "control recursive updating of submodules",
 			    PARSE_OPT_OPTARG, option_parse_recurse_submodules_worktree_updater },
 		OPT_BOOL(0, "progress", &opts.show_progress, N_("force progress reporting")),
+		OPT_BOOL(0, "overlay", &opts.overlay_mode, N_("use overlay mode")),
 		OPT_END(),
 	};
 
@@ -1274,6 +1308,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 	opts.overwrite_ignore = 1;
 	opts.prefix = prefix;
 	opts.show_progress = -1;
+	opts.overlay_mode = -1;
 
 	git_config(git_checkout_config, &opts);
 
@@ -1297,6 +1332,9 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 	if ((!!opts.new_branch + !!opts.new_branch_force + !!opts.new_orphan_branch) > 1)
 		die(_("-b, -B and --orphan are mutually exclusive"));
 
+	if (opts.overlay_mode == 1 && opts.patch_mode)
+		die(_("-p and --overlay are mutually exclusive"));
+
 	/*
 	 * From here on, new_branch will contain the branch to be checked out,
 	 * and new_branch_force and new_orphan_branch will tell us which one of
diff --git a/t/t2025-checkout-no-overlay.sh b/t/t2025-checkout-no-overlay.sh
new file mode 100755
index 0000000000..3575321382
--- /dev/null
+++ b/t/t2025-checkout-no-overlay.sh
@@ -0,0 +1,47 @@
+#!/bin/sh
+
+test_description='checkout --no-overlay <tree-ish> -- <pathspec>'
+
+. ./test-lib.sh
+
+test_expect_success 'setup' '
+	git commit --allow-empty -m "initial"
+'
+
+test_expect_success 'checkout --no-overlay deletes files not in <tree>' '
+	>file &&
+	mkdir dir &&
+	>dir/file1 &&
+	git add file dir/file1 &&
+	git checkout --no-overlay HEAD -- file &&
+	test_path_is_missing file &&
+	test_path_is_file dir/file1
+'
+
+test_expect_success 'checkout --no-overlay removing last file from directory' '
+	git checkout --no-overlay HEAD -- dir/file1 &&
+	test_path_is_missing dir
+'
+
+test_expect_success 'checkout -p --overlay is disallowed' '
+	test_must_fail git checkout -p --overlay HEAD 2>actual &&
+	test_i18ngrep "fatal: -p and --overlay are mutually exclusive" actual
+'
+
+test_expect_success '--no-overlay --theirs with M/D conflict deletes file' '
+	test_commit file1 file1 &&
+	test_commit file2 file2 &&
+	git rm --cached file1 &&
+	echo 1234 >file1 &&
+	F1=$(git rev-parse HEAD:file1) &&
+	F2=$(git rev-parse HEAD:file2) &&
+	{
+		echo "100644 $F1 1	file1" &&
+		echo "100644 $F2 2	file1"
+	} | git update-index --index-info &&
+	test_path_is_file file1 &&
+	git checkout --theirs --no-overlay -- file1 &&
+	test_path_is_missing file1
+'
+
+test_done
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 175f83d704..a3fd9a9630 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -1436,6 +1436,7 @@ test_expect_success 'double dash "git checkout"' '
 	--progress Z
 	--no-quiet Z
 	--no-... Z
+	--overlay Z
 	EOF
 '
 
-- 
2.20.0.405.gbc1bbc6f85


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH 6/8] checkout: add --cached option
  2018-12-09 20:04 [PATCH 0/8] introduce no-overlay and cached mode in git checkout Thomas Gummerer
                   ` (4 preceding siblings ...)
  2018-12-09 20:04 ` [PATCH 5/8] checkout: introduce --{,no-}overlay option Thomas Gummerer
@ 2018-12-09 20:04 ` Thomas Gummerer
  2018-12-10 16:49   ` Duy Nguyen
  2018-12-10 18:42   ` Elijah Newren
  2018-12-09 20:04 ` [PATCH 7/8] checkout: allow ignoring unmatched pathspec Thomas Gummerer
                   ` (5 subsequent siblings)
  11 siblings, 2 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-09 20:04 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Thomas Gummerer

Add a new --cached option to git checkout, which works only on the
index, but not the working tree, similar to what 'git reset <tree-ish>
-- <pathspec>... does.  Indeed the tests are adapted from the 'git
reset' tests.

In the longer term the idea is to potentially deprecate 'git reset
<tree-ish> -- <pathspec>...', so the 'git reset' command becomes only
about re-pointing the HEAD, and not also about copying entries from
<tree-ish> to the index.

Note that 'git checkout' by default works in overlay mode, meaning
files that match the pathspec that don't exist in <tree-ish>, but
exist in the index would not be removed.  'git checkout --no-overlay
--cached' can be used to get the same behaviour as 'git reset
<tree-ish> -- <pathspec>'.

One thing this patch doesn't currently deal with is conflicts.
Currently 'git checkout --{ours,theirs} -- <file-with-conflicts>'
doesn't do anything with the index, so the --cached option just
mirrors that behaviour.  But given it doesn't even deal with
conflicts, the '--cached' option doesn't make much sense when no
<tree-ish> is given.  As it operates only on the index, it's always a
no-op if no tree-ish is given.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---

Maybe we can just disallow --cached without <tree-ish> given for now,
and possibly later allow it with some different behaviour for
conflicts, not sure what the best way forward here is.  We can also
just make it update the index as appropriate, and have it behave
different than 'git checkout' curerntly does when handling conflicts?

 builtin/checkout.c         |  26 ++++++++--
 t/t2016-checkout-patch.sh  |   8 +++
 t/t2026-checkout-cached.sh | 103 +++++++++++++++++++++++++++++++++++++
 t/t9902-completion.sh      |   1 +
 4 files changed, 135 insertions(+), 3 deletions(-)
 create mode 100755 t/t2026-checkout-cached.sh

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 0aef35bbc4..6ba85e9de5 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -45,6 +45,7 @@ struct checkout_opts {
 	int ignore_other_worktrees;
 	int show_progress;
 	int overlay_mode;
+	int cached;
 	/*
 	 * If new checkout options are added, skip_merge_working_tree
 	 * should be updated accordingly.
@@ -288,6 +289,10 @@ static int checkout_paths(const struct checkout_opts *opts,
 		die(_("Cannot update paths and switch to branch '%s' at the same time."),
 		    opts->new_branch);
 
+	if (opts->patch_mode && opts->cached)
+		return run_add_interactive(revision, "--patch=reset",
+					   &opts->pathspec);
+
 	if (opts->patch_mode)
 		return run_add_interactive(revision, "--patch=checkout",
 					   &opts->pathspec);
@@ -319,7 +324,9 @@ static int checkout_paths(const struct checkout_opts *opts,
 				 * the current index, which means that it should 
 				 * be removed.
 				 */
-				ce->ce_flags |= CE_MATCHED | CE_REMOVE | CE_WT_REMOVE;
+				ce->ce_flags |= CE_MATCHED | CE_REMOVE;
+				if (!opts->cached)
+					ce->ce_flags |= CE_WT_REMOVE;
 				continue;
 			} else {
 				/*
@@ -392,6 +399,9 @@ static int checkout_paths(const struct checkout_opts *opts,
 	for (pos = 0; pos < active_nr; pos++) {
 		struct cache_entry *ce = active_cache[pos];
 		if (ce->ce_flags & CE_MATCHED) {
+			if (opts->cached) {
+				continue;
+			}
 			if (!ce_stage(ce)) {
 				errs |= checkout_entry(ce, &state, NULL);
 				continue;
@@ -571,6 +581,11 @@ static int skip_merge_working_tree(const struct checkout_opts *opts,
 	 * not tested here
 	 */
 
+	/*
+	 * opts->cached cannot be used with switching branches so is
+	 * not tested here
+	 */
+
 	/*
 	 * If we aren't creating a new branch any changes or updates will
 	 * happen in the existing branch.  Since that could only be updating
@@ -1207,9 +1222,13 @@ static int checkout_branch(struct checkout_opts *opts,
 		die(_("'%s' cannot be used with switching branches"),
 		    "--patch");
 
-	if (!opts->overlay_mode)
+	if (opts->overlay_mode != -1)
+		die(_("'%s' cannot be used with switching branches"),
+		    "--overlay/--no-overlay");
+
+	if (opts->cached)
 		die(_("'%s' cannot be used with switching branches"),
-		    "--no-overlay");
+		    "--cached");
 
 	if (opts->writeout_stage)
 		die(_("'%s' cannot be used with switching branches"),
@@ -1300,6 +1319,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 			    PARSE_OPT_OPTARG, option_parse_recurse_submodules_worktree_updater },
 		OPT_BOOL(0, "progress", &opts.show_progress, N_("force progress reporting")),
 		OPT_BOOL(0, "overlay", &opts.overlay_mode, N_("use overlay mode")),
+		OPT_BOOL(0, "cached", &opts.cached, N_("work on the index only")),
 		OPT_END(),
 	};
 
diff --git a/t/t2016-checkout-patch.sh b/t/t2016-checkout-patch.sh
index 47aeb0b167..e8774046e0 100755
--- a/t/t2016-checkout-patch.sh
+++ b/t/t2016-checkout-patch.sh
@@ -108,6 +108,14 @@ test_expect_success PERL 'path limiting works: foo inside dir' '
 	verify_state dir/foo head head
 '
 
+test_expect_success PERL 'git checkout --cached -p' '
+	set_and_save_state dir/foo work work &&
+	test_write_lines n y | git checkout --cached -p >output &&
+	verify_state dir/foo work head &&
+	verify_saved_state bar &&
+	test_i18ngrep "Unstage" output
+'
+
 test_expect_success PERL 'none of this moved HEAD' '
 	verify_saved_head
 '
diff --git a/t/t2026-checkout-cached.sh b/t/t2026-checkout-cached.sh
new file mode 100755
index 0000000000..1b66192727
--- /dev/null
+++ b/t/t2026-checkout-cached.sh
@@ -0,0 +1,103 @@
+#!/bin/sh
+
+test_description='checkout --cached <pathspec>'
+
+. ./test-lib.sh
+
+test_expect_success 'checkout --cached <pathspec>' '
+	echo 1 >file1 &&
+	echo 2 >file2 &&
+	git add file1 file2 &&
+	test_tick &&
+	git commit -m files &&
+	git rm file2 &&
+	echo 3 >file3 &&
+	echo 4 >file1 &&
+	git add file1 file3 &&
+	git checkout --cached HEAD -- file1 file2 &&
+	test_must_fail git diff --quiet &&
+
+	cat >expect <<-\EOF &&
+	diff --git a/file1 b/file1
+	index d00491f..b8626c4 100644
+	--- a/file1
+	+++ b/file1
+	@@ -1 +1 @@
+	-1
+	+4
+	diff --git a/file2 b/file2
+	deleted file mode 100644
+	index 0cfbf08..0000000
+	--- a/file2
+	+++ /dev/null
+	@@ -1 +0,0 @@
+	-2
+	EOF
+	git diff >actual &&
+	test_cmp expect actual &&
+
+	cat >expect <<-\EOF &&
+	diff --git a/file3 b/file3
+	new file mode 100644
+	index 0000000..00750ed
+	--- /dev/null
+	+++ b/file3
+	@@ -0,0 +1 @@
+	+3
+	EOF
+	git diff --cached >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'checking out an unmodified path is a no-op' '
+	git reset --hard &&
+	git checkout --cached HEAD -- file1 &&
+	git diff-files --exit-code &&
+	git diff-index --cached --exit-code HEAD
+'
+
+test_expect_success 'checking out specific path that is unmerged' '
+	test_commit file3 file3 &&
+	git rm --cached file2 &&
+	echo 1234 >file2 &&
+	F1=$(git rev-parse HEAD:file1) &&
+	F2=$(git rev-parse HEAD:file2) &&
+	F3=$(git rev-parse HEAD:file3) &&
+	{
+		echo "100644 $F1 1	file2" &&
+		echo "100644 $F2 2	file2" &&
+		echo "100644 $F3 3	file2"
+	} | git update-index --index-info &&
+	git ls-files -u &&
+	git checkout --cached HEAD file2 &&
+	test_must_fail git diff --quiet &&
+	git diff-index --exit-code --cached HEAD
+'
+
+test_expect_success '--cached without --no-overlay does not remove entry from index' '
+	test_must_fail git checkout --cached HEAD^ file3 &&
+	git ls-files --error-unmatch -- file3
+'
+
+test_expect_success 'file is removed from the index with --no-overlay' '
+	git checkout --cached --no-overlay HEAD^ file3 &&
+	test_path_is_file file3 &&
+	test_must_fail git ls-files --error-unmatch -- file3
+'
+
+test_expect_success 'test checkout --cached --no-overlay at given paths' '
+	mkdir sub &&
+	>sub/file1 &&
+	>sub/file2 &&
+	git update-index --add sub/file1 sub/file2 &&
+	T=$(git write-tree) &&
+	git checkout --cached --no-overlay HEAD sub/file2 &&
+	test_must_fail git diff --quiet &&
+	U=$(git write-tree) &&
+	echo "$T" &&
+	echo "$U" &&
+	test_must_fail git diff-index --cached --exit-code "$T" &&
+	test "$T" != "$U"
+'
+
+test_done
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index a3fd9a9630..cbc304ace8 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -1437,6 +1437,7 @@ test_expect_success 'double dash "git checkout"' '
 	--no-quiet Z
 	--no-... Z
 	--overlay Z
+	--cached Z
 	EOF
 '
 
-- 
2.20.0.405.gbc1bbc6f85


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH 7/8] checkout: allow ignoring unmatched pathspec
  2018-12-09 20:04 [PATCH 0/8] introduce no-overlay and cached mode in git checkout Thomas Gummerer
                   ` (5 preceding siblings ...)
  2018-12-09 20:04 ` [PATCH 6/8] checkout: add --cached option Thomas Gummerer
@ 2018-12-09 20:04 ` Thomas Gummerer
  2018-12-10 16:51   ` Duy Nguyen
  2018-12-10 20:25   ` Elijah Newren
  2018-12-09 20:04 ` [PATCH 8/8] stash: use git checkout --no-overlay Thomas Gummerer
                   ` (4 subsequent siblings)
  11 siblings, 2 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-09 20:04 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Thomas Gummerer

Currently when 'git checkout -- <pathspec>...' is invoked with
multiple pathspecs, where one or more of the pathspecs don't match
anything, checkout errors out.

This can be inconvenient in some cases, such as when using git
checkout from a script.  Introduce a new --ignore-unmatched option,
which which allows us to ignore a non-matching pathspec instead of
erroring out.

In a subsequent commit we're going to start using 'git checkout' in
'git stash' and are going to make use of this feature.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 builtin/checkout.c        | 10 +++++++++-
 t/t2022-checkout-paths.sh |  9 +++++++++
 t/t9902-completion.sh     |  1 +
 3 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 6ba85e9de5..7e7b5cd1d3 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -46,6 +46,7 @@ struct checkout_opts {
 	int show_progress;
 	int overlay_mode;
 	int cached;
+	int ignore_unmatched;
 	/*
 	 * If new checkout options are added, skip_merge_working_tree
 	 * should be updated accordingly.
@@ -358,7 +359,8 @@ static int checkout_paths(const struct checkout_opts *opts,
 			ce->ce_flags |= CE_MATCHED;
 	}
 
-	if (report_path_error(ps_matched, &opts->pathspec, opts->prefix)) {
+	if (!opts->ignore_unmatched &&
+	    report_path_error(ps_matched, &opts->pathspec, opts->prefix)) {
 		free(ps_matched);
 		return 1;
 	}
@@ -586,6 +588,11 @@ static int skip_merge_working_tree(const struct checkout_opts *opts,
 	 * not tested here
 	 */
 
+	/*
+	 * opts->ignore_unmatched cannot be used with switching branches so is
+	 * not tested here
+	 */
+
 	/*
 	 * If we aren't creating a new branch any changes or updates will
 	 * happen in the existing branch.  Since that could only be updating
@@ -1320,6 +1327,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 		OPT_BOOL(0, "progress", &opts.show_progress, N_("force progress reporting")),
 		OPT_BOOL(0, "overlay", &opts.overlay_mode, N_("use overlay mode")),
 		OPT_BOOL(0, "cached", &opts.cached, N_("work on the index only")),
+		OPT_BOOL(0, "ignore-unmatched", &opts.ignore_unmatched, N_("don't error on unmatched pathspecs")),
 		OPT_END(),
 	};
 
diff --git a/t/t2022-checkout-paths.sh b/t/t2022-checkout-paths.sh
index fc3eb43b89..b44cdf7b63 100755
--- a/t/t2022-checkout-paths.sh
+++ b/t/t2022-checkout-paths.sh
@@ -78,4 +78,13 @@ test_expect_success 'do not touch files that are already up-to-date' '
 	test_cmp expect actual
 '
 
+test_expect_success 'checkout --ignore-unmatched' '
+	test_commit file1 &&
+	echo changed >file1.t &&
+	git checkout --ignore-unmatched -- file1.t unknown-file &&
+	echo file1 >expect &&
+	test_cmp expect file1.t
+
+'
+
 test_done
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index cbc304ace8..475debcf95 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -1438,6 +1438,7 @@ test_expect_success 'double dash "git checkout"' '
 	--no-... Z
 	--overlay Z
 	--cached Z
+	--ignore-unmatched Z
 	EOF
 '
 
-- 
2.20.0.405.gbc1bbc6f85


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH 8/8] stash: use git checkout --no-overlay
  2018-12-09 20:04 [PATCH 0/8] introduce no-overlay and cached mode in git checkout Thomas Gummerer
                   ` (6 preceding siblings ...)
  2018-12-09 20:04 ` [PATCH 7/8] checkout: allow ignoring unmatched pathspec Thomas Gummerer
@ 2018-12-09 20:04 ` Thomas Gummerer
  2018-12-10 20:26   ` Elijah Newren
  2018-12-10  3:47 ` [PATCH 0/8] introduce no-overlay and cached mode in git checkout Junio C Hamano
                   ` (3 subsequent siblings)
  11 siblings, 1 reply; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-09 20:04 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Thomas Gummerer

Now that we have 'git checkout --no-overlay', we can use it in git
stash, making the codepaths for 'git stash push' with and without
pathspec more similar, and thus easier to follow.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---

As mentioned in the cover letter, not sure if we want to apply this
now.  There are two reasons I did this:
- Showing the new functionality of git checkout
- Increased test coverage, as we are running the new code with all git
  stash tests for free, which helped look at some cases that I was
  missing initially.

 git-stash.sh | 12 ++++--------
 1 file changed, 4 insertions(+), 8 deletions(-)

diff --git a/git-stash.sh b/git-stash.sh
index 94793c1a91..67be04d996 100755
--- a/git-stash.sh
+++ b/git-stash.sh
@@ -314,19 +314,15 @@ push_stash () {
 
 	if test -z "$patch_mode"
 	then
-		test "$untracked" = "all" && CLEAN_X_OPTION=-x || CLEAN_X_OPTION=
-		if test -n "$untracked" && test $# = 0
+		test "$untracked" = "all" && CLEAN_X_OPTION=-X || CLEAN_X_OPTION=
+		if test -n "$untracked"
 		then
-			git clean --force --quiet -d $CLEAN_X_OPTION
+			git clean --force --quiet -d $CLEAN_X_OPTION -- "$@"
 		fi
 
 		if test $# != 0
 		then
-			test -z "$untracked" && UPDATE_OPTION="-u" || UPDATE_OPTION=
-			test "$untracked" = "all" && FORCE_OPTION="--force" || FORCE_OPTION=
-			git add $UPDATE_OPTION $FORCE_OPTION -- "$@"
-			git diff-index -p --cached --binary HEAD -- "$@" |
-			git apply --index -R
+			git checkout --quiet --no-overlay --ignore-unmatched HEAD -- "$@"
 		else
 			git reset --hard -q
 		fi
-- 
2.20.0.405.gbc1bbc6f85


^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 0/8] introduce no-overlay and cached mode in git checkout
  2018-12-09 20:04 [PATCH 0/8] introduce no-overlay and cached mode in git checkout Thomas Gummerer
                   ` (7 preceding siblings ...)
  2018-12-09 20:04 ` [PATCH 8/8] stash: use git checkout --no-overlay Thomas Gummerer
@ 2018-12-10  3:47 ` Junio C Hamano
  2018-12-20  8:43   ` Thomas Gummerer
  2018-12-10 17:06 ` Duy Nguyen
                   ` (2 subsequent siblings)
  11 siblings, 1 reply; 81+ messages in thread
From: Junio C Hamano @ 2018-12-10  3:47 UTC (permalink / raw)
  To: Thomas Gummerer; +Cc: git, Nguyễn Thái Ngọc Duy, Elijah Newren

Thomas Gummerer <t.gummerer@gmail.com> writes:

> Basically the idea is to also delete files when the match <pathspec>
> in 'git checkout <tree-ish> -- <pathspec>' in the current tree, but
> don't match <pathspec> in <tree-ish>.

I cannot quite parse it, but perhaps.

	"git checkout --no-overlay <tree-ish> -- <pathspec>" can
	remove paths in the index and in the working tree that match
	<pathspec>, if they do not appear in <tree-ish>.

If a new file D/F is in the index and in the working tree but not in
HEAD, "git checkout HEAD D/" or "git checkout HEAD D/F" would not
remove D/F from the index or the working tree.

With the --no-overlay option, it would, and that is often closer to
the wish of the user who wanted to say "restore the working tree
state of D/ (or D/F) from the state recorded in HEAD".

> The final step in the series is to actually make use of this in 'git
> stash', which simplifies the code there a bit.  I am however happy to
> hold off on this step until the stash-in-C series is merged, so we
> don't delay that further.

I think that is probably a good idea, for now.

> In addition to the no-overlay mode, we also add a --cached mode, which
> works only on the index, thus similar to 'git reset <tree-ish> -- <pathspec>'.
>
> Actually deprecating 'git reset <tree-ish> -- <pathspec>' should come
> later,...

Or we may not even need to deprecate it.  IIRC, what "stash" wished
to exist was "git reset --hard <tree-ish> -- <pathspec>", which, if
the command followed "--cached/--index" convention, would have been
called "git reset --index ...".  Did we actually have the need for
"--cached" mode?

> probably not before Duy's restore-files command lands, as 'git
> checkout --no-overlay <tree-ish> -- <pathspec>' is a bit cumbersome to
> type compared to 'git reset <tree-ish> -- <pathspec>'.

Yes, between "checkout --cached" and "checkout --no-overlay", the
latter is much more important, as the latter is what a missing "git
reset --hard <tree-ish> -- <pathspec>" would have been, but the
former can be written with an existing command.

> My hope is also that the no-overlay mode could become the new default
> in the restore-files command Duy is currently working on.

Yup, that is my hope, too ;-).

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 1/8] move worktree tests to t24*
  2018-12-09 20:04 ` [PATCH 1/8] move worktree tests to t24* Thomas Gummerer
@ 2018-12-10  3:48   ` Junio C Hamano
  2018-12-10 15:32   ` Duy Nguyen
  1 sibling, 0 replies; 81+ messages in thread
From: Junio C Hamano @ 2018-12-10  3:48 UTC (permalink / raw)
  To: Thomas Gummerer; +Cc: git, Nguyễn Thái Ngọc Duy, Elijah Newren

Thomas Gummerer <t.gummerer@gmail.com> writes:

> The 'git worktree' command used to be just another mode in 'git
> checkout', namely 'git checkout --to'.  When the tests for the latter
> were retrofitted for the former, the test name was adjusted, but the
> test number was kept, even though the test is testing a different
> command now.  t/README states: "Second digit tells the particular
> command we are testing.", so 'git worktree' should have a separate
> number just for itself.

That probably was written in the old world where there were only 10
commands in each category ;-) Nevertheless I have no problem with
this move (and I do not think there are in-flight topics in these
areas).

Thanks.

>
> Move the worktree tests to t24* to adhere to that guideline. We're
> going to make use of the free'd up numbers in a subsequent commit.
>
> Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> ---
>  t/{t2025-worktree-add.sh => t2400-worktree-add.sh}     | 0
>  t/{t2026-worktree-prune.sh => t2401-worktree-prune.sh} | 0
>  t/{t2027-worktree-list.sh => t2402-worktree-list.sh}   | 0
>  3 files changed, 0 insertions(+), 0 deletions(-)
>  rename t/{t2025-worktree-add.sh => t2400-worktree-add.sh} (100%)
>  rename t/{t2026-worktree-prune.sh => t2401-worktree-prune.sh} (100%)
>  rename t/{t2027-worktree-list.sh => t2402-worktree-list.sh} (100%)
>
> diff --git a/t/t2025-worktree-add.sh b/t/t2400-worktree-add.sh
> similarity index 100%
> rename from t/t2025-worktree-add.sh
> rename to t/t2400-worktree-add.sh
> diff --git a/t/t2026-worktree-prune.sh b/t/t2401-worktree-prune.sh
> similarity index 100%
> rename from t/t2026-worktree-prune.sh
> rename to t/t2401-worktree-prune.sh
> diff --git a/t/t2027-worktree-list.sh b/t/t2402-worktree-list.sh
> similarity index 100%
> rename from t/t2027-worktree-list.sh
> rename to t/t2402-worktree-list.sh

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 1/8] move worktree tests to t24*
  2018-12-09 20:04 ` [PATCH 1/8] move worktree tests to t24* Thomas Gummerer
  2018-12-10  3:48   ` Junio C Hamano
@ 2018-12-10 15:32   ` Duy Nguyen
  2018-12-11 21:50     ` Thomas Gummerer
  1 sibling, 1 reply; 81+ messages in thread
From: Duy Nguyen @ 2018-12-10 15:32 UTC (permalink / raw)
  To: Thomas Gummerer; +Cc: Git Mailing List, Junio C Hamano, Elijah Newren

On Sun, Dec 9, 2018 at 9:04 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
>
> The 'git worktree' command used to be just another mode in 'git
> checkout', namely 'git checkout --to'.  When the tests for the latter
> were retrofitted for the former, the test name was adjusted, but the
> test number was kept, even though the test is testing a different
> command now.  t/README states: "Second digit tells the particular
> command we are testing.", so 'git worktree' should have a separate
> number just for itself.
>
> Move the worktree tests to t24* to adhere to that guideline. We're
> going to make use of the free'd up numbers in a subsequent commit.
>
> Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> ---
>  t/{t2025-worktree-add.sh => t2400-worktree-add.sh}     | 0
>  t/{t2026-worktree-prune.sh => t2401-worktree-prune.sh} | 0
>  t/{t2027-worktree-list.sh => t2402-worktree-list.sh}   | 0
>  3 files changed, 0 insertions(+), 0 deletions(-)
>  rename t/{t2025-worktree-add.sh => t2400-worktree-add.sh} (100%)
>  rename t/{t2026-worktree-prune.sh => t2401-worktree-prune.sh} (100%)
>  rename t/{t2027-worktree-list.sh => t2402-worktree-list.sh} (100%)

Heh.. I did the same thing (in my unsent switch-branch/restore-files
series) and even used the same 24xx range :D You probably want to move
t2028 and t2029 too (not sure if they have landed on 'master')
-- 
Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 2/8] entry: factor out unlink_entry function
  2018-12-09 20:04 ` [PATCH 2/8] entry: factor out unlink_entry function Thomas Gummerer
@ 2018-12-10 15:49   ` Duy Nguyen
  2018-12-10 17:23     ` Elijah Newren
                       ` (2 more replies)
  0 siblings, 3 replies; 81+ messages in thread
From: Duy Nguyen @ 2018-12-10 15:49 UTC (permalink / raw)
  To: Thomas Gummerer; +Cc: Git Mailing List, Junio C Hamano, Elijah Newren

On Sun, Dec 9, 2018 at 9:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
>
> Factor out the 'unlink_entry()' function from unpack-trees.c to
> entry.c.  It will be used in other places as well in subsequent
> steps.
>
> As it's no longer a static function, also move the documentation to
> the header file to make it more discoverable.
>
> Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> ---
>  cache.h        |  5 +++++
>  entry.c        | 15 +++++++++++++++
>  unpack-trees.c | 19 -------------------
>  3 files changed, 20 insertions(+), 19 deletions(-)
>
> diff --git a/cache.h b/cache.h
> index ca36b44ee0..c1c953e810 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -1542,6 +1542,11 @@ struct checkout {
>  extern int checkout_entry(struct cache_entry *ce, const struct checkout *state, char *topath);
>  extern void enable_delayed_checkout(struct checkout *state);
>  extern int finish_delayed_checkout(struct checkout *state);
> +/*
> + * Unlink the last component and schedule the leading directories for
> + * removal, such that empty directories get removed.
> + */
> +extern void unlink_entry(const struct cache_entry *ce);

I'm torn. We try to remove 'extern' but I can see you may want to add
it here to be consistent with others. And removing extern even from
functions from entry.c only would cause some conflicts.

I wonder if we should move the 'removal' variable in symlinks to
'struct checkout' to reduce another global variable. But I guess
that's the problem for another day. It's not the focus of this series.
-- 
Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 3/8] entry: support CE_WT_REMOVE flag in checkout_entry
  2018-12-09 20:04 ` [PATCH 3/8] entry: support CE_WT_REMOVE flag in checkout_entry Thomas Gummerer
@ 2018-12-10 15:58   ` Duy Nguyen
  2018-12-11  2:28     ` Junio C Hamano
  2018-12-10 17:49   ` Elijah Newren
  1 sibling, 1 reply; 81+ messages in thread
From: Duy Nguyen @ 2018-12-10 15:58 UTC (permalink / raw)
  To: Thomas Gummerer; +Cc: Git Mailing List, Junio C Hamano, Elijah Newren

On Sun, Dec 9, 2018 at 9:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
>
> 'checkout_entry()' currently only supports creating new entries in the
> working tree, but not deleting them.  Add the ability to remove
> entries at the same time if the entry is marked with the CE_WT_REMOVE
> flag.
>
> Currently this doesn't have any effect, as the CE_WT_REMOVE flag is
> only used in unpack-tree, however we will make use of this in a
> subsequent step in the series.
>
> Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> ---
>  entry.c | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/entry.c b/entry.c
> index 3ec148ceee..cd1c6601b6 100644
> --- a/entry.c
> +++ b/entry.c
> @@ -441,6 +441,13 @@ int checkout_entry(struct cache_entry *ce,
>         static struct strbuf path = STRBUF_INIT;
>         struct stat st;
>
> +       if (ce->ce_flags & CE_WT_REMOVE) {
> +               if (topath)
> +                       BUG("Can't remove entry to a path");
> +               unlink_entry(ce);
> +               return 0;
> +       }

This makes the path counting in nd/checkout-noisy less accurate. But
it's not your fault of course.

Junio, do you still want to merge that series down to 'next' or drop
it? If it will be merged down, I'll keep a note and fix it once this
one lands too.

> +
>         if (topath)
>                 return write_entry(ce, topath, state, 1);
>
> --
> 2.20.0.405.gbc1bbc6f85
>


-- 
Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries
  2018-12-09 20:04 ` [PATCH 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries Thomas Gummerer
@ 2018-12-10 16:08   ` Duy Nguyen
  2018-12-10 18:09     ` Elijah Newren
  2018-12-11  2:42     ` Junio C Hamano
  0 siblings, 2 replies; 81+ messages in thread
From: Duy Nguyen @ 2018-12-10 16:08 UTC (permalink / raw)
  To: Thomas Gummerer; +Cc: Git Mailing List, Junio C Hamano, Elijah Newren

On Sun, Dec 9, 2018 at 9:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
>
> When marking cache entries for removal, and later removing them all at
> once using 'remove_marked_cache_entries()', cache entries currently
> have to be invalidated manually in the cache tree and in the untracked
> cache.
>
> Add an invalidate flag to the function.  With the flag set, the
> function will take care of invalidating the path in the cache tree and
> in the untracked cache.
>
> This will be useful in a subsequent commit.
>
> Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> ---
>
> For the two current callsites, unpack-trees seems to do this
> invalidation itself internally.

I'm still a bit scared of this invalidation business in unpack-trees.
The thing is, we handle two separate index_state there, src_index and
result and invalidation has to be done on the right one (because index
extensions are on src_index until the very end of unpack-trees;
invalidating on 'result' would be no-op and wrong).
remove_marked_cache_entries() seems to be called on 'result' while
invalidate_ce_path() is on src_index, hm....

> I don't quite understand why we don't
> need it in split-index mode though.  I assume it's because the cache
> tree in the main index would already have been invalidated?  I didn't
> have much time to dig, but couldn't produce any failures with it
> either, so I assume not invalidating paths is the right thing to do
> here.

Yeah I think it's because cache-tree and untracked cache are already
properly invalidated. This merge base thingy is done when we load the
index files up, not when we write them down. The "front" index may
record that a few paths in the base index are no longer valid and need
to be deleted. But untracked cache and cache-tree both should have
recorded that same info when these paths are marked for delete at
index write time.
-- 
Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 5/8] checkout: introduce --{,no-}overlay option
  2018-12-09 20:04 ` [PATCH 5/8] checkout: introduce --{,no-}overlay option Thomas Gummerer
@ 2018-12-10 16:42   ` Duy Nguyen
  2018-12-11 22:42     ` Thomas Gummerer
  2018-12-10 18:19   ` Elijah Newren
  1 sibling, 1 reply; 81+ messages in thread
From: Duy Nguyen @ 2018-12-10 16:42 UTC (permalink / raw)
  To: Thomas Gummerer; +Cc: Git Mailing List, Junio C Hamano, Elijah Newren

On Sun, Dec 9, 2018 at 9:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> @@ -302,15 +310,29 @@ static int checkout_paths(const struct checkout_opts *opts,
>                 ce->ce_flags &= ~CE_MATCHED;
>                 if (!opts->ignore_skipworktree && ce_skip_worktree(ce))
>                         continue;
> -               if (opts->source_tree && !(ce->ce_flags & CE_UPDATE))
> -                       /*
> -                        * "git checkout tree-ish -- path", but this entry
> -                        * is in the original index; it will not be checked
> -                        * out to the working tree and it does not matter
> -                        * if pathspec matched this entry.  We will not do
> -                        * anything to this entry at all.
> -                        */
> -                       continue;
> +               if (opts->source_tree && !(ce->ce_flags & CE_UPDATE)) {
> +                       if (!opts->overlay_mode &&
> +                           ce_path_match(&the_index, ce, &opts->pathspec, ps_matched)) {
> +                               /*
> +                                * "git checkout --no-overlay <tree-ish> -- path",
> +                                * and the path is not in tree-ish, but is in
> +                                * the current index, which means that it should
> +                                * be removed.
> +                                */
> +                               ce->ce_flags |= CE_MATCHED | CE_REMOVE | CE_WT_REMOVE;
> +                               continue;
> +                       } else {

In non-overlay mode but when pathspec does not match, we come here too.

> +                               /*
> +                                * "git checkout tree-ish -- path", but this
> +                                * entry is in the original index; it will not

I think the missing key point in this comment block is "..is in the
original index _and it's not in tree-ish_". In non-overlay mode, if
pathspec does not match then it's safe to ignore too. But this logic
starts too get to complex and hurt my brain.

> +                                * be checked out to the working tree and it
> +                                * does not matter if pathspec matched this
> +                                * entry.  We will not do anything to this entry
> +                                * at all.
> +                                */
> +                               continue;
> +                       }
> +               }
>                 /*
>                  * Either this entry came from the tree-ish we are
>                  * checking the paths out of, or we are checking out

> @@ -1266,6 +1299,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
>                             "checkout", "control recursive updating of submodules",
>                             PARSE_OPT_OPTARG, option_parse_recurse_submodules_worktree_updater },
>                 OPT_BOOL(0, "progress", &opts.show_progress, N_("force progress reporting")),
> +               OPT_BOOL(0, "overlay", &opts.overlay_mode, N_("use overlay mode")),

maybe add " (default)" to the help string.

>                 OPT_END(),
>         };
>
-- 
Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 6/8] checkout: add --cached option
  2018-12-09 20:04 ` [PATCH 6/8] checkout: add --cached option Thomas Gummerer
@ 2018-12-10 16:49   ` Duy Nguyen
  2018-12-11  3:13     ` Junio C Hamano
  2018-12-10 18:42   ` Elijah Newren
  1 sibling, 1 reply; 81+ messages in thread
From: Duy Nguyen @ 2018-12-10 16:49 UTC (permalink / raw)
  To: Thomas Gummerer; +Cc: Git Mailing List, Junio C Hamano, Elijah Newren

On Sun, Dec 9, 2018 at 9:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
>
> Add a new --cached option to git checkout, which works only on the
> index, but not the working tree, similar to what 'git reset <tree-ish>
> -- <pathspec>... does.

Elijah wanted another mode (and I agree) that modifies worktree but
leaves the index alone. This is most useful (or least confusing) when
used with <tree-ish> and would be default in restore-files. I'm not
saying you have to implement it, but how do the new command line
options are designed to make sense?

I guess if --cached is "update index only" then --no-cached goes back
to the default "update both worktree and index" and we need a another
option for "worktree only"? Can we have one option with three possible
values (index-only, index-and-worktree, worktree-only) maybe?
-- 
Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 7/8] checkout: allow ignoring unmatched pathspec
  2018-12-09 20:04 ` [PATCH 7/8] checkout: allow ignoring unmatched pathspec Thomas Gummerer
@ 2018-12-10 16:51   ` Duy Nguyen
  2018-12-11 22:23     ` Thomas Gummerer
  2018-12-10 20:25   ` Elijah Newren
  1 sibling, 1 reply; 81+ messages in thread
From: Duy Nguyen @ 2018-12-10 16:51 UTC (permalink / raw)
  To: Thomas Gummerer; +Cc: Git Mailing List, Junio C Hamano, Elijah Newren

On Sun, Dec 9, 2018 at 9:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
>
> Currently when 'git checkout -- <pathspec>...' is invoked with
> multiple pathspecs, where one or more of the pathspecs don't match
> anything, checkout errors out.
>
> This can be inconvenient in some cases, such as when using git
> checkout from a script.

Wait, should scripts go with read-tree, checkout-index or other
plumbing commands instead?
-- 
Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 0/8] introduce no-overlay and cached mode in git checkout
  2018-12-09 20:04 [PATCH 0/8] introduce no-overlay and cached mode in git checkout Thomas Gummerer
                   ` (8 preceding siblings ...)
  2018-12-10  3:47 ` [PATCH 0/8] introduce no-overlay and cached mode in git checkout Junio C Hamano
@ 2018-12-10 17:06 ` Duy Nguyen
  2018-12-10 17:18 ` Elijah Newren
  2018-12-20 13:48 ` [PATCH v2 0/8] introduce no-overlay " Thomas Gummerer
  11 siblings, 0 replies; 81+ messages in thread
From: Duy Nguyen @ 2018-12-10 17:06 UTC (permalink / raw)
  To: Thomas Gummerer; +Cc: Git Mailing List, Junio C Hamano, Elijah Newren

On Sun, Dec 9, 2018 at 9:04 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> My hope is also that the no-overlay mode could become the new default
> in the restore-files command Duy is currently working on.

I already wrote something like that in git-restore-files.txt even
though the implementation is not there :D

There will be a hell of conflicts when the two series enter 'pu' (or
even worse, when the third one to update worktree only appears) so I'm
going to send the switch-branch/restore-files series out but mostly to
gather comments and will rebase once the other series land.

(Alternatively I'll split my series and let the switch-branch part
land first, may be simpler)
-- 
Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 0/8] introduce no-overlay and cached mode in git checkout
  2018-12-09 20:04 [PATCH 0/8] introduce no-overlay and cached mode in git checkout Thomas Gummerer
                   ` (9 preceding siblings ...)
  2018-12-10 17:06 ` Duy Nguyen
@ 2018-12-10 17:18 ` Elijah Newren
  2018-12-10 18:37   ` Duy Nguyen
  2018-12-11 22:52   ` Thomas Gummerer
  2018-12-20 13:48 ` [PATCH v2 0/8] introduce no-overlay " Thomas Gummerer
  11 siblings, 2 replies; 81+ messages in thread
From: Elijah Newren @ 2018-12-10 17:18 UTC (permalink / raw)
  To: Thomas Gummerer
  Cc: Git Mailing List, Junio C Hamano, Nguyễn Thái Ngọc

On Sun, Dec 9, 2018 at 12:04 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
>
> Here's the series I mentioned a couple of times on the list already,
> introducing a no-overlay mode in 'git checkout'.  The inspiration for
> this came from Junios message in [*1*].
>
> Basically the idea is to also delete files when the match <pathspec>
> in 'git checkout <tree-ish> -- <pathspec>' in the current tree, but
> don't match <pathspec> in <tree-ish>.  The rest of the cases are
> already properly taken care of by 'git checkout'.

Yes, but I'd put it a little differently:

"""
Basically, the idea is when the user run "git checkout --no-overlay
<tree-ish> -- <pathspec>" that the given pathspecs should exactly
match <tree-ish> after the operation completes.  This means that we
also want to delete files that match <pathspec> if those paths are not
found in <tree-ish>.
"""

...and maybe even toss in some comments about the fact that this is
the way git checkout should have always behaved, it just traditionally
hasn't.  (You could also work in comments about how with this new mode
the user can run git diff afterward with the given commit-ish and
pathspecs and get back an empty diff, as expected, which wasn't true
before.  But maybe I'm belaboring the point.)


> The final step in the series is to actually make use of this in 'git
> stash', which simplifies the code there a bit.  I am however happy to
> hold off on this step until the stash-in-C series is merged, so we
> don't delay that further.
>
> In addition to the no-overlay mode, we also add a --cached mode, which
> works only on the index, thus similar to 'git reset <tree-ish> -- <pathspec>'.

If you're adding a --cached mode to make it only work on the index,
should there be a similar mode to allow it to only work on the working
tree?  (I'm not as concerned with that here, but I really think the
new restore-files command by default should only operate on the
working tree, and then have options to affect the index either in
addition or instead of the working tree.)

> Actually deprecating 'git reset <tree-ish> -- <pathspec>' should come
> later, probably not before Duy's restore-files command lands, as 'git
> checkout --no-overlay <tree-ish> -- <pathspec>' is a bit cumbersome to
> type compared to 'git reset <tree-ish> -- <pathspec>'.

Makes sense.

> My hope is also that the no-overlay mode could become the new default
> in the restore-files command Duy is currently working on.

Absolutely, yes.  I don't want another broken command.  :-)


> No documentation yet, as I wanted to get this out for review first.
> I'm not familiar with most of the code I touched here, so there may
> well be much better ways to implement some of this, that I wasn't able
> to figure out.  I'd be very happy with some feedback around that.
>
> Another thing I'm not sure about is how to deal with conflicts.  In
> the cached mode this patch series is not dealing with it at all, as
> 'git checkout -- <pathspec>' when pathspec matches a file with
> conflicts doesn't update the index.  For the no-overlay mode, the file
> is removed if the corresponding stage is not found in the index.  I'm
> however not sure this is the right thing to do in all cases?

Here's how I'd go about analyzing that...

If the user passes a <tree-ish>, then the answer about what to do is
pretty obvious; the <tree-ish> didn't have conflicts, so conflicted
paths in the index that match the pathspec should be overwritten with
whatever version of those paths existed in <tree-ish> (possibly
implying deletion of some paths).

Also, as you point out, --cached means only modify the index and not
the working tree; so if they specify both --cached and provide no
tree, then they've specified a no-op.

So it's only interesting when you have conflicts in the index and
specify --no-overlay without a <tree-ish> or --cached.  This boils
down to "how do we update the working tree to match the index, when
the index is conflicted?"  A couple points to consider:
  * This is somewhat of an edge case
  * In the normal case --no-overlay is only different from --overlay
behavior for directories; it'd be nice if that extended to all cases
  * How does this command behave without a <tree-ish> when
--no-overlay is specified and a directory is given for a <pathspec>
and there aren't any conflicts?  Are we being consistent with that
behavior?


However, I think it turns out that the answer is much simpler than all
that initial analysis or what you say you've implemented.  Here's why:

If <pathspec> is a file which is present in both the working tree and
the index and it has conflicts, then "git checkout -- <pathspec>" will
currently throw an error:

$ git checkout -- subdir/counting
error: path 'subdir/counting' is unmerged

In fact, even if every entry in subdir/ is a path that is in both the
index and the working tree (so that --no-overlay and --overlay ought
to behave the same), if any one of the files in subdir is conflicted,
attempting to checkout the subdir will abort with this same error
message and no paths will be updated at all:

$ git checkout -- subdir
error: path 'subdir/counting' is unmerged

as such, the answer with what to do with --no-overlay mode is pretty
clear: if the <pathspec> matches _any_ path that is conflicted, simply
throw an error and abort the operation without making any changes at
all.

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 2/8] entry: factor out unlink_entry function
  2018-12-10 15:49   ` Duy Nguyen
@ 2018-12-10 17:23     ` Elijah Newren
  2018-12-10 17:27       ` Duy Nguyen
  2018-12-11  2:23     ` Junio C Hamano
  2018-12-20 13:36     ` Thomas Gummerer
  2 siblings, 1 reply; 81+ messages in thread
From: Elijah Newren @ 2018-12-10 17:23 UTC (permalink / raw)
  To: Nguyễn Thái Ngọc
  Cc: Thomas Gummerer, Git Mailing List, Junio C Hamano

On Mon, Dec 10, 2018 at 7:50 AM Duy Nguyen <pclouds@gmail.com> wrote:
>
> On Sun, Dec 9, 2018 at 9:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> >
> > Factor out the 'unlink_entry()' function from unpack-trees.c to
> > entry.c.  It will be used in other places as well in subsequent
> > steps.
> >
> > As it's no longer a static function, also move the documentation to
> > the header file to make it more discoverable.

I also started using unlink_entry() in another place in a local patch
series that I haven't submitted yet (and which I need to get back to
at some point).  So this will help me too.  :-)

> > Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> > ---
> >  cache.h        |  5 +++++
> >  entry.c        | 15 +++++++++++++++
> >  unpack-trees.c | 19 -------------------
> >  3 files changed, 20 insertions(+), 19 deletions(-)
> >
> > diff --git a/cache.h b/cache.h
> > index ca36b44ee0..c1c953e810 100644
> > --- a/cache.h
> > +++ b/cache.h
> > @@ -1542,6 +1542,11 @@ struct checkout {
> >  extern int checkout_entry(struct cache_entry *ce, const struct checkout *state, char *topath);
> >  extern void enable_delayed_checkout(struct checkout *state);
> >  extern int finish_delayed_checkout(struct checkout *state);
> > +/*
> > + * Unlink the last component and schedule the leading directories for
> > + * removal, such that empty directories get removed.
> > + */
> > +extern void unlink_entry(const struct cache_entry *ce);
>
> I'm torn. We try to remove 'extern' but I can see you may want to add
> it here to be consistent with others. And removing extern even from
> functions from entry.c only would cause some conflicts.
>
> I wonder if we should move the 'removal' variable in symlinks to
> 'struct checkout' to reduce another global variable. But I guess
> that's the problem for another day. It's not the focus of this series.

"move the 'removal' variable in symlinks"?  I'm having a really hard
time parsing that phrase and the sentence it's embedded in.  Could you
reword for me Duy?

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 2/8] entry: factor out unlink_entry function
  2018-12-10 17:23     ` Elijah Newren
@ 2018-12-10 17:27       ` Duy Nguyen
  0 siblings, 0 replies; 81+ messages in thread
From: Duy Nguyen @ 2018-12-10 17:27 UTC (permalink / raw)
  To: Elijah Newren; +Cc: Thomas Gummerer, Git Mailing List, Junio C Hamano

On Mon, Dec 10, 2018 at 6:23 PM Elijah Newren <newren@gmail.com> wrote:
> > I wonder if we should move the 'removal' variable in symlinks to
> > 'struct checkout' to reduce another global variable. But I guess
> > that's the problem for another day. It's not the focus of this series.
>
> "move the 'removal' variable in symlinks"?  I'm having a really hard
> time parsing that phrase and the sentence it's embedded in.  Could you
> reword for me Duy?

Sorry s/in symlinks/&.c/. There's a global variable named 'removal' in
symlinks.c which is used by schedule_dir_for_removal() and this
function in turn is used by unlink_entry().
-- 
Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 3/8] entry: support CE_WT_REMOVE flag in checkout_entry
  2018-12-09 20:04 ` [PATCH 3/8] entry: support CE_WT_REMOVE flag in checkout_entry Thomas Gummerer
  2018-12-10 15:58   ` Duy Nguyen
@ 2018-12-10 17:49   ` Elijah Newren
  2018-12-11 22:00     ` Thomas Gummerer
  1 sibling, 1 reply; 81+ messages in thread
From: Elijah Newren @ 2018-12-10 17:49 UTC (permalink / raw)
  To: Thomas Gummerer
  Cc: Git Mailing List, Junio C Hamano, Nguyễn Thái Ngọc

On Sun, Dec 9, 2018 at 12:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
>
> 'checkout_entry()' currently only supports creating new entries in the
> working tree, but not deleting them.  Add the ability to remove
> entries at the same time if the entry is marked with the CE_WT_REMOVE
> flag.
>
> Currently this doesn't have any effect, as the CE_WT_REMOVE flag is
> only used in unpack-tree, however we will make use of this in a
> subsequent step in the series.
>
> Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> ---
>  entry.c | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/entry.c b/entry.c
> index 3ec148ceee..cd1c6601b6 100644
> --- a/entry.c
> +++ b/entry.c
> @@ -441,6 +441,13 @@ int checkout_entry(struct cache_entry *ce,
>         static struct strbuf path = STRBUF_INIT;
>         struct stat st;
>
> +       if (ce->ce_flags & CE_WT_REMOVE) {
> +               if (topath)
> +                       BUG("Can't remove entry to a path");

Minor nit: This error message is kinda hard to parse, for someone not
that familiar with all the *_entry functions, like myself.  Maybe add
a comment before this line:
    /* No content and thus no path to create, so we have no pathname
to return */
or reword the error slightly?  Or maybe it's fine and I was just
confused from lack of code familiarity, but I'll throw it out there
since I stumbled on it a bit.

> +               unlink_entry(ce);
> +               return 0;
> +       }
> +
>         if (topath)
>                 return write_entry(ce, topath, state, 1);
>
> --
> 2.20.0.405.gbc1bbc6f85

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries
  2018-12-10 16:08   ` Duy Nguyen
@ 2018-12-10 18:09     ` Elijah Newren
  2018-12-10 18:19       ` Duy Nguyen
  2018-12-11 21:59       ` Thomas Gummerer
  2018-12-11  2:42     ` Junio C Hamano
  1 sibling, 2 replies; 81+ messages in thread
From: Elijah Newren @ 2018-12-10 18:09 UTC (permalink / raw)
  To: Nguyễn Thái Ngọc
  Cc: Thomas Gummerer, Git Mailing List, Junio C Hamano

On Mon, Dec 10, 2018 at 8:09 AM Duy Nguyen <pclouds@gmail.com> wrote:
>
> On Sun, Dec 9, 2018 at 9:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> >
> > When marking cache entries for removal, and later removing them all at
> > once using 'remove_marked_cache_entries()', cache entries currently
> > have to be invalidated manually in the cache tree and in the untracked
> > cache.
> >
> > Add an invalidate flag to the function.  With the flag set, the
> > function will take care of invalidating the path in the cache tree and
> > in the untracked cache.
> >
> > This will be useful in a subsequent commit.
> >
> > Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> > ---
> >
> > For the two current callsites, unpack-trees seems to do this
> > invalidation itself internally.
>
> I'm still a bit scared of this invalidation business in unpack-trees.
> The thing is, we handle two separate index_state there, src_index and
> result and invalidation has to be done on the right one (because index
> extensions are on src_index until the very end of unpack-trees;
> invalidating on 'result' would be no-op and wrong).
> remove_marked_cache_entries() seems to be called on 'result' while
> invalidate_ce_path() is on src_index, hm....

Is Thomas avoiding problems here simply because merge is the only
caller of unpack_trees with src_index != dst_index?  Or does src_index
== dst_index for checkout not actually help?

If that does help with the checkout case, then allow me to find a
different way to muddy the waters...  I think I might want to make use
of this function in the merge machinery at some point, so I either
need to figure out how to convince you to verify if all this cache
tree invalidation stuff is sane, or somehow figure out all the
cache_tree stuff stuff myself so I can figure out what is right here.
:-)

> > I don't quite understand why we don't
> > need it in split-index mode though.  I assume it's because the cache
> > tree in the main index would already have been invalidated?  I didn't
> > have much time to dig, but couldn't produce any failures with it
> > either, so I assume not invalidating paths is the right thing to do
> > here.
>
> Yeah I think it's because cache-tree and untracked cache are already
> properly invalidated. This merge base thingy is done when we load the
> index files up, not when we write them down. The "front" index may
> record that a few paths in the base index are no longer valid and need
> to be deleted. But untracked cache and cache-tree both should have
> recorded that same info when these paths are marked for delete at
> index write time.

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries
  2018-12-10 18:09     ` Elijah Newren
@ 2018-12-10 18:19       ` Duy Nguyen
  2018-12-10 18:25         ` Elijah Newren
  2018-12-11 21:59       ` Thomas Gummerer
  1 sibling, 1 reply; 81+ messages in thread
From: Duy Nguyen @ 2018-12-10 18:19 UTC (permalink / raw)
  To: Elijah Newren; +Cc: Thomas Gummerer, Git Mailing List, Junio C Hamano

On Mon, Dec 10, 2018 at 7:09 PM Elijah Newren <newren@gmail.com> wrote:
> > > For the two current callsites, unpack-trees seems to do this
> > > invalidation itself internally.
> >
> > I'm still a bit scared of this invalidation business in unpack-trees.
> > The thing is, we handle two separate index_state there, src_index and
> > result and invalidation has to be done on the right one (because index
> > extensions are on src_index until the very end of unpack-trees;
> > invalidating on 'result' would be no-op and wrong).
> > remove_marked_cache_entries() seems to be called on 'result' while
> > invalidate_ce_path() is on src_index, hm....
>
> Is Thomas avoiding problems here simply because merge is the only
> caller of unpack_trees with src_index != dst_index?  Or does src_index
> == dst_index for checkout not actually help?

I think it would not help. 'result' is a temporary index where we copy
things to (and it does not have anything from the beginning). If you
invalidate stuff in there, you invalidate nothing, regardless whether
dst_index == src_index.

> If that does help with the checkout case, then allow me to find a
> different way to muddy the waters...  I think I might want to make use
> of this function in the merge machinery at some point, so I either
> need to figure out how to convince you to verify if all this cache
> tree invalidation stuff is sane, or somehow figure out all the
> cache_tree stuff stuff myself so I can figure out what is right here.
> :-)

I'm not the unpack-trees man (I think that would still be Junio). And
I'm not saying it's sane either. I think it's just some leftover
things since Linus split "the index" in unpack-tree operation to
'src', 'result' and 'dst' many years ago and nobody was brave enough
to clean it up (then I piled on with untracked cache and split index,
but I did not see it clearly either). That person could be you ;-)
-- 
Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 5/8] checkout: introduce --{,no-}overlay option
  2018-12-09 20:04 ` [PATCH 5/8] checkout: introduce --{,no-}overlay option Thomas Gummerer
  2018-12-10 16:42   ` Duy Nguyen
@ 2018-12-10 18:19   ` Elijah Newren
  2018-12-11  3:07     ` Junio C Hamano
  1 sibling, 1 reply; 81+ messages in thread
From: Elijah Newren @ 2018-12-10 18:19 UTC (permalink / raw)
  To: Thomas Gummerer
  Cc: Git Mailing List, Junio C Hamano, Nguyễn Thái Ngọc

On Sun, Dec 9, 2018 at 12:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
>
> Currently 'git checkout' is defined as an overlay operation, which
> means that if in 'git checkout <tree-ish> -- [<pathspec>]' we have an
> entry in the index that matches <pathspec>, but that doesn't exist in
> <tree-ish>, that entry will not be removed from the index or the
> working tree.
>
> Introduce a new --{,no-}overlay option, which allows using 'git
> checkout' in non-overlay mode, thus removing files from the working
> tree if they do not exist in <tree-ish> but match <pathspec>.
>
> Note that 'git checkout -p <tree-ish> -- [<pathspec>]' already works
> this way, so no changes are needed for the patch mode.  We disallow
> 'git checkout --overlay -p' to avoid confusing users who would expect
> to be able to force overlay mode in 'git checkout -p' this way.

Whoa...that's interesting.  To me, that argues even further that the
traditional checkout behavior was wrong all along and the choice of
--overlay vs. --no-overlay in the original implementation was a total
oversight.  I'm really tempted to say that --no-overlay should just be
the default in checkout too...but maybe that's too high a hill to
climb, at least for now.

Making --overlap and -p incompatible is a reasonable first step.  But
you should probably add a comment to the -p option documentation that
it implies --no-overlay.

> Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> ---
>  builtin/checkout.c             | 64 +++++++++++++++++++++++++++-------
>  t/t2025-checkout-no-overlay.sh | 47 +++++++++++++++++++++++++
>  t/t9902-completion.sh          |  1 +
>  3 files changed, 99 insertions(+), 13 deletions(-)
>  create mode 100755 t/t2025-checkout-no-overlay.sh
>
> diff --git a/builtin/checkout.c b/builtin/checkout.c
> index acdafc6e4c..0aef35bbc4 100644
> --- a/builtin/checkout.c
> +++ b/builtin/checkout.c
> @@ -44,6 +44,7 @@ struct checkout_opts {
>         int ignore_skipworktree;
>         int ignore_other_worktrees;
>         int show_progress;
> +       int overlay_mode;
>         /*
>          * If new checkout options are added, skip_merge_working_tree
>          * should be updated accordingly.
> @@ -132,7 +133,8 @@ static int skip_same_name(const struct cache_entry *ce, int pos)
>         return pos;
>  }
>
> -static int check_stage(int stage, const struct cache_entry *ce, int pos)
> +static int check_stage(int stage, const struct cache_entry *ce, int pos,
> +                      int overlay_mode)
>  {
>         while (pos < active_nr &&
>                !strcmp(active_cache[pos]->name, ce->name)) {
> @@ -140,6 +142,8 @@ static int check_stage(int stage, const struct cache_entry *ce, int pos)
>                         return 0;
>                 pos++;
>         }
> +       if (!overlay_mode)
> +               return 0;
>         if (stage == 2)
>                 return error(_("path '%s' does not have our version"), ce->name);
>         else
> @@ -165,7 +169,7 @@ static int check_stages(unsigned stages, const struct cache_entry *ce, int pos)
>  }
>
>  static int checkout_stage(int stage, const struct cache_entry *ce, int pos,
> -                         const struct checkout *state)
> +                         const struct checkout *state, int overlay_mode)
>  {
>         while (pos < active_nr &&
>                !strcmp(active_cache[pos]->name, ce->name)) {
> @@ -173,6 +177,10 @@ static int checkout_stage(int stage, const struct cache_entry *ce, int pos,
>                         return checkout_entry(active_cache[pos], state, NULL);
>                 pos++;
>         }
> +       if (!overlay_mode) {
> +               unlink_entry(ce);
> +               return 0;
> +       }
>         if (stage == 2)
>                 return error(_("path '%s' does not have our version"), ce->name);
>         else
> @@ -302,15 +310,29 @@ static int checkout_paths(const struct checkout_opts *opts,
>                 ce->ce_flags &= ~CE_MATCHED;
>                 if (!opts->ignore_skipworktree && ce_skip_worktree(ce))
>                         continue;
> -               if (opts->source_tree && !(ce->ce_flags & CE_UPDATE))
> -                       /*
> -                        * "git checkout tree-ish -- path", but this entry
> -                        * is in the original index; it will not be checked
> -                        * out to the working tree and it does not matter
> -                        * if pathspec matched this entry.  We will not do
> -                        * anything to this entry at all.
> -                        */
> -                       continue;
> +               if (opts->source_tree && !(ce->ce_flags & CE_UPDATE)) {
> +                       if (!opts->overlay_mode &&
> +                           ce_path_match(&the_index, ce, &opts->pathspec, ps_matched)) {
> +                               /*
> +                                * "git checkout --no-overlay <tree-ish> -- path",
> +                                * and the path is not in tree-ish, but is in
> +                                * the current index, which means that it should
> +                                * be removed.
> +                                */
> +                               ce->ce_flags |= CE_MATCHED | CE_REMOVE | CE_WT_REMOVE;
> +                               continue;
> +                       } else {
> +                               /*
> +                                * "git checkout tree-ish -- path", but this
> +                                * entry is in the original index; it will not
> +                                * be checked out to the working tree and it
> +                                * does not matter if pathspec matched this
> +                                * entry.  We will not do anything to this entry
> +                                * at all.
> +                                */
> +                               continue;
> +                       }
> +               }
>                 /*
>                  * Either this entry came from the tree-ish we are
>                  * checking the paths out of, or we are checking out
> @@ -348,7 +370,7 @@ static int checkout_paths(const struct checkout_opts *opts,
>                         if (opts->force) {
>                                 warning(_("path '%s' is unmerged"), ce->name);
>                         } else if (opts->writeout_stage) {
> -                               errs |= check_stage(opts->writeout_stage, ce, pos);
> +                               errs |= check_stage(opts->writeout_stage, ce, pos, opts->overlay_mode);
>                         } else if (opts->merge) {
>                                 errs |= check_stages((1<<2) | (1<<3), ce, pos);
>                         } else {
> @@ -375,12 +397,14 @@ static int checkout_paths(const struct checkout_opts *opts,
>                                 continue;
>                         }
>                         if (opts->writeout_stage)
> -                               errs |= checkout_stage(opts->writeout_stage, ce, pos, &state);
> +                               errs |= checkout_stage(opts->writeout_stage, ce, pos, &state, opts->overlay_mode);
>                         else if (opts->merge)
>                                 errs |= checkout_merged(pos, &state);
>                         pos = skip_same_name(ce, pos) - 1;
>                 }
>         }
> +       remove_marked_cache_entries(&the_index, 1);
> +       remove_scheduled_dirs();
>         errs |= finish_delayed_checkout(&state);
>
>         if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
> @@ -542,6 +566,11 @@ static int skip_merge_working_tree(const struct checkout_opts *opts,
>          * opts->show_progress only impacts output so doesn't require a merge
>          */
>
> +       /*
> +        * opts->overlay_mode cannot be used with switching branches so is
> +        * not tested here
> +        */
> +
>         /*
>          * If we aren't creating a new branch any changes or updates will
>          * happen in the existing branch.  Since that could only be updating
> @@ -1178,6 +1207,10 @@ static int checkout_branch(struct checkout_opts *opts,
>                 die(_("'%s' cannot be used with switching branches"),
>                     "--patch");
>
> +       if (!opts->overlay_mode)
> +               die(_("'%s' cannot be used with switching branches"),
> +                   "--no-overlay");
> +
>         if (opts->writeout_stage)
>                 die(_("'%s' cannot be used with switching branches"),
>                     "--ours/--theirs");
> @@ -1266,6 +1299,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
>                             "checkout", "control recursive updating of submodules",
>                             PARSE_OPT_OPTARG, option_parse_recurse_submodules_worktree_updater },
>                 OPT_BOOL(0, "progress", &opts.show_progress, N_("force progress reporting")),
> +               OPT_BOOL(0, "overlay", &opts.overlay_mode, N_("use overlay mode")),
>                 OPT_END(),
>         };
>
> @@ -1274,6 +1308,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
>         opts.overwrite_ignore = 1;
>         opts.prefix = prefix;
>         opts.show_progress = -1;
> +       opts.overlay_mode = -1;
>
>         git_config(git_checkout_config, &opts);
>
> @@ -1297,6 +1332,9 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
>         if ((!!opts.new_branch + !!opts.new_branch_force + !!opts.new_orphan_branch) > 1)
>                 die(_("-b, -B and --orphan are mutually exclusive"));
>
> +       if (opts.overlay_mode == 1 && opts.patch_mode)
> +               die(_("-p and --overlay are mutually exclusive"));
> +
>         /*
>          * From here on, new_branch will contain the branch to be checked out,
>          * and new_branch_force and new_orphan_branch will tell us which one of
> diff --git a/t/t2025-checkout-no-overlay.sh b/t/t2025-checkout-no-overlay.sh
> new file mode 100755
> index 0000000000..3575321382
> --- /dev/null
> +++ b/t/t2025-checkout-no-overlay.sh
> @@ -0,0 +1,47 @@
> +#!/bin/sh
> +
> +test_description='checkout --no-overlay <tree-ish> -- <pathspec>'
> +
> +. ./test-lib.sh
> +
> +test_expect_success 'setup' '
> +       git commit --allow-empty -m "initial"
> +'
> +
> +test_expect_success 'checkout --no-overlay deletes files not in <tree>' '
> +       >file &&
> +       mkdir dir &&
> +       >dir/file1 &&
> +       git add file dir/file1 &&
> +       git checkout --no-overlay HEAD -- file &&
> +       test_path_is_missing file &&
> +       test_path_is_file dir/file1
> +'
> +
> +test_expect_success 'checkout --no-overlay removing last file from directory' '
> +       git checkout --no-overlay HEAD -- dir/file1 &&
> +       test_path_is_missing dir
> +'
> +
> +test_expect_success 'checkout -p --overlay is disallowed' '
> +       test_must_fail git checkout -p --overlay HEAD 2>actual &&
> +       test_i18ngrep "fatal: -p and --overlay are mutually exclusive" actual
> +'
> +
> +test_expect_success '--no-overlay --theirs with M/D conflict deletes file' '
> +       test_commit file1 file1 &&
> +       test_commit file2 file2 &&
> +       git rm --cached file1 &&
> +       echo 1234 >file1 &&
> +       F1=$(git rev-parse HEAD:file1) &&
> +       F2=$(git rev-parse HEAD:file2) &&
> +       {
> +               echo "100644 $F1 1      file1" &&
> +               echo "100644 $F2 2      file1"
> +       } | git update-index --index-info &&
> +       test_path_is_file file1 &&
> +       git checkout --theirs --no-overlay -- file1 &&
> +       test_path_is_missing file1
> +'
> +
> +test_done
> diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
> index 175f83d704..a3fd9a9630 100755
> --- a/t/t9902-completion.sh
> +++ b/t/t9902-completion.sh
> @@ -1436,6 +1436,7 @@ test_expect_success 'double dash "git checkout"' '
>         --progress Z
>         --no-quiet Z
>         --no-... Z
> +       --overlay Z
>         EOF
>  '
>
> --
> 2.20.0.405.gbc1bbc6f85
>

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries
  2018-12-10 18:19       ` Duy Nguyen
@ 2018-12-10 18:25         ` Elijah Newren
  2018-12-10 18:33           ` Duy Nguyen
  0 siblings, 1 reply; 81+ messages in thread
From: Elijah Newren @ 2018-12-10 18:25 UTC (permalink / raw)
  To: Nguyễn Thái Ngọc
  Cc: Thomas Gummerer, Git Mailing List, Junio C Hamano

On Mon, Dec 10, 2018 at 10:19 AM Duy Nguyen <pclouds@gmail.com> wrote:
>
> On Mon, Dec 10, 2018 at 7:09 PM Elijah Newren <newren@gmail.com> wrote:
> > > > For the two current callsites, unpack-trees seems to do this
> > > > invalidation itself internally.
> > >
> > > I'm still a bit scared of this invalidation business in unpack-trees.
> > > The thing is, we handle two separate index_state there, src_index and
> > > result and invalidation has to be done on the right one (because index
> > > extensions are on src_index until the very end of unpack-trees;
> > > invalidating on 'result' would be no-op and wrong).
> > > remove_marked_cache_entries() seems to be called on 'result' while
> > > invalidate_ce_path() is on src_index, hm....
> >
> > Is Thomas avoiding problems here simply because merge is the only
> > caller of unpack_trees with src_index != dst_index?  Or does src_index
> > == dst_index for checkout not actually help?
>
> I think it would not help. 'result' is a temporary index where we copy
> things to (and it does not have anything from the beginning). If you
> invalidate stuff in there, you invalidate nothing, regardless whether
> dst_index == src_index.
>
> > If that does help with the checkout case, then allow me to find a
> > different way to muddy the waters...  I think I might want to make use
> > of this function in the merge machinery at some point, so I either
> > need to figure out how to convince you to verify if all this cache
> > tree invalidation stuff is sane, or somehow figure out all the
> > cache_tree stuff stuff myself so I can figure out what is right here.
> > :-)
>
> I'm not the unpack-trees man (I think that would still be Junio). And
> I'm not saying it's sane either. I think it's just some leftover
> things since Linus split "the index" in unpack-tree operation to
> 'src', 'result' and 'dst' many years ago and nobody was brave enough
> to clean it up (then I piled on with untracked cache and split index,
> but I did not see it clearly either). That person could be you ;-)

Hmm, might make a good New Year's resolution: Enter the abyss, find
out if one can return from it...  or maybe I could just sanely run
away screaming.  We'll see.

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries
  2018-12-10 18:25         ` Elijah Newren
@ 2018-12-10 18:33           ` Duy Nguyen
  2018-12-10 18:47             ` Elijah Newren
  0 siblings, 1 reply; 81+ messages in thread
From: Duy Nguyen @ 2018-12-10 18:33 UTC (permalink / raw)
  To: Elijah Newren; +Cc: Thomas Gummerer, Git Mailing List, Junio C Hamano

On Mon, Dec 10, 2018 at 7:25 PM Elijah Newren <newren@gmail.com> wrote:
> > I'm not the unpack-trees man (I think that would still be Junio). And
> > I'm not saying it's sane either. I think it's just some leftover
> > things since Linus split "the index" in unpack-tree operation to
> > 'src', 'result' and 'dst' many years ago and nobody was brave enough
> > to clean it up (then I piled on with untracked cache and split index,
> > but I did not see it clearly either). That person could be you ;-)
>
> Hmm, might make a good New Year's resolution: Enter the abyss, find
> out if one can return from it...  or maybe I could just sanely run
> away screaming.  We'll see.

I'm getting off topic. But my new years resolution would be optimize
for the case where src_index == dst_index, which is somewhat ironic
because we used to do everything in the same index, but it was a messy
mess and had to be split up.
-- 
Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 0/8] introduce no-overlay and cached mode in git checkout
  2018-12-10 17:18 ` Elijah Newren
@ 2018-12-10 18:37   ` Duy Nguyen
  2018-12-11 22:52   ` Thomas Gummerer
  1 sibling, 0 replies; 81+ messages in thread
From: Duy Nguyen @ 2018-12-10 18:37 UTC (permalink / raw)
  To: Elijah Newren; +Cc: Thomas Gummerer, Git Mailing List, Junio C Hamano

On Mon, Dec 10, 2018 at 6:18 PM Elijah Newren <newren@gmail.com> wrote:
> > The final step in the series is to actually make use of this in 'git
> > stash', which simplifies the code there a bit.  I am however happy to
> > hold off on this step until the stash-in-C series is merged, so we
> > don't delay that further.
> >
> > In addition to the no-overlay mode, we also add a --cached mode, which
> > works only on the index, thus similar to 'git reset <tree-ish> -- <pathspec>'.
>
> If you're adding a --cached mode to make it only work on the index,
> should there be a similar mode to allow it to only work on the working
> tree?  (I'm not as concerned with that here, but I really think the
> new restore-files command by default should only operate on the
> working tree, and then have options to affect the index either in
> addition or instead of the working tree.)

In the context of restore-files, --target=<worktree|index|both> is a
very good candidate because "restore-files --from=foo --target=index"
is almost like saying "restore files in the index from "foo"". For
checkout, probably not as good. But then we can have different option
names for the two commands. So if "git checkout" is going to never
have "update worktree only" mode, then --cached is a still good way to
go.
-- 
Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 6/8] checkout: add --cached option
  2018-12-09 20:04 ` [PATCH 6/8] checkout: add --cached option Thomas Gummerer
  2018-12-10 16:49   ` Duy Nguyen
@ 2018-12-10 18:42   ` Elijah Newren
  2018-12-11 22:18     ` Thomas Gummerer
  1 sibling, 1 reply; 81+ messages in thread
From: Elijah Newren @ 2018-12-10 18:42 UTC (permalink / raw)
  To: Thomas Gummerer
  Cc: Git Mailing List, Junio C Hamano, Nguyễn Thái Ngọc

On Sun, Dec 9, 2018 at 12:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
>
> Add a new --cached option to git checkout, which works only on the
> index, but not the working tree, similar to what 'git reset <tree-ish>
> -- <pathspec>... does.  Indeed the tests are adapted from the 'git
> reset' tests.
>
> In the longer term the idea is to potentially deprecate 'git reset
> <tree-ish> -- <pathspec>...', so the 'git reset' command becomes only
> about re-pointing the HEAD, and not also about copying entries from
> <tree-ish> to the index.
>
> Note that 'git checkout' by default works in overlay mode, meaning
> files that match the pathspec that don't exist in <tree-ish>, but
> exist in the index would not be removed.  'git checkout --no-overlay
> --cached' can be used to get the same behaviour as 'git reset
> <tree-ish> -- <pathspec>'.

I think this argues _even more_ that --no-overlay should be the
default.  Your series is valuable even if we don't push on that, I'm
just being noisy about what I think would be an even better world.

Also, I don't think I've mentioned it yet, but I'm really excited
about this series and what you're doing.  It's super cool.  (Which I
expected when I saw the description of the desired behavior, but I'm
also liking and contemplating re-using some code...)

> One thing this patch doesn't currently deal with is conflicts.
> Currently 'git checkout --{ours,theirs} -- <file-with-conflicts>'
> doesn't do anything with the index, so the --cached option just
> mirrors that behaviour.  But given it doesn't even deal with
> conflicts, the '--cached' option doesn't make much sense when no
> <tree-ish> is given.  As it operates only on the index, it's always a
> no-op if no tree-ish is given.
>
> Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> ---
>
> Maybe we can just disallow --cached without <tree-ish> given for now,
> and possibly later allow it with some different behaviour for
> conflicts, not sure what the best way forward here is.  We can also
> just make it update the index as appropriate, and have it behave
> different than 'git checkout' curerntly does when handling conflicts?

Huh?
  "git checkout -- <path>"
means update <path> from the index, meaning the index is left alone
(it's the source) and only the working tree is touched.

When you add a flag named --cached to only update the index and not
the working tree, then the index becomes the sole destination.

Now we combine: no tree is specified means the index is the source of
the writing, and --cached being specified means the index is the sole
destination of the writing.  Thus, you have a no-op.  If the user
specifies --cached and no tree, you should immediately exit with a
message along the lines of "Nothing to do; no tree given and --cached
specified."  The presence of conflicts seems completely irrelevant to
me here.

>  builtin/checkout.c         |  26 ++++++++--
>  t/t2016-checkout-patch.sh  |   8 +++
>  t/t2026-checkout-cached.sh | 103 +++++++++++++++++++++++++++++++++++++
>  t/t9902-completion.sh      |   1 +
>  4 files changed, 135 insertions(+), 3 deletions(-)
>  create mode 100755 t/t2026-checkout-cached.sh
>
> diff --git a/builtin/checkout.c b/builtin/checkout.c
> index 0aef35bbc4..6ba85e9de5 100644
> --- a/builtin/checkout.c
> +++ b/builtin/checkout.c
> @@ -45,6 +45,7 @@ struct checkout_opts {
>         int ignore_other_worktrees;
>         int show_progress;
>         int overlay_mode;
> +       int cached;
>         /*
>          * If new checkout options are added, skip_merge_working_tree
>          * should be updated accordingly.
> @@ -288,6 +289,10 @@ static int checkout_paths(const struct checkout_opts *opts,
>                 die(_("Cannot update paths and switch to branch '%s' at the same time."),
>                     opts->new_branch);
>
> +       if (opts->patch_mode && opts->cached)
> +               return run_add_interactive(revision, "--patch=reset",
> +                                          &opts->pathspec);
> +
>         if (opts->patch_mode)
>                 return run_add_interactive(revision, "--patch=checkout",
>                                            &opts->pathspec);
> @@ -319,7 +324,9 @@ static int checkout_paths(const struct checkout_opts *opts,
>                                  * the current index, which means that it should
>                                  * be removed.
>                                  */
> -                               ce->ce_flags |= CE_MATCHED | CE_REMOVE | CE_WT_REMOVE;
> +                               ce->ce_flags |= CE_MATCHED | CE_REMOVE;
> +                               if (!opts->cached)
> +                                       ce->ce_flags |= CE_WT_REMOVE;
>                                 continue;
>                         } else {
>                                 /*
> @@ -392,6 +399,9 @@ static int checkout_paths(const struct checkout_opts *opts,
>         for (pos = 0; pos < active_nr; pos++) {
>                 struct cache_entry *ce = active_cache[pos];
>                 if (ce->ce_flags & CE_MATCHED) {
> +                       if (opts->cached) {
> +                               continue;
> +                       }
>                         if (!ce_stage(ce)) {
>                                 errs |= checkout_entry(ce, &state, NULL);
>                                 continue;
> @@ -571,6 +581,11 @@ static int skip_merge_working_tree(const struct checkout_opts *opts,
>          * not tested here
>          */
>
> +       /*
> +        * opts->cached cannot be used with switching branches so is
> +        * not tested here
> +        */
> +
>         /*
>          * If we aren't creating a new branch any changes or updates will
>          * happen in the existing branch.  Since that could only be updating
> @@ -1207,9 +1222,13 @@ static int checkout_branch(struct checkout_opts *opts,
>                 die(_("'%s' cannot be used with switching branches"),
>                     "--patch");
>
> -       if (!opts->overlay_mode)
> +       if (opts->overlay_mode != -1)
> +               die(_("'%s' cannot be used with switching branches"),
> +                   "--overlay/--no-overlay");
> +
> +       if (opts->cached)
>                 die(_("'%s' cannot be used with switching branches"),
> -                   "--no-overlay");
> +                   "--cached");
>
>         if (opts->writeout_stage)
>                 die(_("'%s' cannot be used with switching branches"),
> @@ -1300,6 +1319,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
>                             PARSE_OPT_OPTARG, option_parse_recurse_submodules_worktree_updater },
>                 OPT_BOOL(0, "progress", &opts.show_progress, N_("force progress reporting")),
>                 OPT_BOOL(0, "overlay", &opts.overlay_mode, N_("use overlay mode")),
> +               OPT_BOOL(0, "cached", &opts.cached, N_("work on the index only")),
>                 OPT_END(),
>         };
>
> diff --git a/t/t2016-checkout-patch.sh b/t/t2016-checkout-patch.sh
> index 47aeb0b167..e8774046e0 100755
> --- a/t/t2016-checkout-patch.sh
> +++ b/t/t2016-checkout-patch.sh
> @@ -108,6 +108,14 @@ test_expect_success PERL 'path limiting works: foo inside dir' '
>         verify_state dir/foo head head
>  '
>
> +test_expect_success PERL 'git checkout --cached -p' '
> +       set_and_save_state dir/foo work work &&
> +       test_write_lines n y | git checkout --cached -p >output &&
> +       verify_state dir/foo work head &&
> +       verify_saved_state bar &&
> +       test_i18ngrep "Unstage" output
> +'
> +
>  test_expect_success PERL 'none of this moved HEAD' '
>         verify_saved_head
>  '
> diff --git a/t/t2026-checkout-cached.sh b/t/t2026-checkout-cached.sh
> new file mode 100755
> index 0000000000..1b66192727
> --- /dev/null
> +++ b/t/t2026-checkout-cached.sh
> @@ -0,0 +1,103 @@
> +#!/bin/sh
> +
> +test_description='checkout --cached <pathspec>'
> +
> +. ./test-lib.sh
> +
> +test_expect_success 'checkout --cached <pathspec>' '
> +       echo 1 >file1 &&
> +       echo 2 >file2 &&
> +       git add file1 file2 &&
> +       test_tick &&
> +       git commit -m files &&
> +       git rm file2 &&
> +       echo 3 >file3 &&
> +       echo 4 >file1 &&
> +       git add file1 file3 &&
> +       git checkout --cached HEAD -- file1 file2 &&
> +       test_must_fail git diff --quiet &&
> +
> +       cat >expect <<-\EOF &&
> +       diff --git a/file1 b/file1
> +       index d00491f..b8626c4 100644
> +       --- a/file1
> +       +++ b/file1
> +       @@ -1 +1 @@
> +       -1
> +       +4
> +       diff --git a/file2 b/file2
> +       deleted file mode 100644
> +       index 0cfbf08..0000000
> +       --- a/file2
> +       +++ /dev/null
> +       @@ -1 +0,0 @@
> +       -2
> +       EOF
> +       git diff >actual &&
> +       test_cmp expect actual &&
> +
> +       cat >expect <<-\EOF &&
> +       diff --git a/file3 b/file3
> +       new file mode 100644
> +       index 0000000..00750ed
> +       --- /dev/null
> +       +++ b/file3
> +       @@ -0,0 +1 @@
> +       +3
> +       EOF
> +       git diff --cached >actual &&
> +       test_cmp expect actual
> +'
> +
> +test_expect_success 'checking out an unmodified path is a no-op' '
> +       git reset --hard &&
> +       git checkout --cached HEAD -- file1 &&
> +       git diff-files --exit-code &&
> +       git diff-index --cached --exit-code HEAD
> +'
> +
> +test_expect_success 'checking out specific path that is unmerged' '
> +       test_commit file3 file3 &&
> +       git rm --cached file2 &&
> +       echo 1234 >file2 &&
> +       F1=$(git rev-parse HEAD:file1) &&
> +       F2=$(git rev-parse HEAD:file2) &&
> +       F3=$(git rev-parse HEAD:file3) &&
> +       {
> +               echo "100644 $F1 1      file2" &&
> +               echo "100644 $F2 2      file2" &&
> +               echo "100644 $F3 3      file2"
> +       } | git update-index --index-info &&
> +       git ls-files -u &&
> +       git checkout --cached HEAD file2 &&
> +       test_must_fail git diff --quiet &&
> +       git diff-index --exit-code --cached HEAD
> +'
> +
> +test_expect_success '--cached without --no-overlay does not remove entry from index' '
> +       test_must_fail git checkout --cached HEAD^ file3 &&
> +       git ls-files --error-unmatch -- file3
> +'
> +
> +test_expect_success 'file is removed from the index with --no-overlay' '
> +       git checkout --cached --no-overlay HEAD^ file3 &&
> +       test_path_is_file file3 &&
> +       test_must_fail git ls-files --error-unmatch -- file3
> +'
> +
> +test_expect_success 'test checkout --cached --no-overlay at given paths' '
> +       mkdir sub &&
> +       >sub/file1 &&
> +       >sub/file2 &&
> +       git update-index --add sub/file1 sub/file2 &&
> +       T=$(git write-tree) &&
> +       git checkout --cached --no-overlay HEAD sub/file2 &&
> +       test_must_fail git diff --quiet &&
> +       U=$(git write-tree) &&

Do we need to worry at all about losing the exit status of write-tree
in either invocation?  In particular, if the second one for U fails
somehow, we'd end up with $U being a blank string and we'd still
probably get "$T" != "$U" below.

You also had some rev-parse invocations hidden in a sub-shell in both
this patch and patch 5, but subsequent commands relied on non-empty
output out of those, so I figured those were fine.  This one might be
too, but I thought I'd at least mention it.

> +       echo "$T" &&
> +       echo "$U" &&
> +       test_must_fail git diff-index --cached --exit-code "$T" &&
> +       test "$T" != "$U"
> +'
> +
> +test_done
> diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
> index a3fd9a9630..cbc304ace8 100755
> --- a/t/t9902-completion.sh
> +++ b/t/t9902-completion.sh
> @@ -1437,6 +1437,7 @@ test_expect_success 'double dash "git checkout"' '
>         --no-quiet Z
>         --no-... Z
>         --overlay Z
> +       --cached Z
>         EOF
>  '
>
> --
> 2.20.0.405.gbc1bbc6f85

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries
  2018-12-10 18:33           ` Duy Nguyen
@ 2018-12-10 18:47             ` Elijah Newren
  0 siblings, 0 replies; 81+ messages in thread
From: Elijah Newren @ 2018-12-10 18:47 UTC (permalink / raw)
  To: Nguyễn Thái Ngọc
  Cc: Thomas Gummerer, Git Mailing List, Junio C Hamano

On Mon, Dec 10, 2018 at 10:34 AM Duy Nguyen <pclouds@gmail.com> wrote:
>
> On Mon, Dec 10, 2018 at 7:25 PM Elijah Newren <newren@gmail.com> wrote:
> > > I'm not the unpack-trees man (I think that would still be Junio). And
> > > I'm not saying it's sane either. I think it's just some leftover
> > > things since Linus split "the index" in unpack-tree operation to
> > > 'src', 'result' and 'dst' many years ago and nobody was brave enough
> > > to clean it up (then I piled on with untracked cache and split index,
> > > but I did not see it clearly either). That person could be you ;-)
> >
> > Hmm, might make a good New Year's resolution: Enter the abyss, find
> > out if one can return from it...  or maybe I could just sanely run
> > away screaming.  We'll see.
>
> I'm getting off topic. But my new years resolution would be optimize
> for the case where src_index == dst_index, which is somewhat ironic
> because we used to do everything in the same index, but it was a messy
> mess and had to be split up.

Ooh, that sounds cool too.  I look forward to seeing it.

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 7/8] checkout: allow ignoring unmatched pathspec
  2018-12-09 20:04 ` [PATCH 7/8] checkout: allow ignoring unmatched pathspec Thomas Gummerer
  2018-12-10 16:51   ` Duy Nguyen
@ 2018-12-10 20:25   ` Elijah Newren
  2018-12-11 22:36     ` Thomas Gummerer
  1 sibling, 1 reply; 81+ messages in thread
From: Elijah Newren @ 2018-12-10 20:25 UTC (permalink / raw)
  To: Thomas Gummerer
  Cc: Git Mailing List, Junio C Hamano, Nguyễn Thái Ngọc

On Sun, Dec 9, 2018 at 12:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
>
> Currently when 'git checkout -- <pathspec>...' is invoked with
> multiple pathspecs, where one or more of the pathspecs don't match
> anything, checkout errors out.
>
> This can be inconvenient in some cases, such as when using git
> checkout from a script.  Introduce a new --ignore-unmatched option,
> which which allows us to ignore a non-matching pathspec instead of
> erroring out.
>
> In a subsequent commit we're going to start using 'git checkout' in
> 'git stash' and are going to make use of this feature.

This makes sense, but seems incomplete.  But to explain it, I think
there's another bug I need to demonstrate first because it's related
on builds on it.  First, the setup:

  $ echo foo >subdir/newfile
  $ git add subdir/newfile
  $ echo bar >>subdir/newfile
  $ git status
  On branch A
  Changes to be committed:
    (use "git reset HEAD <file>..." to unstage)

      new file:   subdir/newfile

  Changes not staged for commit:
    (use "git add <file>..." to update what will be committed)
    (use "git checkout -- <file>..." to discard changes in working directory)

      modified:   subdir/newfile

Now, does it do what we expect?

  $ git checkout HEAD -- subdir/newfile
  error: pathspec 'subdir/newfile' did not match any file(s) known to git

This is the old overlay behavior; kinda lame, but you made no claims
about fixing the default behavior.  What about with your new option?

  $ git checkout --no-overlay HEAD -- subdir
  $ git status
  On branch A
  nothing to commit, working tree clean

Yes, the feature seems to work as advertised.  However, let's try
again with a different variant:

  $ echo foo >subdir/newfile
  $ git checkout --no-overlay HEAD -- subdir
  $ git status
  On branch A
  Untracked files:
    (use "git add <file>..." to include in what will be committed)

      subdir/newfile

Why is the file ignored and left there?  Also:

  $ git checkout --no-overlay HEAD -- subdir/newfile
  error: pathspec 'subdir/newfile' did not match any file(s) known to git

That seems wrong to me.  The point of no-overlay is to make it match
HEAD, and while subdir/newfile doesn't exist in HEAD or the index it
does match in the working tree so the intent is clear. But let's say
that the user did go ahead and specify your new flag:


  $ git checkout --no-overlay --ignore-unmatch HEAD -- subdir/newfile
  $ git status
  On branch A
  Untracked files:
    (use "git add <file>..." to include in what will be committed)

      subdir/newfile

  nothing added to commit but untracked files present (use "git add" to track)

So now it avoids erroring out when the user does more work than
necessary, but it still misses appropriately cleaning up the file.

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 8/8] stash: use git checkout --no-overlay
  2018-12-09 20:04 ` [PATCH 8/8] stash: use git checkout --no-overlay Thomas Gummerer
@ 2018-12-10 20:26   ` Elijah Newren
  0 siblings, 0 replies; 81+ messages in thread
From: Elijah Newren @ 2018-12-10 20:26 UTC (permalink / raw)
  To: Thomas Gummerer
  Cc: Git Mailing List, Junio C Hamano, Nguyễn Thái Ngọc

On Sun, Dec 9, 2018 at 12:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
>
> Now that we have 'git checkout --no-overlay', we can use it in git
> stash, making the codepaths for 'git stash push' with and without
> pathspec more similar, and thus easier to follow.
>
> Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> ---
>
> As mentioned in the cover letter, not sure if we want to apply this
> now.  There are two reasons I did this:
> - Showing the new functionality of git checkout
> - Increased test coverage, as we are running the new code with all git
>   stash tests for free, which helped look at some cases that I was
>   missing initially.
>
>  git-stash.sh | 12 ++++--------
>  1 file changed, 4 insertions(+), 8 deletions(-)
>
> diff --git a/git-stash.sh b/git-stash.sh
> index 94793c1a91..67be04d996 100755
> --- a/git-stash.sh
> +++ b/git-stash.sh
> @@ -314,19 +314,15 @@ push_stash () {
>
>         if test -z "$patch_mode"
>         then
> -               test "$untracked" = "all" && CLEAN_X_OPTION=-x || CLEAN_X_OPTION=
> -               if test -n "$untracked" && test $# = 0
> +               test "$untracked" = "all" && CLEAN_X_OPTION=-X || CLEAN_X_OPTION=
> +               if test -n "$untracked"
>                 then
> -                       git clean --force --quiet -d $CLEAN_X_OPTION
> +                       git clean --force --quiet -d $CLEAN_X_OPTION -- "$@"
>                 fi
>
>                 if test $# != 0
>                 then
> -                       test -z "$untracked" && UPDATE_OPTION="-u" || UPDATE_OPTION=
> -                       test "$untracked" = "all" && FORCE_OPTION="--force" || FORCE_OPTION=
> -                       git add $UPDATE_OPTION $FORCE_OPTION -- "$@"
> -                       git diff-index -p --cached --binary HEAD -- "$@" |
> -                       git apply --index -R
> +                       git checkout --quiet --no-overlay --ignore-unmatched HEAD -- "$@"

Nice.  :-)

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 2/8] entry: factor out unlink_entry function
  2018-12-10 15:49   ` Duy Nguyen
  2018-12-10 17:23     ` Elijah Newren
@ 2018-12-11  2:23     ` Junio C Hamano
  2018-12-20 13:36     ` Thomas Gummerer
  2 siblings, 0 replies; 81+ messages in thread
From: Junio C Hamano @ 2018-12-11  2:23 UTC (permalink / raw)
  To: Duy Nguyen; +Cc: Thomas Gummerer, Git Mailing List, Elijah Newren

Duy Nguyen <pclouds@gmail.com> writes:

> I wonder if we should move the 'removal' variable in symlinks to
> 'struct checkout' to reduce another global variable. But I guess
> that's the problem for another day. It's not the focus of this series.

Before any such move, I think it is important to notice that the
thing is not thread friendly and devise a way to deal with it.

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 3/8] entry: support CE_WT_REMOVE flag in checkout_entry
  2018-12-10 15:58   ` Duy Nguyen
@ 2018-12-11  2:28     ` Junio C Hamano
  2018-12-12  6:16       ` Duy Nguyen
  0 siblings, 1 reply; 81+ messages in thread
From: Junio C Hamano @ 2018-12-11  2:28 UTC (permalink / raw)
  To: Duy Nguyen; +Cc: Thomas Gummerer, Git Mailing List, Elijah Newren

Duy Nguyen <pclouds@gmail.com> writes:

>> +       if (ce->ce_flags & CE_WT_REMOVE) {
>> +               if (topath)
>> +                       BUG("Can't remove entry to a path");
>> +               unlink_entry(ce);
>> +               return 0;
>> +       }
>
> This makes the path counting in nd/checkout-noisy less accurate. But
> it's not your fault of course.

When we check out absense of one path, how do we want to count it?
Do we say "one path checked out?" when we remove one path?

> Junio, do you still want to merge that series down to 'next' or drop
> it? If it will be merged down, I'll keep a note and fix it once this
> one lands too.

Sure, I still agree with you that "git checkout" that reports what
it did when given a "<branch>", but does not report what it did when
given a "<pathspec>", is being inconsistent.  If it makes it easier
to manage, I can kick nd/checkout-noisy out of 'next' to be rebased
on whatever more appropriate when rewinding its tip.

Thanks.


^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries
  2018-12-10 16:08   ` Duy Nguyen
  2018-12-10 18:09     ` Elijah Newren
@ 2018-12-11  2:42     ` Junio C Hamano
  1 sibling, 0 replies; 81+ messages in thread
From: Junio C Hamano @ 2018-12-11  2:42 UTC (permalink / raw)
  To: Duy Nguyen; +Cc: Thomas Gummerer, Git Mailing List, Elijah Newren

Duy Nguyen <pclouds@gmail.com> writes:

> I'm still a bit scared of this invalidation business in unpack-trees.

I too was (and I suspect that I would realize that I still am, if I
take another fresh look at the current code) afraid when I did the
cache-tree work and decided to invalidate it as a whole upfront.

> The thing is, we handle two separate index_state there, src_index and
> result and invalidation has to be done on the right one (because index
> extensions are on src_index until the very end of unpack-trees;
> invalidating on 'result' would be no-op and wrong).
> ...
> Yeah I think it's because cache-tree and untracked cache are already
> properly invalidated. ...


^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 5/8] checkout: introduce --{,no-}overlay option
  2018-12-10 18:19   ` Elijah Newren
@ 2018-12-11  3:07     ` Junio C Hamano
  2018-12-11  6:04       ` Elijah Newren
  2018-12-11 22:07       ` Thomas Gummerer
  0 siblings, 2 replies; 81+ messages in thread
From: Junio C Hamano @ 2018-12-11  3:07 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Thomas Gummerer, Git Mailing List, Nguyễn Thái Ngọc

Elijah Newren <newren@gmail.com> writes:

>> Note that 'git checkout -p <tree-ish> -- [<pathspec>]' already works
>> this way, so no changes are needed for the patch mode.  We disallow
>> 'git checkout --overlay -p' to avoid confusing users who would expect
>> to be able to force overlay mode in 'git checkout -p' this way.
>
> Whoa...that's interesting.  To me, that argues even further that the
> traditional checkout behavior was wrong all along and the choice of
> --overlay vs. --no-overlay in the original implementation was a total
> oversight.  I'm really tempted to say that --no-overlay should just be
> the default in checkout too...but maybe that's too high a hill to
> climb, at least for now.

You are exhibiting a typical reaction to a shiny new toy.

The original "checkout paths out of the trees and/or the index to
recover what was lost" is modeled after "cp -R .../else/where .",
which is an overlay operation, and you wouldn't imagine complaining
that "cp -R" is wrong not to remove things in the target, to be
equivalent to "rm -fr where && cp -R .../else/where .".

Each has its own uses.  We just didn't have the other half of the
pair.

If anything, I would consider "checkout -p" that does not do overlay
is a bug that needs to be fixed.


^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 6/8] checkout: add --cached option
  2018-12-10 16:49   ` Duy Nguyen
@ 2018-12-11  3:13     ` Junio C Hamano
  2018-12-11  6:12       ` Elijah Newren
  0 siblings, 1 reply; 81+ messages in thread
From: Junio C Hamano @ 2018-12-11  3:13 UTC (permalink / raw)
  To: Duy Nguyen; +Cc: Thomas Gummerer, Git Mailing List, Elijah Newren

Duy Nguyen <pclouds@gmail.com> writes:

> Elijah wanted another mode (and I agree) that modifies worktree but
> leaves the index alone. This is most useful (or least confusing) when
> used with <tree-ish> and would be default in restore-files. I'm not
> saying you have to implement it, but how do the new command line
> options are designed to make sense?

I'd model it after "git apply", i.e.

	git restore-files [--tree=<treeish>] <pathspec>

would work only on the working tree files,

	git restore-files --tree=<treeish> --cached <pathspec>

would match the entries in the index that match pathspec to the
given treeish without touching the working tree, and

	git restore-files --tree=<treeish> --index <pathspec>

would be both.

I have never been happy with the phraso, the (arbitrary) distinction
between --cached/--index we use, so in the very longer term (like,
introducing synonym at 3.0 boundary and deprecating older ones at
4.0 boundary), it may not be a bad idea to rename "--index" to
"--index-and-working-tree" and "--cached" to "--index-only".  

But until that happens, it would be better to use these two
consistently.

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 5/8] checkout: introduce --{,no-}overlay option
  2018-12-11  3:07     ` Junio C Hamano
@ 2018-12-11  6:04       ` Elijah Newren
  2018-12-11 22:07       ` Thomas Gummerer
  1 sibling, 0 replies; 81+ messages in thread
From: Elijah Newren @ 2018-12-11  6:04 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Thomas Gummerer, Git Mailing List, Nguyễn Thái Ngọc

On Mon, Dec 10, 2018 at 7:07 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> >> Note that 'git checkout -p <tree-ish> -- [<pathspec>]' already works
> >> this way, so no changes are needed for the patch mode.  We disallow
> >> 'git checkout --overlay -p' to avoid confusing users who would expect
> >> to be able to force overlay mode in 'git checkout -p' this way.
> >
> > Whoa...that's interesting.  To me, that argues even further that the
> > traditional checkout behavior was wrong all along and the choice of
> > --overlay vs. --no-overlay in the original implementation was a total
> > oversight.  I'm really tempted to say that --no-overlay should just be
> > the default in checkout too...but maybe that's too high a hill to
> > climb, at least for now.
>
> You are exhibiting a typical reaction to a shiny new toy.
>
> The original "checkout paths out of the trees and/or the index to
> recover what was lost" is modeled after "cp -R .../else/where .",
> which is an overlay operation, and you wouldn't imagine complaining
> that "cp -R" is wrong not to remove things in the target, to be
> equivalent to "rm -fr where && cp -R .../else/where .".
>
> Each has its own uses.  We just didn't have the other half of the
> pair.

Ah, modelled on cp -R.  I think that rather than reacting "to a shiny
new toy", I just had always had a different mental model AND failed to
figure out what the original model was, leading me to always view it
as buggy.  Thanks for giving me the model I was missing.

> If anything, I would consider "checkout -p" that does not do overlay
> is a bug that needs to be fixed.

Yeah, --no-overlay being the default for -p, and --overlay being the
default otherwise is rather inconsistent.  (Though I'm also fine with
that being fixed by some future patch series.)

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 6/8] checkout: add --cached option
  2018-12-11  3:13     ` Junio C Hamano
@ 2018-12-11  6:12       ` Elijah Newren
  2018-12-11 19:23         ` Duy Nguyen
  0 siblings, 1 reply; 81+ messages in thread
From: Elijah Newren @ 2018-12-11  6:12 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Nguyễn Thái Ngọc, Thomas Gummerer, Git Mailing List

On Mon, Dec 10, 2018 at 7:13 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Duy Nguyen <pclouds@gmail.com> writes:
>
> > Elijah wanted another mode (and I agree) that modifies worktree but
> > leaves the index alone. This is most useful (or least confusing) when
> > used with <tree-ish> and would be default in restore-files. I'm not
> > saying you have to implement it, but how do the new command line
> > options are designed to make sense?
>
> I'd model it after "git apply", i.e.
>
>         git restore-files [--tree=<treeish>] <pathspec>
>
> would work only on the working tree files,
>
>         git restore-files --tree=<treeish> --cached <pathspec>
>
> would match the entries in the index that match pathspec to the
> given treeish without touching the working tree, and
>
>         git restore-files --tree=<treeish> --index <pathspec>
>
> would be both.
>
> I have never been happy with the phraso, the (arbitrary) distinction
> between --cached/--index we use, so in the very longer term (like,
> introducing synonym at 3.0 boundary and deprecating older ones at
> 4.0 boundary), it may not be a bad idea to rename "--index" to
> "--index-and-working-tree" and "--cached" to "--index-only".
>
> But until that happens, it would be better to use these two
> consistently.

I agree that consistency is important, but trying to distinguish
between "--cached" and "--index" is _extremely_ painful.  I still
can't keep the distinction straight and have to look it up every time
I want to use either.  I don't know how to possibly teach users the
meaning.  Could we introduce synonyms earlier at least, and make the
synonyms more prominent than the "--cached" and "--index" terms in the
documentation?

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 6/8] checkout: add --cached option
  2018-12-11  6:12       ` Elijah Newren
@ 2018-12-11 19:23         ` Duy Nguyen
  0 siblings, 0 replies; 81+ messages in thread
From: Duy Nguyen @ 2018-12-11 19:23 UTC (permalink / raw)
  To: Elijah Newren; +Cc: Junio C Hamano, Thomas Gummerer, Git Mailing List

On Tue, Dec 11, 2018 at 7:12 AM Elijah Newren <newren@gmail.com> wrote:
>
> On Mon, Dec 10, 2018 at 7:13 PM Junio C Hamano <gitster@pobox.com> wrote:
> >
> > Duy Nguyen <pclouds@gmail.com> writes:
> >
> > > Elijah wanted another mode (and I agree) that modifies worktree but
> > > leaves the index alone. This is most useful (or least confusing) when
> > > used with <tree-ish> and would be default in restore-files. I'm not
> > > saying you have to implement it, but how do the new command line
> > > options are designed to make sense?
> >
> > I'd model it after "git apply", i.e.
> >
> >         git restore-files [--tree=<treeish>] <pathspec>
> >
> > would work only on the working tree files,
> >
> >         git restore-files --tree=<treeish> --cached <pathspec>
> >
> > would match the entries in the index that match pathspec to the
> > given treeish without touching the working tree, and
> >
> >         git restore-files --tree=<treeish> --index <pathspec>
> >
> > would be both.
> >
> > I have never been happy with the phraso, the (arbitrary) distinction
> > between --cached/--index we use, so in the very longer term (like,
> > introducing synonym at 3.0 boundary and deprecating older ones at
> > 4.0 boundary), it may not be a bad idea to rename "--index" to
> > "--index-and-working-tree" and "--cached" to "--index-only".
> >
> > But until that happens, it would be better to use these two
> > consistently.
>
> I agree that consistency is important, but trying to distinguish
> between "--cached" and "--index" is _extremely_ painful.  I still
> can't keep the distinction straight and have to look it up every time
> I want to use either.  I don't know how to possibly teach users the
> meaning.  Could we introduce synonyms earlier at least, and make the
> synonyms more prominent than the "--cached" and "--index" terms in the
> documentation?

For git-checkout I think --index and --cached fit. For restore-files,
if you come up with better names, I'll gladly use that. Otherwise I'll
just use these.
-- 
Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 1/8] move worktree tests to t24*
  2018-12-10 15:32   ` Duy Nguyen
@ 2018-12-11 21:50     ` Thomas Gummerer
  2018-12-12 13:26       ` Eric Sunshine
  0 siblings, 1 reply; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-11 21:50 UTC (permalink / raw)
  To: Duy Nguyen; +Cc: Git Mailing List, Junio C Hamano, Elijah Newren

On 12/10, Duy Nguyen wrote:
> On Sun, Dec 9, 2018 at 9:04 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> >
> > The 'git worktree' command used to be just another mode in 'git
> > checkout', namely 'git checkout --to'.  When the tests for the latter
> > were retrofitted for the former, the test name was adjusted, but the
> > test number was kept, even though the test is testing a different
> > command now.  t/README states: "Second digit tells the particular
> > command we are testing.", so 'git worktree' should have a separate
> > number just for itself.
> >
> > Move the worktree tests to t24* to adhere to that guideline. We're
> > going to make use of the free'd up numbers in a subsequent commit.
> >
> > Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> > ---
> >  t/{t2025-worktree-add.sh => t2400-worktree-add.sh}     | 0
> >  t/{t2026-worktree-prune.sh => t2401-worktree-prune.sh} | 0
> >  t/{t2027-worktree-list.sh => t2402-worktree-list.sh}   | 0
> >  3 files changed, 0 insertions(+), 0 deletions(-)
> >  rename t/{t2025-worktree-add.sh => t2400-worktree-add.sh} (100%)
> >  rename t/{t2026-worktree-prune.sh => t2401-worktree-prune.sh} (100%)
> >  rename t/{t2027-worktree-list.sh => t2402-worktree-list.sh} (100%)
> 
> Heh.. I did the same thing (in my unsent switch-branch/restore-files
> series) and even used the same 24xx range :D You probably want to move
> t2028 and t2029 too (not sure if they have landed on 'master')

:)  I unfortunately didn't have time to read the
switch-branch/restore-files series in detail, but good to know someone
thought the same way.  I started this work before t2028 and t2029
landed on master, so I failed to notice them.  But I'll rebase on
master and move these two tests as well, thanks for noticing.

> -- 
> Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries
  2018-12-10 18:09     ` Elijah Newren
  2018-12-10 18:19       ` Duy Nguyen
@ 2018-12-11 21:59       ` Thomas Gummerer
  1 sibling, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-11 21:59 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Nguyễn Thái Ngọc, Git Mailing List, Junio C Hamano

On 12/10, Elijah Newren wrote:
> On Mon, Dec 10, 2018 at 8:09 AM Duy Nguyen <pclouds@gmail.com> wrote:
> >
> > On Sun, Dec 9, 2018 at 9:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> > >
> > > When marking cache entries for removal, and later removing them all at
> > > once using 'remove_marked_cache_entries()', cache entries currently
> > > have to be invalidated manually in the cache tree and in the untracked
> > > cache.
> > >
> > > Add an invalidate flag to the function.  With the flag set, the
> > > function will take care of invalidating the path in the cache tree and
> > > in the untracked cache.
> > >
> > > This will be useful in a subsequent commit.
> > >
> > > Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> > > ---
> > >
> > > For the two current callsites, unpack-trees seems to do this
> > > invalidation itself internally.
> >
> > I'm still a bit scared of this invalidation business in unpack-trees.
> > The thing is, we handle two separate index_state there, src_index and
> > result and invalidation has to be done on the right one (because index
> > extensions are on src_index until the very end of unpack-trees;
> > invalidating on 'result' would be no-op and wrong).
> > remove_marked_cache_entries() seems to be called on 'result' while
> > invalidate_ce_path() is on src_index, hm....
> 
> Is Thomas avoiding problems here simply because merge is the only
> caller of unpack_trees with src_index != dst_index?  Or does src_index
> == dst_index for checkout not actually help?

I'm trying to avoid problems in this patch by keeping status quo, and
not changing the cache-tree invalidation in any way.  'git checkout --
<pathspec>' doesn't use unpack-trees, so I don't think I have to worry
about src_index vs. dst_index.

In what I was saying above I was merely trying to explain why we don't
need invalidate the cache-tree in the 'remove_marked_cache_entries()'
function.

> If that does help with the checkout case, then allow me to find a
> different way to muddy the waters...  I think I might want to make use
> of this function in the merge machinery at some point, so I either
> need to figure out how to convince you to verify if all this cache
> tree invalidation stuff is sane, or somehow figure out all the
> cache_tree stuff stuff myself so I can figure out what is right here.
> :-)
> 
> > > I don't quite understand why we don't
> > > need it in split-index mode though.  I assume it's because the cache
> > > tree in the main index would already have been invalidated?  I didn't
> > > have much time to dig, but couldn't produce any failures with it
> > > either, so I assume not invalidating paths is the right thing to do
> > > here.
> >
> > Yeah I think it's because cache-tree and untracked cache are already
> > properly invalidated. This merge base thingy is done when we load the
> > index files up, not when we write them down. The "front" index may
> > record that a few paths in the base index are no longer valid and need
> > to be deleted. But untracked cache and cache-tree both should have
> > recorded that same info when these paths are marked for delete at
> > index write time.

Thanks, that makes sense to me.

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 3/8] entry: support CE_WT_REMOVE flag in checkout_entry
  2018-12-10 17:49   ` Elijah Newren
@ 2018-12-11 22:00     ` Thomas Gummerer
  0 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-11 22:00 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Git Mailing List, Junio C Hamano, Nguyễn Thái Ngọc

On 12/10, Elijah Newren wrote:
> On Sun, Dec 9, 2018 at 12:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> >
> > 'checkout_entry()' currently only supports creating new entries in the
> > working tree, but not deleting them.  Add the ability to remove
> > entries at the same time if the entry is marked with the CE_WT_REMOVE
> > flag.
> >
> > Currently this doesn't have any effect, as the CE_WT_REMOVE flag is
> > only used in unpack-tree, however we will make use of this in a
> > subsequent step in the series.
> >
> > Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> > ---
> >  entry.c | 7 +++++++
> >  1 file changed, 7 insertions(+)
> >
> > diff --git a/entry.c b/entry.c
> > index 3ec148ceee..cd1c6601b6 100644
> > --- a/entry.c
> > +++ b/entry.c
> > @@ -441,6 +441,13 @@ int checkout_entry(struct cache_entry *ce,
> >         static struct strbuf path = STRBUF_INIT;
> >         struct stat st;
> >
> > +       if (ce->ce_flags & CE_WT_REMOVE) {
> > +               if (topath)
> > +                       BUG("Can't remove entry to a path");
> 
> Minor nit: This error message is kinda hard to parse, for someone not
> that familiar with all the *_entry functions, like myself.  Maybe add
> a comment before this line:
>     /* No content and thus no path to create, so we have no pathname
> to return */
> or reword the error slightly?  Or maybe it's fine and I was just
> confused from lack of code familiarity, but I'll throw it out there
> since I stumbled on it a bit.

I'll try to make it more clear in the new round, thanks!

> > +               unlink_entry(ce);
> > +               return 0;
> > +       }
> > +
> >         if (topath)
> >                 return write_entry(ce, topath, state, 1);
> >
> > --
> > 2.20.0.405.gbc1bbc6f85

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 5/8] checkout: introduce --{,no-}overlay option
  2018-12-11  3:07     ` Junio C Hamano
  2018-12-11  6:04       ` Elijah Newren
@ 2018-12-11 22:07       ` Thomas Gummerer
  1 sibling, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-11 22:07 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Elijah Newren, Git Mailing List, Nguyễn Thái Ngọc

On 12/11, Junio C Hamano wrote:
> Elijah Newren <newren@gmail.com> writes:
> 
> >> Note that 'git checkout -p <tree-ish> -- [<pathspec>]' already works
> >> this way, so no changes are needed for the patch mode.  We disallow
> >> 'git checkout --overlay -p' to avoid confusing users who would expect
> >> to be able to force overlay mode in 'git checkout -p' this way.
> >
> > Whoa...that's interesting.  To me, that argues even further that the
> > traditional checkout behavior was wrong all along and the choice of
> > --overlay vs. --no-overlay in the original implementation was a total
> > oversight.  I'm really tempted to say that --no-overlay should just be
> > the default in checkout too...but maybe that's too high a hill to
> > climb, at least for now.
> 
> You are exhibiting a typical reaction to a shiny new toy.

I wonder whether it's worth introducing a config option for this, so
people could use this new mode by default if they wanted to, without
having to type the extra argument?

'git checkout' is a plumbing command, but I see that some of the shell
scripts in git.git are using it.  Though they are only using it in the
checkout branch mode, rather than the checkout paths mode.

> The original "checkout paths out of the trees and/or the index to
> recover what was lost" is modeled after "cp -R .../else/where .",
> which is an overlay operation, and you wouldn't imagine complaining
> that "cp -R" is wrong not to remove things in the target, to be
> equivalent to "rm -fr where && cp -R .../else/where .".
> 
> Each has its own uses.  We just didn't have the other half of the
> pair.
> 
> If anything, I would consider "checkout -p" that does not do overlay
> is a bug that needs to be fixed.
> 

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 6/8] checkout: add --cached option
  2018-12-10 18:42   ` Elijah Newren
@ 2018-12-11 22:18     ` Thomas Gummerer
  0 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-11 22:18 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Git Mailing List, Junio C Hamano, Nguyễn Thái Ngọc

On 12/10, Elijah Newren wrote:
> On Sun, Dec 9, 2018 at 12:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> >
> > Add a new --cached option to git checkout, which works only on the
> > index, but not the working tree, similar to what 'git reset <tree-ish>
> > -- <pathspec>... does.  Indeed the tests are adapted from the 'git
> > reset' tests.
> >
> > In the longer term the idea is to potentially deprecate 'git reset
> > <tree-ish> -- <pathspec>...', so the 'git reset' command becomes only
> > about re-pointing the HEAD, and not also about copying entries from
> > <tree-ish> to the index.
> >
> > Note that 'git checkout' by default works in overlay mode, meaning
> > files that match the pathspec that don't exist in <tree-ish>, but
> > exist in the index would not be removed.  'git checkout --no-overlay
> > --cached' can be used to get the same behaviour as 'git reset
> > <tree-ish> -- <pathspec>'.
> 
> I think this argues _even more_ that --no-overlay should be the
> default.  Your series is valuable even if we don't push on that, I'm
> just being noisy about what I think would be an even better world.

I think just having that mode in 'git restore-files' Duy is working on
may have to be enough for now.

> Also, I don't think I've mentioned it yet, but I'm really excited
> about this series and what you're doing.  It's super cool.  (Which I
> expected when I saw the description of the desired behavior, but I'm
> also liking and contemplating re-using some code...)

Thanks :)

> > One thing this patch doesn't currently deal with is conflicts.
> > Currently 'git checkout --{ours,theirs} -- <file-with-conflicts>'
> > doesn't do anything with the index, so the --cached option just
> > mirrors that behaviour.  But given it doesn't even deal with
> > conflicts, the '--cached' option doesn't make much sense when no
> > <tree-ish> is given.  As it operates only on the index, it's always a
> > no-op if no tree-ish is given.
> >
> > Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> > ---
> >
> > Maybe we can just disallow --cached without <tree-ish> given for now,
> > and possibly later allow it with some different behaviour for
> > conflicts, not sure what the best way forward here is.  We can also
> > just make it update the index as appropriate, and have it behave
> > different than 'git checkout' curerntly does when handling conflicts?
> 
> Huh?
>   "git checkout -- <path>"
> means update <path> from the index, meaning the index is left alone
> (it's the source) and only the working tree is touched.
> 
> When you add a flag named --cached to only update the index and not
> the working tree, then the index becomes the sole destination.
> 
> Now we combine: no tree is specified means the index is the source of
> the writing, and --cached being specified means the index is the sole
> destination of the writing.  Thus, you have a no-op.  If the user
> specifies --cached and no tree, you should immediately exit with a
> message along the lines of "Nothing to do; no tree given and --cached
> specified."  The presence of conflicts seems completely irrelevant to
> me here.

Ah yeah you're right, thanks for a sanity check.  The command I was
most worried about was 'git checkout --cached --{ours,theirs} -- <pathspec>', 
which I thought should update the index.  But as we don't give any
tree-ish, I'm not sure anymore it should.  Maybe just always exiting
with the message you mention above is the right thing to do.

> >  builtin/checkout.c         |  26 ++++++++--
> >  t/t2016-checkout-patch.sh  |   8 +++
> >  t/t2026-checkout-cached.sh | 103 +++++++++++++++++++++++++++++++++++++
> >  t/t9902-completion.sh      |   1 +
> >  4 files changed, 135 insertions(+), 3 deletions(-)
> >  create mode 100755 t/t2026-checkout-cached.sh
> >
> > diff --git a/builtin/checkout.c b/builtin/checkout.c
> > index 0aef35bbc4..6ba85e9de5 100644
> > --- a/builtin/checkout.c
> > +++ b/builtin/checkout.c
> > @@ -45,6 +45,7 @@ struct checkout_opts {
> >         int ignore_other_worktrees;
> >         int show_progress;
> >         int overlay_mode;
> > +       int cached;
> >         /*
> >          * If new checkout options are added, skip_merge_working_tree
> >          * should be updated accordingly.
> > @@ -288,6 +289,10 @@ static int checkout_paths(const struct checkout_opts *opts,
> >                 die(_("Cannot update paths and switch to branch '%s' at the same time."),
> >                     opts->new_branch);
> >
> > +       if (opts->patch_mode && opts->cached)
> > +               return run_add_interactive(revision, "--patch=reset",
> > +                                          &opts->pathspec);
> > +
> >         if (opts->patch_mode)
> >                 return run_add_interactive(revision, "--patch=checkout",
> >                                            &opts->pathspec);
> > @@ -319,7 +324,9 @@ static int checkout_paths(const struct checkout_opts *opts,
> >                                  * the current index, which means that it should
> >                                  * be removed.
> >                                  */
> > -                               ce->ce_flags |= CE_MATCHED | CE_REMOVE | CE_WT_REMOVE;
> > +                               ce->ce_flags |= CE_MATCHED | CE_REMOVE;
> > +                               if (!opts->cached)
> > +                                       ce->ce_flags |= CE_WT_REMOVE;
> >                                 continue;
> >                         } else {
> >                                 /*
> > @@ -392,6 +399,9 @@ static int checkout_paths(const struct checkout_opts *opts,
> >         for (pos = 0; pos < active_nr; pos++) {
> >                 struct cache_entry *ce = active_cache[pos];
> >                 if (ce->ce_flags & CE_MATCHED) {
> > +                       if (opts->cached) {
> > +                               continue;
> > +                       }
> >                         if (!ce_stage(ce)) {
> >                                 errs |= checkout_entry(ce, &state, NULL);
> >                                 continue;
> > @@ -571,6 +581,11 @@ static int skip_merge_working_tree(const struct checkout_opts *opts,
> >          * not tested here
> >          */
> >
> > +       /*
> > +        * opts->cached cannot be used with switching branches so is
> > +        * not tested here
> > +        */
> > +
> >         /*
> >          * If we aren't creating a new branch any changes or updates will
> >          * happen in the existing branch.  Since that could only be updating
> > @@ -1207,9 +1222,13 @@ static int checkout_branch(struct checkout_opts *opts,
> >                 die(_("'%s' cannot be used with switching branches"),
> >                     "--patch");
> >
> > -       if (!opts->overlay_mode)
> > +       if (opts->overlay_mode != -1)
> > +               die(_("'%s' cannot be used with switching branches"),
> > +                   "--overlay/--no-overlay");
> > +
> > +       if (opts->cached)
> >                 die(_("'%s' cannot be used with switching branches"),
> > -                   "--no-overlay");
> > +                   "--cached");
> >
> >         if (opts->writeout_stage)
> >                 die(_("'%s' cannot be used with switching branches"),
> > @@ -1300,6 +1319,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
> >                             PARSE_OPT_OPTARG, option_parse_recurse_submodules_worktree_updater },
> >                 OPT_BOOL(0, "progress", &opts.show_progress, N_("force progress reporting")),
> >                 OPT_BOOL(0, "overlay", &opts.overlay_mode, N_("use overlay mode")),
> > +               OPT_BOOL(0, "cached", &opts.cached, N_("work on the index only")),
> >                 OPT_END(),
> >         };
> >
> > diff --git a/t/t2016-checkout-patch.sh b/t/t2016-checkout-patch.sh
> > index 47aeb0b167..e8774046e0 100755
> > --- a/t/t2016-checkout-patch.sh
> > +++ b/t/t2016-checkout-patch.sh
> > @@ -108,6 +108,14 @@ test_expect_success PERL 'path limiting works: foo inside dir' '
> >         verify_state dir/foo head head
> >  '
> >
> > +test_expect_success PERL 'git checkout --cached -p' '
> > +       set_and_save_state dir/foo work work &&
> > +       test_write_lines n y | git checkout --cached -p >output &&
> > +       verify_state dir/foo work head &&
> > +       verify_saved_state bar &&
> > +       test_i18ngrep "Unstage" output
> > +'
> > +
> >  test_expect_success PERL 'none of this moved HEAD' '
> >         verify_saved_head
> >  '
> > diff --git a/t/t2026-checkout-cached.sh b/t/t2026-checkout-cached.sh
> > new file mode 100755
> > index 0000000000..1b66192727
> > --- /dev/null
> > +++ b/t/t2026-checkout-cached.sh
> > @@ -0,0 +1,103 @@
> > +#!/bin/sh
> > +
> > +test_description='checkout --cached <pathspec>'
> > +
> > +. ./test-lib.sh
> > +
> > +test_expect_success 'checkout --cached <pathspec>' '
> > +       echo 1 >file1 &&
> > +       echo 2 >file2 &&
> > +       git add file1 file2 &&
> > +       test_tick &&
> > +       git commit -m files &&
> > +       git rm file2 &&
> > +       echo 3 >file3 &&
> > +       echo 4 >file1 &&
> > +       git add file1 file3 &&
> > +       git checkout --cached HEAD -- file1 file2 &&
> > +       test_must_fail git diff --quiet &&
> > +
> > +       cat >expect <<-\EOF &&
> > +       diff --git a/file1 b/file1
> > +       index d00491f..b8626c4 100644
> > +       --- a/file1
> > +       +++ b/file1
> > +       @@ -1 +1 @@
> > +       -1
> > +       +4
> > +       diff --git a/file2 b/file2
> > +       deleted file mode 100644
> > +       index 0cfbf08..0000000
> > +       --- a/file2
> > +       +++ /dev/null
> > +       @@ -1 +0,0 @@
> > +       -2
> > +       EOF
> > +       git diff >actual &&
> > +       test_cmp expect actual &&
> > +
> > +       cat >expect <<-\EOF &&
> > +       diff --git a/file3 b/file3
> > +       new file mode 100644
> > +       index 0000000..00750ed
> > +       --- /dev/null
> > +       +++ b/file3
> > +       @@ -0,0 +1 @@
> > +       +3
> > +       EOF
> > +       git diff --cached >actual &&
> > +       test_cmp expect actual
> > +'
> > +
> > +test_expect_success 'checking out an unmodified path is a no-op' '
> > +       git reset --hard &&
> > +       git checkout --cached HEAD -- file1 &&
> > +       git diff-files --exit-code &&
> > +       git diff-index --cached --exit-code HEAD
> > +'
> > +
> > +test_expect_success 'checking out specific path that is unmerged' '
> > +       test_commit file3 file3 &&
> > +       git rm --cached file2 &&
> > +       echo 1234 >file2 &&
> > +       F1=$(git rev-parse HEAD:file1) &&
> > +       F2=$(git rev-parse HEAD:file2) &&
> > +       F3=$(git rev-parse HEAD:file3) &&
> > +       {
> > +               echo "100644 $F1 1      file2" &&
> > +               echo "100644 $F2 2      file2" &&
> > +               echo "100644 $F3 3      file2"
> > +       } | git update-index --index-info &&
> > +       git ls-files -u &&
> > +       git checkout --cached HEAD file2 &&
> > +       test_must_fail git diff --quiet &&
> > +       git diff-index --exit-code --cached HEAD
> > +'
> > +
> > +test_expect_success '--cached without --no-overlay does not remove entry from index' '
> > +       test_must_fail git checkout --cached HEAD^ file3 &&
> > +       git ls-files --error-unmatch -- file3
> > +'
> > +
> > +test_expect_success 'file is removed from the index with --no-overlay' '
> > +       git checkout --cached --no-overlay HEAD^ file3 &&
> > +       test_path_is_file file3 &&
> > +       test_must_fail git ls-files --error-unmatch -- file3
> > +'
> > +
> > +test_expect_success 'test checkout --cached --no-overlay at given paths' '
> > +       mkdir sub &&
> > +       >sub/file1 &&
> > +       >sub/file2 &&
> > +       git update-index --add sub/file1 sub/file2 &&
> > +       T=$(git write-tree) &&
> > +       git checkout --cached --no-overlay HEAD sub/file2 &&
> > +       test_must_fail git diff --quiet &&
> > +       U=$(git write-tree) &&
> 
> Do we need to worry at all about losing the exit status of write-tree
> in either invocation?  In particular, if the second one for U fails
> somehow, we'd end up with $U being a blank string and we'd still
> probably get "$T" != "$U" below.

Hmm this seems to be a fairly common pattern in our test suite:

$ git grep -F '$(git write-tree)' t/* | wc -l
112

But maybe it's just something we used to do, but should move away
from.  Just writing the output to a file shouldn't be much harder
either, I'll do that in the next iteration.

> You also had some rev-parse invocations hidden in a sub-shell in both
> this patch and patch 5, but subsequent commands relied on non-empty
> output out of those, so I figured those were fine.  This one might be
> too, but I thought I'd at least mention it.
> 
> > +       echo "$T" &&
> > +       echo "$U" &&
> > +       test_must_fail git diff-index --cached --exit-code "$T" &&
> > +       test "$T" != "$U"
> > +'
> > +
> > +test_done
> > diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
> > index a3fd9a9630..cbc304ace8 100755
> > --- a/t/t9902-completion.sh
> > +++ b/t/t9902-completion.sh
> > @@ -1437,6 +1437,7 @@ test_expect_success 'double dash "git checkout"' '
> >         --no-quiet Z
> >         --no-... Z
> >         --overlay Z
> > +       --cached Z
> >         EOF
> >  '
> >
> > --
> > 2.20.0.405.gbc1bbc6f85

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 7/8] checkout: allow ignoring unmatched pathspec
  2018-12-10 16:51   ` Duy Nguyen
@ 2018-12-11 22:23     ` Thomas Gummerer
  0 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-11 22:23 UTC (permalink / raw)
  To: Duy Nguyen; +Cc: Git Mailing List, Junio C Hamano, Elijah Newren

On 12/10, Duy Nguyen wrote:
> On Sun, Dec 9, 2018 at 9:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> >
> > Currently when 'git checkout -- <pathspec>...' is invoked with
> > multiple pathspecs, where one or more of the pathspecs don't match
> > anything, checkout errors out.
> >
> > This can be inconvenient in some cases, such as when using git
> > checkout from a script.
> 
> Wait, should scripts go with read-tree, checkout-index or other
> plumbing commands instead?

Possibly.  As mentioned in an other email, we do seem to have some
scripts in git.git that are using 'git checkout' already, but they are
using it in the checkout branch mode, rather than the checkout paths
mode that I would like to use it in git-stash.

But with the rewrite of 'git stash' in C, maybe this step is moot
anyway, and we can just call the checkout_paths function internally
without using the run_command API at all.  We could then have an
internal mode for ignoring unmatched pathspecs that we wouldn't need
to expose to users.

> -- 
> Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 7/8] checkout: allow ignoring unmatched pathspec
  2018-12-10 20:25   ` Elijah Newren
@ 2018-12-11 22:36     ` Thomas Gummerer
  0 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-11 22:36 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Git Mailing List, Junio C Hamano, Nguyễn Thái Ngọc

On 12/10, Elijah Newren wrote:
> On Sun, Dec 9, 2018 at 12:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> >
> > Currently when 'git checkout -- <pathspec>...' is invoked with
> > multiple pathspecs, where one or more of the pathspecs don't match
> > anything, checkout errors out.
> >
> > This can be inconvenient in some cases, such as when using git
> > checkout from a script.  Introduce a new --ignore-unmatched option,
> > which which allows us to ignore a non-matching pathspec instead of
> > erroring out.
> >
> > In a subsequent commit we're going to start using 'git checkout' in
> > 'git stash' and are going to make use of this feature.
> 
> This makes sense, but seems incomplete.  But to explain it, I think
> there's another bug I need to demonstrate first because it's related
> on builds on it.  First, the setup:
> 
>   $ echo foo >subdir/newfile
>   $ git add subdir/newfile
>   $ echo bar >>subdir/newfile
>   $ git status
>   On branch A
>   Changes to be committed:
>     (use "git reset HEAD <file>..." to unstage)
> 
>       new file:   subdir/newfile
> 
>   Changes not staged for commit:
>     (use "git add <file>..." to update what will be committed)
>     (use "git checkout -- <file>..." to discard changes in working directory)
> 
>       modified:   subdir/newfile
> 
> Now, does it do what we expect?
> 
>   $ git checkout HEAD -- subdir/newfile
>   error: pathspec 'subdir/newfile' did not match any file(s) known to git
> 
> This is the old overlay behavior; kinda lame, but you made no claims
> about fixing the default behavior.  What about with your new option?
> 
>   $ git checkout --no-overlay HEAD -- subdir
>   $ git status
>   On branch A
>   nothing to commit, working tree clean
> 
> Yes, the feature seems to work as advertised.  However, let's try
> again with a different variant:
> 
>   $ echo foo >subdir/newfile
>   $ git checkout --no-overlay HEAD -- subdir
>   $ git status
>   On branch A
>   Untracked files:
>     (use "git add <file>..." to include in what will be committed)
> 
>       subdir/newfile
> 
> Why is the file ignored and left there?  Also:
> 
>   $ git checkout --no-overlay HEAD -- subdir/newfile
>   error: pathspec 'subdir/newfile' did not match any file(s) known to git
> 
> That seems wrong to me.

Ah interesting, this is a case I didn't consider.  I'm a bit torn on
this one.  My intention for the no overlay mode was that it would work
similar to what I'd expect 'git reset --hard -- <pathspec>' to work if
it existed, which means not removing untracked files if they exist.

While I think in the example you have above removing subdir/newfile
may be the right behaviour I'm not so sure in the case of 'git
checkout --no-overlay HEAD -- .' or ''git checkout --no-overlay HEAD
-- t/*' for example.  I don't think that should remove all untracked
files in the repository or in the t/ directory.  Removing untracked
files in that case would probably surprise users more than your case
above would.

I think it's okay to keep considering untracked files as special with
respect to how they are treated by 'git checkout --no-overlay'.

>                          The point of no-overlay is to make it match
> HEAD, and while subdir/newfile doesn't exist in HEAD or the index it
> does match in the working tree so the intent is clear. But let's say
> that the user did go ahead and specify your new flag:
> 
> 
>   $ git checkout --no-overlay --ignore-unmatch HEAD -- subdir/newfile
>   $ git status
>   On branch A
>   Untracked files:
>     (use "git add <file>..." to include in what will be committed)
> 
>       subdir/newfile
> 
>   nothing added to commit but untracked files present (use "git add" to track)
> 
> So now it avoids erroring out when the user does more work than
> necessary, but it still misses appropriately cleaning up the file.

Yeah this is a good point, this could be more confusing to the user
than the previous case in my opinion.  Maybe I'll just drop this patch
for now (and the next one, as it's better to hold of until stash in C
lands anyway), and then try to do all this in-core for 'git stash'.

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 5/8] checkout: introduce --{,no-}overlay option
  2018-12-10 16:42   ` Duy Nguyen
@ 2018-12-11 22:42     ` Thomas Gummerer
  0 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-11 22:42 UTC (permalink / raw)
  To: Duy Nguyen; +Cc: Git Mailing List, Junio C Hamano, Elijah Newren

On 12/10, Duy Nguyen wrote:
> On Sun, Dec 9, 2018 at 9:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> > @@ -302,15 +310,29 @@ static int checkout_paths(const struct checkout_opts *opts,
> >                 ce->ce_flags &= ~CE_MATCHED;
> >                 if (!opts->ignore_skipworktree && ce_skip_worktree(ce))
> >                         continue;
> > -               if (opts->source_tree && !(ce->ce_flags & CE_UPDATE))
> > -                       /*
> > -                        * "git checkout tree-ish -- path", but this entry
> > -                        * is in the original index; it will not be checked
> > -                        * out to the working tree and it does not matter
> > -                        * if pathspec matched this entry.  We will not do
> > -                        * anything to this entry at all.
> > -                        */
> > -                       continue;
> > +               if (opts->source_tree && !(ce->ce_flags & CE_UPDATE)) {
> > +                       if (!opts->overlay_mode &&
> > +                           ce_path_match(&the_index, ce, &opts->pathspec, ps_matched)) {
> > +                               /*
> > +                                * "git checkout --no-overlay <tree-ish> -- path",
> > +                                * and the path is not in tree-ish, but is in
> > +                                * the current index, which means that it should
> > +                                * be removed.
> > +                                */
> > +                               ce->ce_flags |= CE_MATCHED | CE_REMOVE | CE_WT_REMOVE;
> > +                               continue;
> > +                       } else {
> 
> In non-overlay mode but when pathspec does not match, we come here too.
> 
> > +                               /*
> > +                                * "git checkout tree-ish -- path", but this
> > +                                * entry is in the original index; it will not
> 
> I think the missing key point in this comment block is "..is in the
> original index _and it's not in tree-ish_". In non-overlay mode, if
> pathspec does not match then it's safe to ignore too. But this logic
> starts too get to complex and hurt my brain.

Yes, that would make it a bit easier to read. I took a while to try
and refactor this to make it easier to read, but couldn't come up with
anything much better unfortunately.  I'll have another stab at
simplifying the logic a bit for v2.

> > +                                * be checked out to the working tree and it
> > +                                * does not matter if pathspec matched this
> > +                                * entry.  We will not do anything to this entry
> > +                                * at all.
> > +                                */
> > +                               continue;
> > +                       }
> > +               }
> >                 /*
> >                  * Either this entry came from the tree-ish we are
> >                  * checking the paths out of, or we are checking out
> 
> > @@ -1266,6 +1299,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
> >                             "checkout", "control recursive updating of submodules",
> >                             PARSE_OPT_OPTARG, option_parse_recurse_submodules_worktree_updater },
> >                 OPT_BOOL(0, "progress", &opts.show_progress, N_("force progress reporting")),
> > +               OPT_BOOL(0, "overlay", &opts.overlay_mode, N_("use overlay mode")),
> 
> maybe add " (default)" to the help string.

Makes sense, will add.

> >                 OPT_END(),
> >         };
> >
> -- 
> Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 0/8] introduce no-overlay and cached mode in git checkout
  2018-12-10 17:18 ` Elijah Newren
  2018-12-10 18:37   ` Duy Nguyen
@ 2018-12-11 22:52   ` Thomas Gummerer
  2018-12-12  7:28     ` Junio C Hamano
  1 sibling, 1 reply; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-11 22:52 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Git Mailing List, Junio C Hamano, Nguyễn Thái Ngọc

On 12/10, Elijah Newren wrote:
> On Sun, Dec 9, 2018 at 12:04 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> >
> > Here's the series I mentioned a couple of times on the list already,
> > introducing a no-overlay mode in 'git checkout'.  The inspiration for
> > this came from Junios message in [*1*].
> >
> > Basically the idea is to also delete files when the match <pathspec>
> > in 'git checkout <tree-ish> -- <pathspec>' in the current tree, but
> > don't match <pathspec> in <tree-ish>.  The rest of the cases are
> > already properly taken care of by 'git checkout'.
> 
> Yes, but I'd put it a little differently:
> 
> """
> Basically, the idea is when the user run "git checkout --no-overlay
> <tree-ish> -- <pathspec>" that the given pathspecs should exactly
> match <tree-ish> after the operation completes.  This means that we
> also want to delete files that match <pathspec> if those paths are not
> found in <tree-ish>.
> """
> 
> ...and maybe even toss in some comments about the fact that this is
> the way git checkout should have always behaved, it just traditionally
> hasn't.  (You could also work in comments about how with this new mode
> the user can run git diff afterward with the given commit-ish and
> pathspecs and get back an empty diff, as expected, which wasn't true
> before.  But maybe I'm belaboring the point.)
> 
> 
> > The final step in the series is to actually make use of this in 'git
> > stash', which simplifies the code there a bit.  I am however happy to
> > hold off on this step until the stash-in-C series is merged, so we
> > don't delay that further.
> >
> > In addition to the no-overlay mode, we also add a --cached mode, which
> > works only on the index, thus similar to 'git reset <tree-ish> -- <pathspec>'.
> 
> If you're adding a --cached mode to make it only work on the index,
> should there be a similar mode to allow it to only work on the working
> tree?  (I'm not as concerned with that here, but I really think the
> new restore-files command by default should only operate on the
> working tree, and then have options to affect the index either in
> addition or instead of the working tree.)

Yeah I think that would be nice to have, though I'm not sure what we
would name it in 'git checkout'.  Maybe just having it in 'git
restore-files' is good enough?

> > Actually deprecating 'git reset <tree-ish> -- <pathspec>' should come
> > later, probably not before Duy's restore-files command lands, as 'git
> > checkout --no-overlay <tree-ish> -- <pathspec>' is a bit cumbersome to
> > type compared to 'git reset <tree-ish> -- <pathspec>'.
> 
> Makes sense.
> 
> > My hope is also that the no-overlay mode could become the new default
> > in the restore-files command Duy is currently working on.
> 
> Absolutely, yes.  I don't want another broken command.  :-)
> 
> 
> > No documentation yet, as I wanted to get this out for review first.
> > I'm not familiar with most of the code I touched here, so there may
> > well be much better ways to implement some of this, that I wasn't able
> > to figure out.  I'd be very happy with some feedback around that.
> >
> > Another thing I'm not sure about is how to deal with conflicts.  In
> > the cached mode this patch series is not dealing with it at all, as
> > 'git checkout -- <pathspec>' when pathspec matches a file with
> > conflicts doesn't update the index.  For the no-overlay mode, the file
> > is removed if the corresponding stage is not found in the index.  I'm
> > however not sure this is the right thing to do in all cases?
> 
> Here's how I'd go about analyzing that...
> 
> If the user passes a <tree-ish>, then the answer about what to do is
> pretty obvious; the <tree-ish> didn't have conflicts, so conflicted
> paths in the index that match the pathspec should be overwritten with
> whatever version of those paths existed in <tree-ish> (possibly
> implying deletion of some paths).
> 
> Also, as you point out, --cached means only modify the index and not
> the working tree; so if they specify both --cached and provide no
> tree, then they've specified a no-op.
> 
> So it's only interesting when you have conflicts in the index and
> specify --no-overlay without a <tree-ish> or --cached.  This boils
> down to "how do we update the working tree to match the index, when
> the index is conflicted?"  A couple points to consider:
>   * This is somewhat of an edge case
>   * In the normal case --no-overlay is only different from --overlay
> behavior for directories; it'd be nice if that extended to all cases

I'm not sure I follow what you mean here.  How is --no-overlay
different from --overlay with respect to directories?  It's only
different with respect to deletions, no?

>   * How does this command behave without a <tree-ish> when
> --no-overlay is specified and a directory is given for a <pathspec>
> and there aren't any conflicts?  Are we being consistent with that
> behavior?
> 
> 
> However, I think it turns out that the answer is much simpler than all
> that initial analysis or what you say you've implemented.  Here's why:
> 
> If <pathspec> is a file which is present in both the working tree and
> the index and it has conflicts, then "git checkout -- <pathspec>" will
> currently throw an error:

I think what was missing from my original description, that actually
makes it slightly more interesting from what you describe below is the
'--ours' and '--theirs' flags in 'git checkout', with which one can
check out a version of the file in the working tree.  This is where it
gets more interesting.

I think I got the right solution for that in patch 5, with deleting
the file if it's deleted in "their" version and we pass --theirs to
'git checkout', and analogous for --ours.  I was just wondering if
there were any further edge cases that I can't think of right no.

> $ git checkout -- subdir/counting
> error: path 'subdir/counting' is unmerged
> 
> In fact, even if every entry in subdir/ is a path that is in both the
> index and the working tree (so that --no-overlay and --overlay ought
> to behave the same), if any one of the files in subdir is conflicted,
> attempting to checkout the subdir will abort with this same error
> message and no paths will be updated at all:
> 
> $ git checkout -- subdir
> error: path 'subdir/counting' is unmerged
> 
> as such, the answer with what to do with --no-overlay mode is pretty
> clear: if the <pathspec> matches _any_ path that is conflicted, simply
> throw an error and abort the operation without making any changes at
> all.

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 3/8] entry: support CE_WT_REMOVE flag in checkout_entry
  2018-12-11  2:28     ` Junio C Hamano
@ 2018-12-12  6:16       ` Duy Nguyen
  2018-12-12  7:36         ` Junio C Hamano
  0 siblings, 1 reply; 81+ messages in thread
From: Duy Nguyen @ 2018-12-12  6:16 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Thomas Gummerer, Git Mailing List, Elijah Newren

On Tue, Dec 11, 2018 at 3:28 AM Junio C Hamano <gitster@pobox.com> wrote:
>
> Duy Nguyen <pclouds@gmail.com> writes:
>
> >> +       if (ce->ce_flags & CE_WT_REMOVE) {
> >> +               if (topath)
> >> +                       BUG("Can't remove entry to a path");
> >> +               unlink_entry(ce);
> >> +               return 0;
> >> +       }
> >
> > This makes the path counting in nd/checkout-noisy less accurate. But
> > it's not your fault of course.
>
> When we check out absense of one path, how do we want to count it?
> Do we say "one path checked out?" when we remove one path?

It is still "checked out" according to this non-overlay concept.
Although we could make it clear by saying "5 paths updated, 2 deleted"
(but that may make us say "3 paths added" as well, hmm). Or maybe just
"%d paths updated" where updates include file creation and deletion.
-- 
Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 0/8] introduce no-overlay and cached mode in git checkout
  2018-12-11 22:52   ` Thomas Gummerer
@ 2018-12-12  7:28     ` Junio C Hamano
  0 siblings, 0 replies; 81+ messages in thread
From: Junio C Hamano @ 2018-12-12  7:28 UTC (permalink / raw)
  To: Thomas Gummerer
  Cc: Elijah Newren, Git Mailing List, Nguyễn Thái Ngọc

Thomas Gummerer <t.gummerer@gmail.com> writes:

> I think I got the right solution for that in patch 5, with deleting
> the file if it's deleted in "their" version and we pass --theirs to
> 'git checkout', and analogous for --ours.  I was just wondering if
> there were any further edge cases that I can't think of right no.

That sounds quite sensible.

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 3/8] entry: support CE_WT_REMOVE flag in checkout_entry
  2018-12-12  6:16       ` Duy Nguyen
@ 2018-12-12  7:36         ` Junio C Hamano
  0 siblings, 0 replies; 81+ messages in thread
From: Junio C Hamano @ 2018-12-12  7:36 UTC (permalink / raw)
  To: Duy Nguyen; +Cc: Thomas Gummerer, Git Mailing List, Elijah Newren

Duy Nguyen <pclouds@gmail.com> writes:

> Although we could make it clear by saying "5 paths updated, 2 deleted"
> (but that may make us say "3 paths added" as well, hmm). Or maybe just
> "%d paths updated" where updates include file creation and deletion.

Yeah, the last one is the simplest and good.

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 1/8] move worktree tests to t24*
  2018-12-11 21:50     ` Thomas Gummerer
@ 2018-12-12 13:26       ` Eric Sunshine
  2018-12-12 17:07         ` Duy Nguyen
  0 siblings, 1 reply; 81+ messages in thread
From: Eric Sunshine @ 2018-12-12 13:26 UTC (permalink / raw)
  To: Thomas Gummerer
  Cc: Nguyễn Thái Ngọc Duy, Git List, Junio C Hamano,
	Elijah Newren

On Tue, Dec 11, 2018 at 4:50 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> On 12/10, Duy Nguyen wrote:
> > On Sun, Dec 9, 2018 at 9:04 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> > > Move the worktree tests to t24* to adhere to that guideline. We're
> > > going to make use of the free'd up numbers in a subsequent commit.
> >
> > Heh.. I did the same thing (in my unsent switch-branch/restore-files
> > series) and even used the same 24xx range :D You probably want to move
> > t2028 and t2029 too (not sure if they have landed on 'master')
>
> [...] good to know someone
> thought the same way.  I started this work before t2028 and t2029
> landed on master, so I failed to notice them.

The thought of renumbering the test script came up as early as
2015-06-30. See the last bullet point of [1], for instance.

[1]: https://public-inbox.org/git/1435640202-95945-1-git-send-email-sunshine@sunshineco.com/

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 1/8] move worktree tests to t24*
  2018-12-12 13:26       ` Eric Sunshine
@ 2018-12-12 17:07         ` Duy Nguyen
  0 siblings, 0 replies; 81+ messages in thread
From: Duy Nguyen @ 2018-12-12 17:07 UTC (permalink / raw)
  To: Eric Sunshine
  Cc: Thomas Gummerer, Git Mailing List, Junio C Hamano, Elijah Newren

On Wed, Dec 12, 2018 at 2:27 PM Eric Sunshine <sunshine@sunshineco.com> wrote:
>
> On Tue, Dec 11, 2018 at 4:50 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> > On 12/10, Duy Nguyen wrote:
> > > On Sun, Dec 9, 2018 at 9:04 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> > > > Move the worktree tests to t24* to adhere to that guideline. We're
> > > > going to make use of the free'd up numbers in a subsequent commit.
> > >
> > > Heh.. I did the same thing (in my unsent switch-branch/restore-files
> > > series) and even used the same 24xx range :D You probably want to move
> > > t2028 and t2029 too (not sure if they have landed on 'master')
> >
> > [...] good to know someone
> > thought the same way.  I started this work before t2028 and t2029
> > landed on master, so I failed to notice them.
>
> The thought of renumbering the test script came up as early as
> 2015-06-30. See the last bullet point of [1], for instance.
>
> [1]: https://public-inbox.org/git/1435640202-95945-1-git-send-email-sunshine@sunshineco.com/

Ah good. I thought I was just being lazy and picked a random range to add tests.
-- 
Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 0/8] introduce no-overlay and cached mode in git checkout
  2018-12-10  3:47 ` [PATCH 0/8] introduce no-overlay and cached mode in git checkout Junio C Hamano
@ 2018-12-20  8:43   ` Thomas Gummerer
  0 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-20  8:43 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Nguyễn Thái Ngọc Duy, Elijah Newren

On 12/10, Junio C Hamano wrote:
> Thomas Gummerer <t.gummerer@gmail.com> writes:
> 
> > Basically the idea is to also delete files when the match <pathspec>
> > in 'git checkout <tree-ish> -- <pathspec>' in the current tree, but
> > don't match <pathspec> in <tree-ish>.
> 
> I cannot quite parse it, but perhaps.
> 
> 	"git checkout --no-overlay <tree-ish> -- <pathspec>" can
> 	remove paths in the index and in the working tree that match
> 	<pathspec>, if they do not appear in <tree-ish>.
> 
> If a new file D/F is in the index and in the working tree but not in
> HEAD, "git checkout HEAD D/" or "git checkout HEAD D/F" would not
> remove D/F from the index or the working tree.
> 
> With the --no-overlay option, it would, and that is often closer to
> the wish of the user who wanted to say "restore the working tree
> state of D/ (or D/F) from the state recorded in HEAD".

Yeah thanks, that reads much better.

> > The final step in the series is to actually make use of this in 'git
> > stash', which simplifies the code there a bit.  I am however happy to
> > hold off on this step until the stash-in-C series is merged, so we
> > don't delay that further.
> 
> I think that is probably a good idea, for now.
> 
> > In addition to the no-overlay mode, we also add a --cached mode, which
> > works only on the index, thus similar to 'git reset <tree-ish> -- <pathspec>'.
> >
> > Actually deprecating 'git reset <tree-ish> -- <pathspec>' should come
> > later,...
> 
> Or we may not even need to deprecate it.  IIRC, what "stash" wished
> to exist was "git reset --hard <tree-ish> -- <pathspec>", which, if
> the command followed "--cached/--index" convention, would have been
> called "git reset --index ...".  Did we actually have the need for
> "--cached" mode?

I don't have a pressing need for "--cached".  I mainly included it
because you described "git reset <tree-ish> -- <pathspec>" as bad UI
in the original thread [*1*], which after reading that message I agree
with.  It also seemed to cause some confusion in [*2*].  Since it was
fairly easy to introduce a "--cached" mode it seemed like a potential
UI improvement in the long term to deprecate 'git reset <tree-ish> --
<pathspec>'.

That said, this series is probably not the right place to introduce
this feature, as it should mainly be focused on the no-overlay mode.
I'll drop the patch from v2.

We can revisit whether we want to introduce a "--cached" mode in 'git
checkout' at some point in the future.  

> > probably not before Duy's restore-files command lands, as 'git
> > checkout --no-overlay <tree-ish> -- <pathspec>' is a bit cumbersome to
> > type compared to 'git reset <tree-ish> -- <pathspec>'.
> 
> Yes, between "checkout --cached" and "checkout --no-overlay", the
> latter is much more important, as the latter is what a missing "git
> reset --hard <tree-ish> -- <pathspec>" would have been, but the
> former can be written with an existing command.
> 
> > My hope is also that the no-overlay mode could become the new default
> > in the restore-files command Duy is currently working on.
> 
> Yup, that is my hope, too ;-).

*1*: <xmqq4loqplou.fsf@gitster.mtv.corp.google.com>
*2*: <alpine.LFD.2.21.1812081103500.29142@localhost.localdomain>

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH 2/8] entry: factor out unlink_entry function
  2018-12-10 15:49   ` Duy Nguyen
  2018-12-10 17:23     ` Elijah Newren
  2018-12-11  2:23     ` Junio C Hamano
@ 2018-12-20 13:36     ` Thomas Gummerer
  2 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-20 13:36 UTC (permalink / raw)
  To: Duy Nguyen; +Cc: Git Mailing List, Junio C Hamano, Elijah Newren

On 12/10, Duy Nguyen wrote:
> On Sun, Dec 9, 2018 at 9:05 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> >
> > Factor out the 'unlink_entry()' function from unpack-trees.c to
> > entry.c.  It will be used in other places as well in subsequent
> > steps.
> >
> > As it's no longer a static function, also move the documentation to
> > the header file to make it more discoverable.
> >
> > Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> > ---
> >  cache.h        |  5 +++++
> >  entry.c        | 15 +++++++++++++++
> >  unpack-trees.c | 19 -------------------
> >  3 files changed, 20 insertions(+), 19 deletions(-)
> >
> > diff --git a/cache.h b/cache.h
> > index ca36b44ee0..c1c953e810 100644
> > --- a/cache.h
> > +++ b/cache.h
> > @@ -1542,6 +1542,11 @@ struct checkout {
> >  extern int checkout_entry(struct cache_entry *ce, const struct checkout *state, char *topath);
> >  extern void enable_delayed_checkout(struct checkout *state);
> >  extern int finish_delayed_checkout(struct checkout *state);
> > +/*
> > + * Unlink the last component and schedule the leading directories for
> > + * removal, such that empty directories get removed.
> > + */
> > +extern void unlink_entry(const struct cache_entry *ce);
> 
> I'm torn. We try to remove 'extern' but I can see you may want to add
> it here to be consistent with others. And removing extern even from
> functions from entry.c only would cause some conflicts.

Yeah I felt like favoring consistency here would be better.  Once your
path counting series and my series land, this may get quieter and we
can remove the 'extern' then?

> I wonder if we should move the 'removal' variable in symlinks to
> 'struct checkout' to reduce another global variable. But I guess
> that's the problem for another day. It's not the focus of this
> series.

Yeah, I'd prefer leaving that for another day :)

> -- 
> Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH v2 0/8] introduce no-overlay mode in git checkout
  2018-12-09 20:04 [PATCH 0/8] introduce no-overlay and cached mode in git checkout Thomas Gummerer
                   ` (10 preceding siblings ...)
  2018-12-10 17:18 ` Elijah Newren
@ 2018-12-20 13:48 ` " Thomas Gummerer
  2018-12-20 13:48   ` [PATCH v2 1/8] move worktree tests to t24* Thomas Gummerer
                     ` (8 more replies)
  11 siblings, 9 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-20 13:48 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Thomas Gummerer

Previous round is at <20181209200449.16342-1-t.gummerer@gmail.com>.

Thanks Junio, Duy and Elijah for your comments and suggestions on the
previous round.

This round drops the last three patches from the previous round,
namely introducing a "--cached" and a "--ignore-unmatched" option, and
using the new no-overlay mode in "git stash".  The --ignore-unmatched
option may not be necessary, while using the new mode in 'git stash'
will be done once the stash-in-C topic landed.

Introducing a --cached and --worktree-only (as suggested by Elijah)
option can come in a future step, they are orthogonal to this topic.

Other changes from v1:
- Rebase onto the current master, so we can also move t2028 and t2029
  to the t24xx range.
- Add a comment clarifying why using the CE_WT_REMOVE flag and topath
  in checkout_entry is a bug.
- clarify a comment in checkout.c
- factor out the function to mark a cache entry as CE_MATCHED, and
  have separate such functions for overlay mode and no-overlay mode.
  This should hopefully make the logic a bit easier to follow.
- Adjust the commit message, justifying why we don't remove untracked
  files even in the new no-overlay mode.
- add documentation for the new feature
- document that -p defaults to no overlay mode, and cannot be used
  with overlay mode.
- add a config option checkout.overlayMode, so overlay mode can be
  turned on by default.

Range-diff can be found after the diffstat.

Thomas Gummerer (8):
  move worktree tests to t24*
  entry: factor out unlink_entry function
  entry: support CE_WT_REMOVE flag in checkout_entry
  read-cache: add invalidate parameter to remove_marked_cache_entries
  checkout: clarify comment
  checkout: factor out mark_cache_entry_for_checkout function
  checkout: introduce --{,no-}overlay option
  checkout: introduce checkout.overlayMode config

 Documentation/config/checkout.txt             |   7 +
 Documentation/git-checkout.txt                |  10 ++
 builtin/checkout.c                            | 133 +++++++++++++-----
 cache.h                                       |   7 +-
 entry.c                                       |  26 ++++
 read-cache.c                                  |   8 +-
 split-index.c                                 |   2 +-
 t/t2025-checkout-no-overlay.sh                |  57 ++++++++
 ...-worktree-add.sh => t2400-worktree-add.sh} |   0
 ...ktree-prune.sh => t2401-worktree-prune.sh} |   0
 ...orktree-list.sh => t2402-worktree-list.sh} |   0
 ...orktree-move.sh => t2403-worktree-move.sh} |   0
 ...ree-config.sh => t2404-worktree-config.sh} |   0
 t/t9902-completion.sh                         |   1 +
 unpack-trees.c                                |  21 +--
 15 files changed, 213 insertions(+), 59 deletions(-)
 create mode 100755 t/t2025-checkout-no-overlay.sh
 rename t/{t2025-worktree-add.sh => t2400-worktree-add.sh} (100%)
 rename t/{t2026-worktree-prune.sh => t2401-worktree-prune.sh} (100%)
 rename t/{t2027-worktree-list.sh => t2402-worktree-list.sh} (100%)
 rename t/{t2028-worktree-move.sh => t2403-worktree-move.sh} (100%)
 rename t/{t2029-worktree-config.sh => t2404-worktree-config.sh} (100%)

1:  70bd75b202 ! 1:  fa450cda7c move worktree tests to t24*
    @@ -29,3 +29,13 @@
      similarity index 100%
      rename from t/t2027-worktree-list.sh
      rename to t/t2402-worktree-list.sh
    +
    + diff --git a/t/t2028-worktree-move.sh b/t/t2403-worktree-move.sh
    + similarity index 100%
    + rename from t/t2028-worktree-move.sh
    + rename to t/t2403-worktree-move.sh
    +
    + diff --git a/t/t2029-worktree-config.sh b/t/t2404-worktree-config.sh
    + similarity index 100%
    + rename from t/t2029-worktree-config.sh
    + rename to t/t2404-worktree-config.sh
2:  0fd9be987d = 2:  9ada8d3484 entry: factor out unlink_entry function
3:  4d6112b112 ! 3:  41c0ea4047 entry: support CE_WT_REMOVE flag in checkout_entry
    @@ -22,6 +22,10 @@
      
     +	if (ce->ce_flags & CE_WT_REMOVE) {
     +		if (topath)
    ++			/*
    ++			 * No content and thus no path to create, so we have
    ++			 * no pathname to return.
    ++			 */
     +			BUG("Can't remove entry to a path");
     +		unlink_entry(ce);
     +		return 0;
4:  6e9f68b8f1 ! 4:  afccb0848d read-cache: add invalidate parameter to remove_marked_cache_entries
    @@ -11,6 +11,10 @@
         function will take care of invalidating the path in the cache tree and
         in the untracked cache.
     
    +    Note that the current callsites already do the invalidation properly
    +    in other places, so we're just passing 0 from there to keep the status
    +    quo.
    +
         This will be useful in a subsequent commit.
     
         Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
-:  ---------- > 5:  8a2b5efdad checkout: clarify comment
-:  ---------- > 6:  c405f20471 checkout: factor out mark_cache_entry_for_checkout function
5:  4a7670d34c ! 7:  e5b18bcd02 checkout: introduce --{,no-}overlay option
    @@ -17,8 +17,43 @@
         'git checkout --overlay -p' to avoid confusing users who would expect
         to be able to force overlay mode in 'git checkout -p' this way.
     
    +    Untracked files are not affected by this change, so 'git checkout
    +    --no-overlay HEAD -- untracked' will not remove untracked from the
    +    working tree.  This is so e.g. 'git checkout --no-overlay HEAD -- dir/'
    +    doesn't delete all untracked files in dir/, but rather just resets the
    +    state of files that are known to git.
    +
    +    Suggested-by: Junio C Hamano <gitster@pobox.com>
         Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
     
    + diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
    + --- a/Documentation/git-checkout.txt
    + +++ b/Documentation/git-checkout.txt
    +@@
    + This means that you can use `git checkout -p` to selectively discard
    + edits from your current working tree. See the ``Interactive Mode''
    + section of linkgit:git-add[1] to learn how to operate the `--patch` mode.
    +++
    ++Note that this option uses the no overlay mode by default (see also
    ++-`--[no-]overlay`), and currently doesn't support overlay mode.
    + 
    + --ignore-other-worktrees::
    + 	`git checkout` refuses when the wanted ref is already checked
    +@@
    + 	Just like linkgit:git-submodule[1], this will detach the
    + 	submodules HEAD.
    + 
    ++--[no-]overlay::
    ++	In the default overlay mode files `git checkout` never
    ++	removes files from the index or the working tree.  When
    ++	specifying --no-overlay, files that appear in the index and
    ++	working tree, but not in <tree-ish> are removed, to make them
    ++	match <tree-ish> exactly.
    ++
    + <branch>::
    + 	Branch to checkout; if it refers to a branch (i.e., a name that,
    + 	when prepended with "refs/heads/", is a valid ref), then that
    +
      diff --git a/builtin/checkout.c b/builtin/checkout.c
      --- a/builtin/checkout.c
      +++ b/builtin/checkout.c
    @@ -70,44 +105,60 @@
      		return error(_("path '%s' does not have our version"), ce->name);
      	else
     @@
    - 		ce->ce_flags &= ~CE_MATCHED;
    - 		if (!opts->ignore_skipworktree && ce_skip_worktree(ce))
    - 			continue;
    --		if (opts->source_tree && !(ce->ce_flags & CE_UPDATE))
    --			/*
    --			 * "git checkout tree-ish -- path", but this entry
    --			 * is in the original index; it will not be checked
    --			 * out to the working tree and it does not matter
    --			 * if pathspec matched this entry.  We will not do
    --			 * anything to this entry at all.
    --			 */
    --			continue;
    -+		if (opts->source_tree && !(ce->ce_flags & CE_UPDATE)) {
    -+			if (!opts->overlay_mode &&
    -+			    ce_path_match(&the_index, ce, &opts->pathspec, ps_matched)) {
    -+				/*
    -+				 * "git checkout --no-overlay <tree-ish> -- path",
    -+				 * and the path is not in tree-ish, but is in
    -+				 * the current index, which means that it should 
    -+				 * be removed.
    -+				 */
    -+				ce->ce_flags |= CE_MATCHED | CE_REMOVE | CE_WT_REMOVE;
    -+				continue;
    -+			} else {
    -+				/*
    -+				 * "git checkout tree-ish -- path", but this
    -+				 * entry is in the original index; it will not
    -+				 * be checked out to the working tree and it
    -+				 * does not matter if pathspec matched this
    -+				 * entry.  We will not do anything to this entry
    -+				 * at all.
    -+				 */
    -+				continue;
    -+			}
    -+		}
    - 		/*
    - 		 * Either this entry came from the tree-ish we are
    - 		 * checking the paths out of, or we are checking out
    + 	return status;
    + }
    + 
    +-static void mark_ce_for_checkout(struct cache_entry *ce,
    +-				 char *ps_matched,
    +-				 const struct checkout_opts *opts)
    ++static void mark_ce_for_checkout_overlay(struct cache_entry *ce,
    ++					 char *ps_matched,
    ++					 const struct checkout_opts *opts)
    + {
    + 	ce->ce_flags &= ~CE_MATCHED;
    + 	if (!opts->ignore_skipworktree && ce_skip_worktree(ce))
    +@@
    + 		ce->ce_flags |= CE_MATCHED;
    + }
    + 
    ++static void mark_ce_for_checkout_no_overlay(struct cache_entry *ce,
    ++					    char *ps_matched,
    ++					    const struct checkout_opts *opts)
    ++{
    ++	ce->ce_flags &= ~CE_MATCHED;
    ++	if (!opts->ignore_skipworktree && ce_skip_worktree(ce))
    ++		return;
    ++	if (ce_path_match(&the_index, ce, &opts->pathspec, ps_matched)) {
    ++		ce->ce_flags |= CE_MATCHED;
    ++		if (opts->source_tree && !(ce->ce_flags & CE_UPDATE))
    ++			/*
    ++			 * In overlay mode, but the path is not in
    ++			 * tree-ish, which means we should remove it
    ++			 * from the index and the working tree.
    ++			 */
    ++			ce->ce_flags |= CE_REMOVE | CE_WT_REMOVE;
    ++	}
    ++}
    ++
    + static int checkout_paths(const struct checkout_opts *opts,
    + 			  const char *revision)
    + {
    +@@
    + 	 * to be checked out.
    + 	 */
    + 	for (pos = 0; pos < active_nr; pos++)
    +-		mark_ce_for_checkout(active_cache[pos], ps_matched, opts);
    ++		if (opts->overlay_mode)
    ++			mark_ce_for_checkout_overlay(active_cache[pos],
    ++						     ps_matched,
    ++						     opts);
    ++		else
    ++			mark_ce_for_checkout_no_overlay(active_cache[pos],
    ++							ps_matched,
    ++							opts);
    + 
    + 	if (report_path_error(ps_matched, &opts->pathspec, opts->prefix)) {
    + 		free(ps_matched);
     @@
      			if (opts->force) {
      				warning(_("path '%s' is unmerged"), ce->name);
    @@ -160,7 +211,7 @@
      			    "checkout", "control recursive updating of submodules",
      			    PARSE_OPT_OPTARG, option_parse_recurse_submodules_worktree_updater },
      		OPT_BOOL(0, "progress", &opts.show_progress, N_("force progress reporting")),
    -+		OPT_BOOL(0, "overlay", &opts.overlay_mode, N_("use overlay mode")),
    ++		OPT_BOOL(0, "overlay", &opts.overlay_mode, N_("use overlay mode (default)")),
      		OPT_END(),
      	};
      
    @@ -198,7 +249,7 @@
     +	git commit --allow-empty -m "initial"
     +'
     +
    -+test_expect_success 'checkout --no-overlay deletes files not in <tree>' '
    ++test_expect_success 'checkout --no-overlay deletes files not in <tree-ish>' '
     +	>file &&
     +	mkdir dir &&
     +	>dir/file1 &&
    @@ -218,7 +269,7 @@
     +	test_i18ngrep "fatal: -p and --overlay are mutually exclusive" actual
     +'
     +
    -+test_expect_success '--no-overlay --theirs with M/D conflict deletes file' '
    ++test_expect_success '--no-overlay --theirs with D/F conflict deletes file' '
     +	test_commit file1 file1 &&
     +	test_commit file2 file2 &&
     +	git rm --cached file1 &&
6:  695b671675 < -:  ---------- checkout: add --cached option
7:  d0b5a356b2 < -:  ---------- checkout: allow ignoring unmatched pathspec
8:  0a4565acc1 < -:  ---------- stash: use git checkout --no-overlay
-:  ---------- > 8:  de24990d57 checkout: introduce checkout.overlayMode config

-- 
2.20.1.415.g653613c723

^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH v2 1/8] move worktree tests to t24*
  2018-12-20 13:48 ` [PATCH v2 0/8] introduce no-overlay " Thomas Gummerer
@ 2018-12-20 13:48   ` Thomas Gummerer
  2018-12-20 13:48   ` [PATCH v2 2/8] entry: factor out unlink_entry function Thomas Gummerer
                     ` (7 subsequent siblings)
  8 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-20 13:48 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Thomas Gummerer

The 'git worktree' command used to be just another mode in 'git
checkout', namely 'git checkout --to'.  When the tests for the latter
were retrofitted for the former, the test name was adjusted, but the
test number was kept, even though the test is testing a different
command now.  t/README states: "Second digit tells the particular
command we are testing.", so 'git worktree' should have a separate
number just for itself.

Move the worktree tests to t24* to adhere to that guideline. We're
going to make use of the free'd up numbers in a subsequent commit.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 t/{t2025-worktree-add.sh => t2400-worktree-add.sh}       | 0
 t/{t2026-worktree-prune.sh => t2401-worktree-prune.sh}   | 0
 t/{t2027-worktree-list.sh => t2402-worktree-list.sh}     | 0
 t/{t2028-worktree-move.sh => t2403-worktree-move.sh}     | 0
 t/{t2029-worktree-config.sh => t2404-worktree-config.sh} | 0
 5 files changed, 0 insertions(+), 0 deletions(-)
 rename t/{t2025-worktree-add.sh => t2400-worktree-add.sh} (100%)
 rename t/{t2026-worktree-prune.sh => t2401-worktree-prune.sh} (100%)
 rename t/{t2027-worktree-list.sh => t2402-worktree-list.sh} (100%)
 rename t/{t2028-worktree-move.sh => t2403-worktree-move.sh} (100%)
 rename t/{t2029-worktree-config.sh => t2404-worktree-config.sh} (100%)

diff --git a/t/t2025-worktree-add.sh b/t/t2400-worktree-add.sh
similarity index 100%
rename from t/t2025-worktree-add.sh
rename to t/t2400-worktree-add.sh
diff --git a/t/t2026-worktree-prune.sh b/t/t2401-worktree-prune.sh
similarity index 100%
rename from t/t2026-worktree-prune.sh
rename to t/t2401-worktree-prune.sh
diff --git a/t/t2027-worktree-list.sh b/t/t2402-worktree-list.sh
similarity index 100%
rename from t/t2027-worktree-list.sh
rename to t/t2402-worktree-list.sh
diff --git a/t/t2028-worktree-move.sh b/t/t2403-worktree-move.sh
similarity index 100%
rename from t/t2028-worktree-move.sh
rename to t/t2403-worktree-move.sh
diff --git a/t/t2029-worktree-config.sh b/t/t2404-worktree-config.sh
similarity index 100%
rename from t/t2029-worktree-config.sh
rename to t/t2404-worktree-config.sh
-- 
2.20.1.415.g653613c723


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH v2 2/8] entry: factor out unlink_entry function
  2018-12-20 13:48 ` [PATCH v2 0/8] introduce no-overlay " Thomas Gummerer
  2018-12-20 13:48   ` [PATCH v2 1/8] move worktree tests to t24* Thomas Gummerer
@ 2018-12-20 13:48   ` Thomas Gummerer
  2018-12-20 13:48   ` [PATCH v2 3/8] entry: support CE_WT_REMOVE flag in checkout_entry Thomas Gummerer
                     ` (6 subsequent siblings)
  8 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-20 13:48 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Thomas Gummerer

Factor out the 'unlink_entry()' function from unpack-trees.c to
entry.c.  It will be used in other places as well in subsequent
steps.

As it's no longer a static function, also move the documentation to
the header file to make it more discoverable.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 cache.h        |  5 +++++
 entry.c        | 15 +++++++++++++++
 unpack-trees.c | 19 -------------------
 3 files changed, 20 insertions(+), 19 deletions(-)

diff --git a/cache.h b/cache.h
index ca36b44ee0..c1c953e810 100644
--- a/cache.h
+++ b/cache.h
@@ -1542,6 +1542,11 @@ struct checkout {
 extern int checkout_entry(struct cache_entry *ce, const struct checkout *state, char *topath);
 extern void enable_delayed_checkout(struct checkout *state);
 extern int finish_delayed_checkout(struct checkout *state);
+/*
+ * Unlink the last component and schedule the leading directories for
+ * removal, such that empty directories get removed.
+ */
+extern void unlink_entry(const struct cache_entry *ce);
 
 struct cache_def {
 	struct strbuf path;
diff --git a/entry.c b/entry.c
index 0a3c451f5f..b9eef57117 100644
--- a/entry.c
+++ b/entry.c
@@ -508,3 +508,18 @@ int checkout_entry(struct cache_entry *ce,
 	create_directories(path.buf, path.len, state);
 	return write_entry(ce, path.buf, state, 0);
 }
+
+void unlink_entry(const struct cache_entry *ce)
+{
+	const struct submodule *sub = submodule_from_ce(ce);
+	if (sub) {
+		/* state.force is set at the caller. */
+		submodule_move_head(ce->name, "HEAD", NULL,
+				    SUBMODULE_MOVE_HEAD_FORCE);
+	}
+	if (!check_leading_path(ce->name, ce_namelen(ce)))
+		return;
+	if (remove_or_warn(ce->ce_mode, ce->name))
+		return;
+	schedule_dir_for_removal(ce->name, ce_namelen(ce));
+}
diff --git a/unpack-trees.c b/unpack-trees.c
index 7570df481b..e8d1a6ac50 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -300,25 +300,6 @@ static void load_gitmodules_file(struct index_state *index,
 	}
 }
 
-/*
- * Unlink the last component and schedule the leading directories for
- * removal, such that empty directories get removed.
- */
-static void unlink_entry(const struct cache_entry *ce)
-{
-	const struct submodule *sub = submodule_from_ce(ce);
-	if (sub) {
-		/* state.force is set at the caller. */
-		submodule_move_head(ce->name, "HEAD", NULL,
-				    SUBMODULE_MOVE_HEAD_FORCE);
-	}
-	if (!check_leading_path(ce->name, ce_namelen(ce)))
-		return;
-	if (remove_or_warn(ce->ce_mode, ce->name))
-		return;
-	schedule_dir_for_removal(ce->name, ce_namelen(ce));
-}
-
 static struct progress *get_progress(struct unpack_trees_options *o)
 {
 	unsigned cnt = 0, total = 0;
-- 
2.20.1.415.g653613c723


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH v2 3/8] entry: support CE_WT_REMOVE flag in checkout_entry
  2018-12-20 13:48 ` [PATCH v2 0/8] introduce no-overlay " Thomas Gummerer
  2018-12-20 13:48   ` [PATCH v2 1/8] move worktree tests to t24* Thomas Gummerer
  2018-12-20 13:48   ` [PATCH v2 2/8] entry: factor out unlink_entry function Thomas Gummerer
@ 2018-12-20 13:48   ` Thomas Gummerer
  2018-12-20 13:48   ` [PATCH v2 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries Thomas Gummerer
                     ` (5 subsequent siblings)
  8 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-20 13:48 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Thomas Gummerer

'checkout_entry()' currently only supports creating new entries in the
working tree, but not deleting them.  Add the ability to remove
entries at the same time if the entry is marked with the CE_WT_REMOVE
flag.

Currently this doesn't have any effect, as the CE_WT_REMOVE flag is
only used in unpack-tree, however we will make use of this in a
subsequent step in the series.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 entry.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/entry.c b/entry.c
index b9eef57117..3d3701e7ae 100644
--- a/entry.c
+++ b/entry.c
@@ -441,6 +441,17 @@ int checkout_entry(struct cache_entry *ce,
 	static struct strbuf path = STRBUF_INIT;
 	struct stat st;
 
+	if (ce->ce_flags & CE_WT_REMOVE) {
+		if (topath)
+			/*
+			 * No content and thus no path to create, so we have
+			 * no pathname to return.
+			 */
+			BUG("Can't remove entry to a path");
+		unlink_entry(ce);
+		return 0;
+	}
+
 	if (topath)
 		return write_entry(ce, topath, state, 1);
 
-- 
2.20.1.415.g653613c723


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH v2 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries
  2018-12-20 13:48 ` [PATCH v2 0/8] introduce no-overlay " Thomas Gummerer
                     ` (2 preceding siblings ...)
  2018-12-20 13:48   ` [PATCH v2 3/8] entry: support CE_WT_REMOVE flag in checkout_entry Thomas Gummerer
@ 2018-12-20 13:48   ` Thomas Gummerer
  2018-12-20 13:48   ` [PATCH v2 5/8] checkout: clarify comment Thomas Gummerer
                     ` (4 subsequent siblings)
  8 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-20 13:48 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Thomas Gummerer

When marking cache entries for removal, and later removing them all at
once using 'remove_marked_cache_entries()', cache entries currently
have to be invalidated manually in the cache tree and in the untracked
cache.

Add an invalidate flag to the function.  With the flag set, the
function will take care of invalidating the path in the cache tree and
in the untracked cache.

Note that the current callsites already do the invalidation properly
in other places, so we're just passing 0 from there to keep the status
quo.

This will be useful in a subsequent commit.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 cache.h        | 2 +-
 read-cache.c   | 8 +++++++-
 split-index.c  | 2 +-
 unpack-trees.c | 2 +-
 4 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/cache.h b/cache.h
index c1c953e810..1deee48f5b 100644
--- a/cache.h
+++ b/cache.h
@@ -751,7 +751,7 @@ extern void rename_index_entry_at(struct index_state *, int pos, const char *new
 /* Remove entry, return true if there are more entries to go. */
 extern int remove_index_entry_at(struct index_state *, int pos);
 
-extern void remove_marked_cache_entries(struct index_state *istate);
+extern void remove_marked_cache_entries(struct index_state *istate, int invalidate);
 extern int remove_file_from_index(struct index_state *, const char *path);
 #define ADD_CACHE_VERBOSE 1
 #define ADD_CACHE_PRETEND 2
diff --git a/read-cache.c b/read-cache.c
index bd45dc3e24..978d43f676 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -590,13 +590,19 @@ int remove_index_entry_at(struct index_state *istate, int pos)
  * CE_REMOVE is set in ce_flags.  This is much more effective than
  * calling remove_index_entry_at() for each entry to be removed.
  */
-void remove_marked_cache_entries(struct index_state *istate)
+void remove_marked_cache_entries(struct index_state *istate, int invalidate)
 {
 	struct cache_entry **ce_array = istate->cache;
 	unsigned int i, j;
 
 	for (i = j = 0; i < istate->cache_nr; i++) {
 		if (ce_array[i]->ce_flags & CE_REMOVE) {
+			if (invalidate) {
+				cache_tree_invalidate_path(istate,
+							   ce_array[i]->name);
+				untracked_cache_remove_from_index(istate,
+								  ce_array[i]->name);
+			}
 			remove_name_hash(istate, ce_array[i]);
 			save_or_free_index_entry(istate, ce_array[i]);
 		}
diff --git a/split-index.c b/split-index.c
index 5820412dc5..8aebc3661b 100644
--- a/split-index.c
+++ b/split-index.c
@@ -162,7 +162,7 @@ void merge_base_index(struct index_state *istate)
 	ewah_each_bit(si->replace_bitmap, replace_entry, istate);
 	ewah_each_bit(si->delete_bitmap, mark_entry_for_delete, istate);
 	if (si->nr_deletions)
-		remove_marked_cache_entries(istate);
+		remove_marked_cache_entries(istate, 0);
 
 	for (i = si->nr_replacements; i < si->saved_cache_nr; i++) {
 		if (!ce_namelen(si->saved_cache[i]))
diff --git a/unpack-trees.c b/unpack-trees.c
index e8d1a6ac50..8e6afa924d 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -392,7 +392,7 @@ static int check_updates(struct unpack_trees_options *o)
 				unlink_entry(ce);
 		}
 	}
-	remove_marked_cache_entries(index);
+	remove_marked_cache_entries(index, 0);
 	remove_scheduled_dirs();
 
 	if (should_update_submodules() && o->update && !o->dry_run)
-- 
2.20.1.415.g653613c723


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH v2 5/8] checkout: clarify comment
  2018-12-20 13:48 ` [PATCH v2 0/8] introduce no-overlay " Thomas Gummerer
                     ` (3 preceding siblings ...)
  2018-12-20 13:48   ` [PATCH v2 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries Thomas Gummerer
@ 2018-12-20 13:48   ` Thomas Gummerer
  2018-12-20 13:48   ` [PATCH v2 6/8] checkout: factor out mark_cache_entry_for_checkout function Thomas Gummerer
                     ` (3 subsequent siblings)
  8 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-20 13:48 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Thomas Gummerer

The key point for the if statement is that read_tree_some did not
update the entry, because either it doesn't exist in tree-ish or
doesn't match the pathspec.  Clarify that.

Suggested-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 builtin/checkout.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index acdafc6e4c..cb166b2e07 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -304,10 +304,10 @@ static int checkout_paths(const struct checkout_opts *opts,
 			continue;
 		if (opts->source_tree && !(ce->ce_flags & CE_UPDATE))
 			/*
-			 * "git checkout tree-ish -- path", but this entry
-			 * is in the original index; it will not be checked
-			 * out to the working tree and it does not matter
-			 * if pathspec matched this entry.  We will not do
+			 * "git checkout tree-ish -- path" and this entry
+			 * is in the original index, but is not in tree-ish
+			 * or does not match the pathspec; it will not be
+			 * checked out to the working tree.  We will not do
 			 * anything to this entry at all.
 			 */
 			continue;
-- 
2.20.1.415.g653613c723


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH v2 6/8] checkout: factor out mark_cache_entry_for_checkout function
  2018-12-20 13:48 ` [PATCH v2 0/8] introduce no-overlay " Thomas Gummerer
                     ` (4 preceding siblings ...)
  2018-12-20 13:48   ` [PATCH v2 5/8] checkout: clarify comment Thomas Gummerer
@ 2018-12-20 13:48   ` Thomas Gummerer
  2018-12-20 13:48   ` [PATCH v2 7/8] checkout: introduce --{,no-}overlay option Thomas Gummerer
                     ` (2 subsequent siblings)
  8 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-20 13:48 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Thomas Gummerer

Factor out the code that marks a cache entry as matched for checkout
into a separate function.  We are going to introduce a new mode in
'git checkout' in a subsequent commit, that is going to have a
slightly different logic.  This would make this code unnecessarily
complex.

Moving that complexity into separate functions will make the code in
the subsequent step easier to follow.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 builtin/checkout.c | 67 +++++++++++++++++++++++++---------------------
 1 file changed, 36 insertions(+), 31 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index cb166b2e07..32c4b7f897 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -247,6 +247,40 @@ static int checkout_merged(int pos, const struct checkout *state)
 	return status;
 }
 
+static void mark_ce_for_checkout(struct cache_entry *ce,
+				 char *ps_matched,
+				 const struct checkout_opts *opts)
+{
+	ce->ce_flags &= ~CE_MATCHED;
+	if (!opts->ignore_skipworktree && ce_skip_worktree(ce))
+		return;
+	if (opts->source_tree && !(ce->ce_flags & CE_UPDATE))
+		/*
+		 * "git checkout tree-ish -- path", but this entry
+		 * is in the original index but is not in tree-ish
+		 * or does not match the pathspec; it will not be
+		 * checked out to the working tree.  We will not do
+		 * anything to this entry at all.
+		 */
+		return;
+	/*
+	 * Either this entry came from the tree-ish we are
+	 * checking the paths out of, or we are checking out
+	 * of the index.
+	 *
+	 * If it comes from the tree-ish, we already know it
+	 * matches the pathspec and could just stamp
+	 * CE_MATCHED to it from update_some(). But we still
+	 * need ps_matched and read_tree_recursive (and
+	 * eventually tree_entry_interesting) cannot fill
+	 * ps_matched yet. Once it can, we can avoid calling
+	 * match_pathspec() for _all_ entries when
+	 * opts->source_tree != NULL.
+	 */
+	if (ce_path_match(&the_index, ce, &opts->pathspec, ps_matched))
+		ce->ce_flags |= CE_MATCHED;
+}
+
 static int checkout_paths(const struct checkout_opts *opts,
 			  const char *revision)
 {
@@ -297,37 +331,8 @@ static int checkout_paths(const struct checkout_opts *opts,
 	 * Make sure all pathspecs participated in locating the paths
 	 * to be checked out.
 	 */
-	for (pos = 0; pos < active_nr; pos++) {
-		struct cache_entry *ce = active_cache[pos];
-		ce->ce_flags &= ~CE_MATCHED;
-		if (!opts->ignore_skipworktree && ce_skip_worktree(ce))
-			continue;
-		if (opts->source_tree && !(ce->ce_flags & CE_UPDATE))
-			/*
-			 * "git checkout tree-ish -- path" and this entry
-			 * is in the original index, but is not in tree-ish
-			 * or does not match the pathspec; it will not be
-			 * checked out to the working tree.  We will not do
-			 * anything to this entry at all.
-			 */
-			continue;
-		/*
-		 * Either this entry came from the tree-ish we are
-		 * checking the paths out of, or we are checking out
-		 * of the index.
-		 *
-		 * If it comes from the tree-ish, we already know it
-		 * matches the pathspec and could just stamp
-		 * CE_MATCHED to it from update_some(). But we still
-		 * need ps_matched and read_tree_recursive (and
-		 * eventually tree_entry_interesting) cannot fill
-		 * ps_matched yet. Once it can, we can avoid calling
-		 * match_pathspec() for _all_ entries when
-		 * opts->source_tree != NULL.
-		 */
-		if (ce_path_match(&the_index, ce, &opts->pathspec, ps_matched))
-			ce->ce_flags |= CE_MATCHED;
-	}
+	for (pos = 0; pos < active_nr; pos++)
+		mark_ce_for_checkout(active_cache[pos], ps_matched, opts);
 
 	if (report_path_error(ps_matched, &opts->pathspec, opts->prefix)) {
 		free(ps_matched);
-- 
2.20.1.415.g653613c723


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH v2 7/8] checkout: introduce --{,no-}overlay option
  2018-12-20 13:48 ` [PATCH v2 0/8] introduce no-overlay " Thomas Gummerer
                     ` (5 preceding siblings ...)
  2018-12-20 13:48   ` [PATCH v2 6/8] checkout: factor out mark_cache_entry_for_checkout function Thomas Gummerer
@ 2018-12-20 13:48   ` Thomas Gummerer
  2018-12-23  8:05     ` Duy Nguyen
  2018-12-20 13:48   ` [PATCH v2 8/8] checkout: introduce checkout.overlayMode config Thomas Gummerer
  2019-01-08 21:52   ` [PATCH v3 0/8] introduce no-overlay mode in git checkout Thomas Gummerer
  8 siblings, 1 reply; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-20 13:48 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Thomas Gummerer

Currently 'git checkout' is defined as an overlay operation, which
means that if in 'git checkout <tree-ish> -- [<pathspec>]' we have an
entry in the index that matches <pathspec>, but that doesn't exist in
<tree-ish>, that entry will not be removed from the index or the
working tree.

Introduce a new --{,no-}overlay option, which allows using 'git
checkout' in non-overlay mode, thus removing files from the working
tree if they do not exist in <tree-ish> but match <pathspec>.

Note that 'git checkout -p <tree-ish> -- [<pathspec>]' already works
this way, so no changes are needed for the patch mode.  We disallow
'git checkout --overlay -p' to avoid confusing users who would expect
to be able to force overlay mode in 'git checkout -p' this way.

Untracked files are not affected by this change, so 'git checkout
--no-overlay HEAD -- untracked' will not remove untracked from the
working tree.  This is so e.g. 'git checkout --no-overlay HEAD -- dir/'
doesn't delete all untracked files in dir/, but rather just resets the
state of files that are known to git.

Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 Documentation/git-checkout.txt | 10 ++++++
 builtin/checkout.c             | 66 +++++++++++++++++++++++++++++-----
 t/t2025-checkout-no-overlay.sh | 47 ++++++++++++++++++++++++
 t/t9902-completion.sh          |  1 +
 4 files changed, 116 insertions(+), 8 deletions(-)
 create mode 100755 t/t2025-checkout-no-overlay.sh

diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
index 801de2f764..4ac8c55865 100644
--- a/Documentation/git-checkout.txt
+++ b/Documentation/git-checkout.txt
@@ -260,6 +260,9 @@ the conflicted merge in the specified paths.
 This means that you can use `git checkout -p` to selectively discard
 edits from your current working tree. See the ``Interactive Mode''
 section of linkgit:git-add[1] to learn how to operate the `--patch` mode.
++
+Note that this option uses the no overlay mode by default (see also
+-`--[no-]overlay`), and currently doesn't support overlay mode.
 
 --ignore-other-worktrees::
 	`git checkout` refuses when the wanted ref is already checked
@@ -276,6 +279,13 @@ section of linkgit:git-add[1] to learn how to operate the `--patch` mode.
 	Just like linkgit:git-submodule[1], this will detach the
 	submodules HEAD.
 
+--[no-]overlay::
+	In the default overlay mode files `git checkout` never
+	removes files from the index or the working tree.  When
+	specifying --no-overlay, files that appear in the index and
+	working tree, but not in <tree-ish> are removed, to make them
+	match <tree-ish> exactly.
+
 <branch>::
 	Branch to checkout; if it refers to a branch (i.e., a name that,
 	when prepended with "refs/heads/", is a valid ref), then that
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 32c4b7f897..0c5fe948ef 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -44,6 +44,7 @@ struct checkout_opts {
 	int ignore_skipworktree;
 	int ignore_other_worktrees;
 	int show_progress;
+	int overlay_mode;
 	/*
 	 * If new checkout options are added, skip_merge_working_tree
 	 * should be updated accordingly.
@@ -132,7 +133,8 @@ static int skip_same_name(const struct cache_entry *ce, int pos)
 	return pos;
 }
 
-static int check_stage(int stage, const struct cache_entry *ce, int pos)
+static int check_stage(int stage, const struct cache_entry *ce, int pos,
+		       int overlay_mode)
 {
 	while (pos < active_nr &&
 	       !strcmp(active_cache[pos]->name, ce->name)) {
@@ -140,6 +142,8 @@ static int check_stage(int stage, const struct cache_entry *ce, int pos)
 			return 0;
 		pos++;
 	}
+	if (!overlay_mode)
+		return 0;
 	if (stage == 2)
 		return error(_("path '%s' does not have our version"), ce->name);
 	else
@@ -165,7 +169,7 @@ static int check_stages(unsigned stages, const struct cache_entry *ce, int pos)
 }
 
 static int checkout_stage(int stage, const struct cache_entry *ce, int pos,
-			  const struct checkout *state)
+			  const struct checkout *state, int overlay_mode)
 {
 	while (pos < active_nr &&
 	       !strcmp(active_cache[pos]->name, ce->name)) {
@@ -173,6 +177,10 @@ static int checkout_stage(int stage, const struct cache_entry *ce, int pos,
 			return checkout_entry(active_cache[pos], state, NULL);
 		pos++;
 	}
+	if (!overlay_mode) {
+		unlink_entry(ce);
+		return 0;
+	}
 	if (stage == 2)
 		return error(_("path '%s' does not have our version"), ce->name);
 	else
@@ -247,9 +255,9 @@ static int checkout_merged(int pos, const struct checkout *state)
 	return status;
 }
 
-static void mark_ce_for_checkout(struct cache_entry *ce,
-				 char *ps_matched,
-				 const struct checkout_opts *opts)
+static void mark_ce_for_checkout_overlay(struct cache_entry *ce,
+					 char *ps_matched,
+					 const struct checkout_opts *opts)
 {
 	ce->ce_flags &= ~CE_MATCHED;
 	if (!opts->ignore_skipworktree && ce_skip_worktree(ce))
@@ -281,6 +289,25 @@ static void mark_ce_for_checkout(struct cache_entry *ce,
 		ce->ce_flags |= CE_MATCHED;
 }
 
+static void mark_ce_for_checkout_no_overlay(struct cache_entry *ce,
+					    char *ps_matched,
+					    const struct checkout_opts *opts)
+{
+	ce->ce_flags &= ~CE_MATCHED;
+	if (!opts->ignore_skipworktree && ce_skip_worktree(ce))
+		return;
+	if (ce_path_match(&the_index, ce, &opts->pathspec, ps_matched)) {
+		ce->ce_flags |= CE_MATCHED;
+		if (opts->source_tree && !(ce->ce_flags & CE_UPDATE))
+			/*
+			 * In overlay mode, but the path is not in
+			 * tree-ish, which means we should remove it
+			 * from the index and the working tree.
+			 */
+			ce->ce_flags |= CE_REMOVE | CE_WT_REMOVE;
+	}
+}
+
 static int checkout_paths(const struct checkout_opts *opts,
 			  const char *revision)
 {
@@ -332,7 +359,14 @@ static int checkout_paths(const struct checkout_opts *opts,
 	 * to be checked out.
 	 */
 	for (pos = 0; pos < active_nr; pos++)
-		mark_ce_for_checkout(active_cache[pos], ps_matched, opts);
+		if (opts->overlay_mode)
+			mark_ce_for_checkout_overlay(active_cache[pos],
+						     ps_matched,
+						     opts);
+		else
+			mark_ce_for_checkout_no_overlay(active_cache[pos],
+							ps_matched,
+							opts);
 
 	if (report_path_error(ps_matched, &opts->pathspec, opts->prefix)) {
 		free(ps_matched);
@@ -353,7 +387,7 @@ static int checkout_paths(const struct checkout_opts *opts,
 			if (opts->force) {
 				warning(_("path '%s' is unmerged"), ce->name);
 			} else if (opts->writeout_stage) {
-				errs |= check_stage(opts->writeout_stage, ce, pos);
+				errs |= check_stage(opts->writeout_stage, ce, pos, opts->overlay_mode);
 			} else if (opts->merge) {
 				errs |= check_stages((1<<2) | (1<<3), ce, pos);
 			} else {
@@ -380,12 +414,14 @@ static int checkout_paths(const struct checkout_opts *opts,
 				continue;
 			}
 			if (opts->writeout_stage)
-				errs |= checkout_stage(opts->writeout_stage, ce, pos, &state);
+				errs |= checkout_stage(opts->writeout_stage, ce, pos, &state, opts->overlay_mode);
 			else if (opts->merge)
 				errs |= checkout_merged(pos, &state);
 			pos = skip_same_name(ce, pos) - 1;
 		}
 	}
+	remove_marked_cache_entries(&the_index, 1);
+	remove_scheduled_dirs();
 	errs |= finish_delayed_checkout(&state);
 
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
@@ -547,6 +583,11 @@ static int skip_merge_working_tree(const struct checkout_opts *opts,
 	 * opts->show_progress only impacts output so doesn't require a merge
 	 */
 
+	/*
+	 * opts->overlay_mode cannot be used with switching branches so is
+	 * not tested here
+	 */
+
 	/*
 	 * If we aren't creating a new branch any changes or updates will
 	 * happen in the existing branch.  Since that could only be updating
@@ -1183,6 +1224,10 @@ static int checkout_branch(struct checkout_opts *opts,
 		die(_("'%s' cannot be used with switching branches"),
 		    "--patch");
 
+	if (!opts->overlay_mode)
+		die(_("'%s' cannot be used with switching branches"),
+		    "--no-overlay");
+
 	if (opts->writeout_stage)
 		die(_("'%s' cannot be used with switching branches"),
 		    "--ours/--theirs");
@@ -1271,6 +1316,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 			    "checkout", "control recursive updating of submodules",
 			    PARSE_OPT_OPTARG, option_parse_recurse_submodules_worktree_updater },
 		OPT_BOOL(0, "progress", &opts.show_progress, N_("force progress reporting")),
+		OPT_BOOL(0, "overlay", &opts.overlay_mode, N_("use overlay mode (default)")),
 		OPT_END(),
 	};
 
@@ -1279,6 +1325,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 	opts.overwrite_ignore = 1;
 	opts.prefix = prefix;
 	opts.show_progress = -1;
+	opts.overlay_mode = -1;
 
 	git_config(git_checkout_config, &opts);
 
@@ -1302,6 +1349,9 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 	if ((!!opts.new_branch + !!opts.new_branch_force + !!opts.new_orphan_branch) > 1)
 		die(_("-b, -B and --orphan are mutually exclusive"));
 
+	if (opts.overlay_mode == 1 && opts.patch_mode)
+		die(_("-p and --overlay are mutually exclusive"));
+
 	/*
 	 * From here on, new_branch will contain the branch to be checked out,
 	 * and new_branch_force and new_orphan_branch will tell us which one of
diff --git a/t/t2025-checkout-no-overlay.sh b/t/t2025-checkout-no-overlay.sh
new file mode 100755
index 0000000000..76330cb5ab
--- /dev/null
+++ b/t/t2025-checkout-no-overlay.sh
@@ -0,0 +1,47 @@
+#!/bin/sh
+
+test_description='checkout --no-overlay <tree-ish> -- <pathspec>'
+
+. ./test-lib.sh
+
+test_expect_success 'setup' '
+	git commit --allow-empty -m "initial"
+'
+
+test_expect_success 'checkout --no-overlay deletes files not in <tree-ish>' '
+	>file &&
+	mkdir dir &&
+	>dir/file1 &&
+	git add file dir/file1 &&
+	git checkout --no-overlay HEAD -- file &&
+	test_path_is_missing file &&
+	test_path_is_file dir/file1
+'
+
+test_expect_success 'checkout --no-overlay removing last file from directory' '
+	git checkout --no-overlay HEAD -- dir/file1 &&
+	test_path_is_missing dir
+'
+
+test_expect_success 'checkout -p --overlay is disallowed' '
+	test_must_fail git checkout -p --overlay HEAD 2>actual &&
+	test_i18ngrep "fatal: -p and --overlay are mutually exclusive" actual
+'
+
+test_expect_success '--no-overlay --theirs with D/F conflict deletes file' '
+	test_commit file1 file1 &&
+	test_commit file2 file2 &&
+	git rm --cached file1 &&
+	echo 1234 >file1 &&
+	F1=$(git rev-parse HEAD:file1) &&
+	F2=$(git rev-parse HEAD:file2) &&
+	{
+		echo "100644 $F1 1	file1" &&
+		echo "100644 $F2 2	file1"
+	} | git update-index --index-info &&
+	test_path_is_file file1 &&
+	git checkout --theirs --no-overlay -- file1 &&
+	test_path_is_missing file1
+'
+
+test_done
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index d01ad8eb25..5758fffa0d 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -1436,6 +1436,7 @@ test_expect_success 'double dash "git checkout"' '
 	--progress Z
 	--no-quiet Z
 	--no-... Z
+	--overlay Z
 	EOF
 '
 
-- 
2.20.1.415.g653613c723


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH v2 8/8] checkout: introduce checkout.overlayMode config
  2018-12-20 13:48 ` [PATCH v2 0/8] introduce no-overlay " Thomas Gummerer
                     ` (6 preceding siblings ...)
  2018-12-20 13:48   ` [PATCH v2 7/8] checkout: introduce --{,no-}overlay option Thomas Gummerer
@ 2018-12-20 13:48   ` Thomas Gummerer
  2019-01-02 23:39     ` Junio C Hamano
  2019-01-08 21:52   ` [PATCH v3 0/8] introduce no-overlay mode in git checkout Thomas Gummerer
  8 siblings, 1 reply; 81+ messages in thread
From: Thomas Gummerer @ 2018-12-20 13:48 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Thomas Gummerer

In the previous patch we introduced a new no-overlay mode for git
checkout.  Some users (such as the author of this commit) may want to
have this mode turned on by default as it matches their mental model
more closely.  Make that possible by introducing a new config option
to that extend.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 Documentation/config/checkout.txt |  7 +++++++
 builtin/checkout.c                |  8 +++++++-
 t/t2025-checkout-no-overlay.sh    | 10 ++++++++++
 3 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/Documentation/config/checkout.txt b/Documentation/config/checkout.txt
index c4118fa196..53f917e15e 100644
--- a/Documentation/config/checkout.txt
+++ b/Documentation/config/checkout.txt
@@ -21,3 +21,10 @@ checkout.optimizeNewBranch::
 	will not update the skip-worktree bit in the index nor add/remove
 	files in the working directory to reflect the current sparse checkout
 	settings nor will it show the local changes.
+
+checkout.overlayMode::
+	In the default overlay mode files `git checkout` never
+	removes files from the index or the working tree.  When
+	setting checkout.overlayMode to false, files that appear in
+	the index and working tree, but not in <tree-ish> are removed,
+	to make them match <tree-ish> exactly.
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 0c5fe948ef..b5dfc45736 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -1019,13 +1019,19 @@ static int switch_branches(const struct checkout_opts *opts,
 
 static int git_checkout_config(const char *var, const char *value, void *cb)
 {
+	struct checkout_opts *opts = cb;
+
 	if (!strcmp(var, "checkout.optimizenewbranch")) {
 		checkout_optimize_new_branch = git_config_bool(var, value);
 		return 0;
 	}
 
+	if (!strcmp(var, "checkout.overlaymode")) {
+		opts->overlay_mode = git_config_bool(var, value);
+		return 0;
+	}
+
 	if (!strcmp(var, "diff.ignoresubmodules")) {
-		struct checkout_opts *opts = cb;
 		handle_ignore_submodules_arg(&opts->diff_options, value);
 		return 0;
 	}
diff --git a/t/t2025-checkout-no-overlay.sh b/t/t2025-checkout-no-overlay.sh
index 76330cb5ab..a4912e35cb 100755
--- a/t/t2025-checkout-no-overlay.sh
+++ b/t/t2025-checkout-no-overlay.sh
@@ -44,4 +44,14 @@ test_expect_success '--no-overlay --theirs with D/F conflict deletes file' '
 	test_path_is_missing file1
 '
 
+test_expect_success 'checkout with checkout.overlayMode=false deletes files not in <tree-ish>' '
+	>file &&
+	mkdir dir &&
+	>dir/file1 &&
+	git add file dir/file1 &&
+	git -c checkout.overlayMode=false checkout HEAD -- file &&
+	test_path_is_missing file &&
+	test_path_is_file dir/file1
+'
+
 test_done
-- 
2.20.1.415.g653613c723


^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH v2 7/8] checkout: introduce --{,no-}overlay option
  2018-12-20 13:48   ` [PATCH v2 7/8] checkout: introduce --{,no-}overlay option Thomas Gummerer
@ 2018-12-23  8:05     ` Duy Nguyen
  2018-12-23  9:44       ` Eric Sunshine
  0 siblings, 1 reply; 81+ messages in thread
From: Duy Nguyen @ 2018-12-23  8:05 UTC (permalink / raw)
  To: Thomas Gummerer; +Cc: Git Mailing List, Junio C Hamano, Elijah Newren

On Thu, Dec 20, 2018 at 2:48 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
> index 801de2f764..4ac8c55865 100644
> --- a/Documentation/git-checkout.txt
> +++ b/Documentation/git-checkout.txt
> @@ -260,6 +260,9 @@ the conflicted merge in the specified paths.
>  This means that you can use `git checkout -p` to selectively discard
>  edits from your current working tree. See the ``Interactive Mode''
>  section of linkgit:git-add[1] to learn how to operate the `--patch` mode.
> ++
> +Note that this option uses the no overlay mode by default (see also
> +-`--[no-]overlay`), and currently doesn't support overlay mode.
>
>  --ignore-other-worktrees::
>         `git checkout` refuses when the wanted ref is already checked
> @@ -276,6 +279,13 @@ section of linkgit:git-add[1] to learn how to operate the `--patch` mode.
>         Just like linkgit:git-submodule[1], this will detach the
>         submodules HEAD.
>
> +--[no-]overlay::
> +       In the default overlay mode files `git checkout` never

-ECANTPARSE. Maybe "files" should be removed from this line?

> +       removes files from the index or the working tree.  When
> +       specifying --no-overlay, files that appear in the index and
> +       working tree, but not in <tree-ish> are removed, to make them
> +       match <tree-ish> exactly.
> +
>  <branch>::
>         Branch to checkout; if it refers to a branch (i.e., a name that,
>         when prepended with "refs/heads/", is a valid ref), then that
-- 
Duy

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH v2 7/8] checkout: introduce --{,no-}overlay option
  2018-12-23  8:05     ` Duy Nguyen
@ 2018-12-23  9:44       ` Eric Sunshine
  2019-01-06 18:18         ` Thomas Gummerer
  0 siblings, 1 reply; 81+ messages in thread
From: Eric Sunshine @ 2018-12-23  9:44 UTC (permalink / raw)
  To: Duy Nguyen
  Cc: Thomas Gummerer, Git Mailing List, Junio C Hamano, Elijah Newren

On Sun, Dec 23, 2018 at 3:05 AM Duy Nguyen <pclouds@gmail.com> wrote:
> On Thu, Dec 20, 2018 at 2:48 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> > +--[no-]overlay::
> > +       In the default overlay mode files `git checkout` never
>
> -ECANTPARSE. Maybe "files" should be removed from this line?

Also, add a comma after "mode".

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH v2 8/8] checkout: introduce checkout.overlayMode config
  2018-12-20 13:48   ` [PATCH v2 8/8] checkout: introduce checkout.overlayMode config Thomas Gummerer
@ 2019-01-02 23:39     ` Junio C Hamano
  2019-01-06 18:32       ` Thomas Gummerer
  0 siblings, 1 reply; 81+ messages in thread
From: Junio C Hamano @ 2019-01-02 23:39 UTC (permalink / raw)
  To: Thomas Gummerer; +Cc: git, Nguyễn Thái Ngọc Duy, Elijah Newren

Thomas Gummerer <t.gummerer@gmail.com> writes:

> In the previous patch we introduced a new no-overlay mode for git
> checkout.  Some users (such as the author of this commit) may want to
> have this mode turned on by default as it matches their mental model
> more closely.  Make that possible by introducing a new config option
> to that extend.
>
> Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> ---
>  Documentation/config/checkout.txt |  7 +++++++
>  builtin/checkout.c                |  8 +++++++-
>  t/t2025-checkout-no-overlay.sh    | 10 ++++++++++
>  3 files changed, 24 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/config/checkout.txt b/Documentation/config/checkout.txt
> index c4118fa196..53f917e15e 100644
> --- a/Documentation/config/checkout.txt
> +++ b/Documentation/config/checkout.txt
> @@ -21,3 +21,10 @@ checkout.optimizeNewBranch::
>  	will not update the skip-worktree bit in the index nor add/remove
>  	files in the working directory to reflect the current sparse checkout
>  	settings nor will it show the local changes.
> +
> +checkout.overlayMode::
> +	In the default overlay mode files `git checkout` never
> +	removes files from the index or the working tree.

Technically the above "never removes" is incorrect.

	$ mv COPYING 1 && mkdir COPYING && mv 1 COPYING/COPYING
	$ git add COPYING
	$ git checkout HEAD COPYING

would remove COPYING/1 from the index and from the working tree to
make room.

Because I think that a bit of white lie like what you wrote would
help readers understand the key point of "overlay or not overlay"
better than an overly precise description of the reason why the
removal in the above three-liner case is the right thing to do, I
think the text in the patch is good enough at least for now, but I'd
mention it in case somebody else can think of a better phrasing to
covey the same key point without being technically incorrect.

Thanks.

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH v2 7/8] checkout: introduce --{,no-}overlay option
  2018-12-23  9:44       ` Eric Sunshine
@ 2019-01-06 18:18         ` Thomas Gummerer
  0 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2019-01-06 18:18 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: Duy Nguyen, Git Mailing List, Junio C Hamano, Elijah Newren

On 12/23, Eric Sunshine wrote:
> On Sun, Dec 23, 2018 at 3:05 AM Duy Nguyen <pclouds@gmail.com> wrote:
> > On Thu, Dec 20, 2018 at 2:48 PM Thomas Gummerer <t.gummerer@gmail.com> wrote:
> > > +--[no-]overlay::
> > > +       In the default overlay mode files `git checkout` never
> >
> > -ECANTPARSE. Maybe "files" should be removed from this line?
> 
> Also, add a comma after "mode".

Will do, thanks both.

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH v2 8/8] checkout: introduce checkout.overlayMode config
  2019-01-02 23:39     ` Junio C Hamano
@ 2019-01-06 18:32       ` Thomas Gummerer
  2019-01-07 17:00         ` Junio C Hamano
  0 siblings, 1 reply; 81+ messages in thread
From: Thomas Gummerer @ 2019-01-06 18:32 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Nguyễn Thái Ngọc Duy, Elijah Newren

On 01/02, Junio C Hamano wrote:
> Thomas Gummerer <t.gummerer@gmail.com> writes:
> 
> > In the previous patch we introduced a new no-overlay mode for git
> > checkout.  Some users (such as the author of this commit) may want to
> > have this mode turned on by default as it matches their mental model
> > more closely.  Make that possible by introducing a new config option
> > to that extend.
> >
> > Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> > ---
> >  Documentation/config/checkout.txt |  7 +++++++
> >  builtin/checkout.c                |  8 +++++++-
> >  t/t2025-checkout-no-overlay.sh    | 10 ++++++++++
> >  3 files changed, 24 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/config/checkout.txt b/Documentation/config/checkout.txt
> > index c4118fa196..53f917e15e 100644
> > --- a/Documentation/config/checkout.txt
> > +++ b/Documentation/config/checkout.txt
> > @@ -21,3 +21,10 @@ checkout.optimizeNewBranch::
> >  	will not update the skip-worktree bit in the index nor add/remove
> >  	files in the working directory to reflect the current sparse checkout
> >  	settings nor will it show the local changes.
> > +
> > +checkout.overlayMode::
> > +	In the default overlay mode files `git checkout` never
> > +	removes files from the index or the working tree.
> 
> Technically the above "never removes" is incorrect.
> 
> 	$ mv COPYING 1 && mkdir COPYING && mv 1 COPYING/COPYING
> 	$ git add COPYING
> 	$ git checkout HEAD COPYING
> 
> would remove COPYING/1 from the index and from the working tree to
> make room.

Right, that's a case I didn't think about.

> Because I think that a bit of white lie like what you wrote would
> help readers understand the key point of "overlay or not overlay"
> better than an overly precise description of the reason why the
> removal in the above three-liner case is the right thing to do, I
> think the text in the patch is good enough at least for now, but I'd
> mention it in case somebody else can think of a better phrasing to
> covey the same key point without being technically incorrect.

Maybe it would be enough to say "... `git checkout` never removes
files, that are not in the tree being checked out, from the index or
the working tree"?  It is more technically correct, but dunno making
the sentence harder to read is worth it.

> Thanks.

^ permalink raw reply	[flat|nested] 81+ messages in thread

* Re: [PATCH v2 8/8] checkout: introduce checkout.overlayMode config
  2019-01-06 18:32       ` Thomas Gummerer
@ 2019-01-07 17:00         ` Junio C Hamano
  0 siblings, 0 replies; 81+ messages in thread
From: Junio C Hamano @ 2019-01-07 17:00 UTC (permalink / raw)
  To: Thomas Gummerer; +Cc: git, Nguyễn Thái Ngọc Duy, Elijah Newren

Thomas Gummerer <t.gummerer@gmail.com> writes:

> Maybe it would be enough to say "... `git checkout` never removes
> files, that are not in the tree being checked out, from the index or
> the working tree"?  It is more technically correct, but dunno making
> the sentence harder to read is worth it.

Yeah, I share the same feeling.  Let's say the text in the posted
patch is good enough and move on.

Thanks.

^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH v3 0/8] introduce no-overlay mode in git checkout
  2018-12-20 13:48 ` [PATCH v2 0/8] introduce no-overlay " Thomas Gummerer
                     ` (7 preceding siblings ...)
  2018-12-20 13:48   ` [PATCH v2 8/8] checkout: introduce checkout.overlayMode config Thomas Gummerer
@ 2019-01-08 21:52   ` Thomas Gummerer
  2019-01-08 21:52     ` [PATCH v3 1/8] move worktree tests to t24* Thomas Gummerer
                       ` (7 more replies)
  8 siblings, 8 replies; 81+ messages in thread
From: Thomas Gummerer @ 2019-01-08 21:52 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Eric Sunshine, Thomas Gummerer

Previous rounds are at <20181209200449.16342-1-t.gummerer@gmail.com>
and <20181220134820.21810-1-t.gummerer@gmail.com>.

Thanks Duy, Eric and Junio for comments on the previous round.

This round fixes some inconsistencies and improves the grammar in the
docs.  Range-diff below:

1:  fa450cda7c = 1:  fa450cda7c move worktree tests to t24*
2:  9ada8d3484 = 2:  9ada8d3484 entry: factor out unlink_entry function
3:  41c0ea4047 = 3:  41c0ea4047 entry: support CE_WT_REMOVE flag in checkout_entry
4:  afccb0848d = 4:  afccb0848d read-cache: add invalidate parameter to remove_marked_cache_entries
5:  8a2b5efdad = 5:  8a2b5efdad checkout: clarify comment
6:  c405f20471 = 6:  c405f20471 checkout: factor out mark_cache_entry_for_checkout function
7:  e5b18bcd02 ! 7:  a291dc78fa checkout: introduce --{,no-}overlay option
    @@ -35,7 +35,7 @@
      section of linkgit:git-add[1] to learn how to operate the `--patch` mode.
     ++
     +Note that this option uses the no overlay mode by default (see also
    -+-`--[no-]overlay`), and currently doesn't support overlay mode.
    ++`--[no-]overlay`), and currently doesn't support overlay mode.
      
      --ignore-other-worktrees::
      	`git checkout` refuses when the wanted ref is already checked
    @@ -44,9 +44,9 @@
      	submodules HEAD.
      
     +--[no-]overlay::
    -+	In the default overlay mode files `git checkout` never
    ++	In the default overlay mode, `git checkout` never
     +	removes files from the index or the working tree.  When
    -+	specifying --no-overlay, files that appear in the index and
    ++	specifying `--no-overlay`, files that appear in the index and
     +	working tree, but not in <tree-ish> are removed, to make them
     +	match <tree-ish> exactly.
     +
8:  de24990d57 ! 8:  8d4070f142 checkout: introduce checkout.overlayMode config
    @@ -19,9 +19,9 @@
      	settings nor will it show the local changes.
     +
     +checkout.overlayMode::
    -+	In the default overlay mode files `git checkout` never
    ++	In the default overlay mode, `git checkout` never
     +	removes files from the index or the working tree.  When
    -+	setting checkout.overlayMode to false, files that appear in
    ++	setting `checkout.overlayMode` to false, files that appear in
     +	the index and working tree, but not in <tree-ish> are removed,
     +	to make them match <tree-ish> exactly.

Thomas Gummerer (8):
  move worktree tests to t24*
  entry: factor out unlink_entry function
  entry: support CE_WT_REMOVE flag in checkout_entry
  read-cache: add invalidate parameter to remove_marked_cache_entries
  checkout: clarify comment
  checkout: factor out mark_cache_entry_for_checkout function
  checkout: introduce --{,no-}overlay option
  checkout: introduce checkout.overlayMode config

 Documentation/config/checkout.txt             |   7 +
 Documentation/git-checkout.txt                |  10 ++
 builtin/checkout.c                            | 133 +++++++++++++-----
 cache.h                                       |   7 +-
 entry.c                                       |  26 ++++
 read-cache.c                                  |   8 +-
 split-index.c                                 |   2 +-
 t/t2025-checkout-no-overlay.sh                |  57 ++++++++
 ...-worktree-add.sh => t2400-worktree-add.sh} |   0
 ...ktree-prune.sh => t2401-worktree-prune.sh} |   0
 ...orktree-list.sh => t2402-worktree-list.sh} |   0
 ...orktree-move.sh => t2403-worktree-move.sh} |   0
 ...ree-config.sh => t2404-worktree-config.sh} |   0
 t/t9902-completion.sh                         |   1 +
 unpack-trees.c                                |  21 +--
 15 files changed, 213 insertions(+), 59 deletions(-)
 create mode 100755 t/t2025-checkout-no-overlay.sh
 rename t/{t2025-worktree-add.sh => t2400-worktree-add.sh} (100%)
 rename t/{t2026-worktree-prune.sh => t2401-worktree-prune.sh} (100%)
 rename t/{t2027-worktree-list.sh => t2402-worktree-list.sh} (100%)
 rename t/{t2028-worktree-move.sh => t2403-worktree-move.sh} (100%)
 rename t/{t2029-worktree-config.sh => t2404-worktree-config.sh} (100%)

-- 
2.20.1.153.gd81d796ee0

     

^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH v3 1/8] move worktree tests to t24*
  2019-01-08 21:52   ` [PATCH v3 0/8] introduce no-overlay mode in git checkout Thomas Gummerer
@ 2019-01-08 21:52     ` Thomas Gummerer
  2019-01-08 21:52     ` [PATCH v3 2/8] entry: factor out unlink_entry function Thomas Gummerer
                       ` (6 subsequent siblings)
  7 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2019-01-08 21:52 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Eric Sunshine, Thomas Gummerer

The 'git worktree' command used to be just another mode in 'git
checkout', namely 'git checkout --to'.  When the tests for the latter
were retrofitted for the former, the test name was adjusted, but the
test number was kept, even though the test is testing a different
command now.  t/README states: "Second digit tells the particular
command we are testing.", so 'git worktree' should have a separate
number just for itself.

Move the worktree tests to t24* to adhere to that guideline. We're
going to make use of the free'd up numbers in a subsequent commit.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 t/{t2025-worktree-add.sh => t2400-worktree-add.sh}       | 0
 t/{t2026-worktree-prune.sh => t2401-worktree-prune.sh}   | 0
 t/{t2027-worktree-list.sh => t2402-worktree-list.sh}     | 0
 t/{t2028-worktree-move.sh => t2403-worktree-move.sh}     | 0
 t/{t2029-worktree-config.sh => t2404-worktree-config.sh} | 0
 5 files changed, 0 insertions(+), 0 deletions(-)
 rename t/{t2025-worktree-add.sh => t2400-worktree-add.sh} (100%)
 rename t/{t2026-worktree-prune.sh => t2401-worktree-prune.sh} (100%)
 rename t/{t2027-worktree-list.sh => t2402-worktree-list.sh} (100%)
 rename t/{t2028-worktree-move.sh => t2403-worktree-move.sh} (100%)
 rename t/{t2029-worktree-config.sh => t2404-worktree-config.sh} (100%)

diff --git a/t/t2025-worktree-add.sh b/t/t2400-worktree-add.sh
similarity index 100%
rename from t/t2025-worktree-add.sh
rename to t/t2400-worktree-add.sh
diff --git a/t/t2026-worktree-prune.sh b/t/t2401-worktree-prune.sh
similarity index 100%
rename from t/t2026-worktree-prune.sh
rename to t/t2401-worktree-prune.sh
diff --git a/t/t2027-worktree-list.sh b/t/t2402-worktree-list.sh
similarity index 100%
rename from t/t2027-worktree-list.sh
rename to t/t2402-worktree-list.sh
diff --git a/t/t2028-worktree-move.sh b/t/t2403-worktree-move.sh
similarity index 100%
rename from t/t2028-worktree-move.sh
rename to t/t2403-worktree-move.sh
diff --git a/t/t2029-worktree-config.sh b/t/t2404-worktree-config.sh
similarity index 100%
rename from t/t2029-worktree-config.sh
rename to t/t2404-worktree-config.sh
-- 
2.20.1.153.gd81d796ee0


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH v3 2/8] entry: factor out unlink_entry function
  2019-01-08 21:52   ` [PATCH v3 0/8] introduce no-overlay mode in git checkout Thomas Gummerer
  2019-01-08 21:52     ` [PATCH v3 1/8] move worktree tests to t24* Thomas Gummerer
@ 2019-01-08 21:52     ` Thomas Gummerer
  2019-01-08 21:52     ` [PATCH v3 3/8] entry: support CE_WT_REMOVE flag in checkout_entry Thomas Gummerer
                       ` (5 subsequent siblings)
  7 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2019-01-08 21:52 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Eric Sunshine, Thomas Gummerer

Factor out the 'unlink_entry()' function from unpack-trees.c to
entry.c.  It will be used in other places as well in subsequent
steps.

As it's no longer a static function, also move the documentation to
the header file to make it more discoverable.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 cache.h        |  5 +++++
 entry.c        | 15 +++++++++++++++
 unpack-trees.c | 19 -------------------
 3 files changed, 20 insertions(+), 19 deletions(-)

diff --git a/cache.h b/cache.h
index ca36b44ee0..c1c953e810 100644
--- a/cache.h
+++ b/cache.h
@@ -1542,6 +1542,11 @@ struct checkout {
 extern int checkout_entry(struct cache_entry *ce, const struct checkout *state, char *topath);
 extern void enable_delayed_checkout(struct checkout *state);
 extern int finish_delayed_checkout(struct checkout *state);
+/*
+ * Unlink the last component and schedule the leading directories for
+ * removal, such that empty directories get removed.
+ */
+extern void unlink_entry(const struct cache_entry *ce);
 
 struct cache_def {
 	struct strbuf path;
diff --git a/entry.c b/entry.c
index 0a3c451f5f..b9eef57117 100644
--- a/entry.c
+++ b/entry.c
@@ -508,3 +508,18 @@ int checkout_entry(struct cache_entry *ce,
 	create_directories(path.buf, path.len, state);
 	return write_entry(ce, path.buf, state, 0);
 }
+
+void unlink_entry(const struct cache_entry *ce)
+{
+	const struct submodule *sub = submodule_from_ce(ce);
+	if (sub) {
+		/* state.force is set at the caller. */
+		submodule_move_head(ce->name, "HEAD", NULL,
+				    SUBMODULE_MOVE_HEAD_FORCE);
+	}
+	if (!check_leading_path(ce->name, ce_namelen(ce)))
+		return;
+	if (remove_or_warn(ce->ce_mode, ce->name))
+		return;
+	schedule_dir_for_removal(ce->name, ce_namelen(ce));
+}
diff --git a/unpack-trees.c b/unpack-trees.c
index 7570df481b..e8d1a6ac50 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -300,25 +300,6 @@ static void load_gitmodules_file(struct index_state *index,
 	}
 }
 
-/*
- * Unlink the last component and schedule the leading directories for
- * removal, such that empty directories get removed.
- */
-static void unlink_entry(const struct cache_entry *ce)
-{
-	const struct submodule *sub = submodule_from_ce(ce);
-	if (sub) {
-		/* state.force is set at the caller. */
-		submodule_move_head(ce->name, "HEAD", NULL,
-				    SUBMODULE_MOVE_HEAD_FORCE);
-	}
-	if (!check_leading_path(ce->name, ce_namelen(ce)))
-		return;
-	if (remove_or_warn(ce->ce_mode, ce->name))
-		return;
-	schedule_dir_for_removal(ce->name, ce_namelen(ce));
-}
-
 static struct progress *get_progress(struct unpack_trees_options *o)
 {
 	unsigned cnt = 0, total = 0;
-- 
2.20.1.153.gd81d796ee0


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH v3 3/8] entry: support CE_WT_REMOVE flag in checkout_entry
  2019-01-08 21:52   ` [PATCH v3 0/8] introduce no-overlay mode in git checkout Thomas Gummerer
  2019-01-08 21:52     ` [PATCH v3 1/8] move worktree tests to t24* Thomas Gummerer
  2019-01-08 21:52     ` [PATCH v3 2/8] entry: factor out unlink_entry function Thomas Gummerer
@ 2019-01-08 21:52     ` Thomas Gummerer
  2019-01-08 21:52     ` [PATCH v3 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries Thomas Gummerer
                       ` (4 subsequent siblings)
  7 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2019-01-08 21:52 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Eric Sunshine, Thomas Gummerer

'checkout_entry()' currently only supports creating new entries in the
working tree, but not deleting them.  Add the ability to remove
entries at the same time if the entry is marked with the CE_WT_REMOVE
flag.

Currently this doesn't have any effect, as the CE_WT_REMOVE flag is
only used in unpack-tree, however we will make use of this in a
subsequent step in the series.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 entry.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/entry.c b/entry.c
index b9eef57117..3d3701e7ae 100644
--- a/entry.c
+++ b/entry.c
@@ -441,6 +441,17 @@ int checkout_entry(struct cache_entry *ce,
 	static struct strbuf path = STRBUF_INIT;
 	struct stat st;
 
+	if (ce->ce_flags & CE_WT_REMOVE) {
+		if (topath)
+			/*
+			 * No content and thus no path to create, so we have
+			 * no pathname to return.
+			 */
+			BUG("Can't remove entry to a path");
+		unlink_entry(ce);
+		return 0;
+	}
+
 	if (topath)
 		return write_entry(ce, topath, state, 1);
 
-- 
2.20.1.153.gd81d796ee0


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH v3 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries
  2019-01-08 21:52   ` [PATCH v3 0/8] introduce no-overlay mode in git checkout Thomas Gummerer
                       ` (2 preceding siblings ...)
  2019-01-08 21:52     ` [PATCH v3 3/8] entry: support CE_WT_REMOVE flag in checkout_entry Thomas Gummerer
@ 2019-01-08 21:52     ` Thomas Gummerer
  2019-01-08 21:52     ` [PATCH v3 5/8] checkout: clarify comment Thomas Gummerer
                       ` (3 subsequent siblings)
  7 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2019-01-08 21:52 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Eric Sunshine, Thomas Gummerer

When marking cache entries for removal, and later removing them all at
once using 'remove_marked_cache_entries()', cache entries currently
have to be invalidated manually in the cache tree and in the untracked
cache.

Add an invalidate flag to the function.  With the flag set, the
function will take care of invalidating the path in the cache tree and
in the untracked cache.

Note that the current callsites already do the invalidation properly
in other places, so we're just passing 0 from there to keep the status
quo.

This will be useful in a subsequent commit.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 cache.h        | 2 +-
 read-cache.c   | 8 +++++++-
 split-index.c  | 2 +-
 unpack-trees.c | 2 +-
 4 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/cache.h b/cache.h
index c1c953e810..1deee48f5b 100644
--- a/cache.h
+++ b/cache.h
@@ -751,7 +751,7 @@ extern void rename_index_entry_at(struct index_state *, int pos, const char *new
 /* Remove entry, return true if there are more entries to go. */
 extern int remove_index_entry_at(struct index_state *, int pos);
 
-extern void remove_marked_cache_entries(struct index_state *istate);
+extern void remove_marked_cache_entries(struct index_state *istate, int invalidate);
 extern int remove_file_from_index(struct index_state *, const char *path);
 #define ADD_CACHE_VERBOSE 1
 #define ADD_CACHE_PRETEND 2
diff --git a/read-cache.c b/read-cache.c
index bd45dc3e24..978d43f676 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -590,13 +590,19 @@ int remove_index_entry_at(struct index_state *istate, int pos)
  * CE_REMOVE is set in ce_flags.  This is much more effective than
  * calling remove_index_entry_at() for each entry to be removed.
  */
-void remove_marked_cache_entries(struct index_state *istate)
+void remove_marked_cache_entries(struct index_state *istate, int invalidate)
 {
 	struct cache_entry **ce_array = istate->cache;
 	unsigned int i, j;
 
 	for (i = j = 0; i < istate->cache_nr; i++) {
 		if (ce_array[i]->ce_flags & CE_REMOVE) {
+			if (invalidate) {
+				cache_tree_invalidate_path(istate,
+							   ce_array[i]->name);
+				untracked_cache_remove_from_index(istate,
+								  ce_array[i]->name);
+			}
 			remove_name_hash(istate, ce_array[i]);
 			save_or_free_index_entry(istate, ce_array[i]);
 		}
diff --git a/split-index.c b/split-index.c
index 5820412dc5..8aebc3661b 100644
--- a/split-index.c
+++ b/split-index.c
@@ -162,7 +162,7 @@ void merge_base_index(struct index_state *istate)
 	ewah_each_bit(si->replace_bitmap, replace_entry, istate);
 	ewah_each_bit(si->delete_bitmap, mark_entry_for_delete, istate);
 	if (si->nr_deletions)
-		remove_marked_cache_entries(istate);
+		remove_marked_cache_entries(istate, 0);
 
 	for (i = si->nr_replacements; i < si->saved_cache_nr; i++) {
 		if (!ce_namelen(si->saved_cache[i]))
diff --git a/unpack-trees.c b/unpack-trees.c
index e8d1a6ac50..8e6afa924d 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -392,7 +392,7 @@ static int check_updates(struct unpack_trees_options *o)
 				unlink_entry(ce);
 		}
 	}
-	remove_marked_cache_entries(index);
+	remove_marked_cache_entries(index, 0);
 	remove_scheduled_dirs();
 
 	if (should_update_submodules() && o->update && !o->dry_run)
-- 
2.20.1.153.gd81d796ee0


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH v3 5/8] checkout: clarify comment
  2019-01-08 21:52   ` [PATCH v3 0/8] introduce no-overlay mode in git checkout Thomas Gummerer
                       ` (3 preceding siblings ...)
  2019-01-08 21:52     ` [PATCH v3 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries Thomas Gummerer
@ 2019-01-08 21:52     ` Thomas Gummerer
  2019-01-08 21:52     ` [PATCH v3 6/8] checkout: factor out mark_cache_entry_for_checkout function Thomas Gummerer
                       ` (2 subsequent siblings)
  7 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2019-01-08 21:52 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Eric Sunshine, Thomas Gummerer

The key point for the if statement is that read_tree_some did not
update the entry, because either it doesn't exist in tree-ish or
doesn't match the pathspec.  Clarify that.

Suggested-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 builtin/checkout.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index acdafc6e4c..cb166b2e07 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -304,10 +304,10 @@ static int checkout_paths(const struct checkout_opts *opts,
 			continue;
 		if (opts->source_tree && !(ce->ce_flags & CE_UPDATE))
 			/*
-			 * "git checkout tree-ish -- path", but this entry
-			 * is in the original index; it will not be checked
-			 * out to the working tree and it does not matter
-			 * if pathspec matched this entry.  We will not do
+			 * "git checkout tree-ish -- path" and this entry
+			 * is in the original index, but is not in tree-ish
+			 * or does not match the pathspec; it will not be
+			 * checked out to the working tree.  We will not do
 			 * anything to this entry at all.
 			 */
 			continue;
-- 
2.20.1.153.gd81d796ee0


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH v3 6/8] checkout: factor out mark_cache_entry_for_checkout function
  2019-01-08 21:52   ` [PATCH v3 0/8] introduce no-overlay mode in git checkout Thomas Gummerer
                       ` (4 preceding siblings ...)
  2019-01-08 21:52     ` [PATCH v3 5/8] checkout: clarify comment Thomas Gummerer
@ 2019-01-08 21:52     ` Thomas Gummerer
  2019-01-08 21:52     ` [PATCH v3 7/8] checkout: introduce --{,no-}overlay option Thomas Gummerer
  2019-01-08 21:52     ` [PATCH v3 8/8] checkout: introduce checkout.overlayMode config Thomas Gummerer
  7 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2019-01-08 21:52 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Eric Sunshine, Thomas Gummerer

Factor out the code that marks a cache entry as matched for checkout
into a separate function.  We are going to introduce a new mode in
'git checkout' in a subsequent commit, that is going to have a
slightly different logic.  This would make this code unnecessarily
complex.

Moving that complexity into separate functions will make the code in
the subsequent step easier to follow.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 builtin/checkout.c | 67 +++++++++++++++++++++++++---------------------
 1 file changed, 36 insertions(+), 31 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index cb166b2e07..32c4b7f897 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -247,6 +247,40 @@ static int checkout_merged(int pos, const struct checkout *state)
 	return status;
 }
 
+static void mark_ce_for_checkout(struct cache_entry *ce,
+				 char *ps_matched,
+				 const struct checkout_opts *opts)
+{
+	ce->ce_flags &= ~CE_MATCHED;
+	if (!opts->ignore_skipworktree && ce_skip_worktree(ce))
+		return;
+	if (opts->source_tree && !(ce->ce_flags & CE_UPDATE))
+		/*
+		 * "git checkout tree-ish -- path", but this entry
+		 * is in the original index but is not in tree-ish
+		 * or does not match the pathspec; it will not be
+		 * checked out to the working tree.  We will not do
+		 * anything to this entry at all.
+		 */
+		return;
+	/*
+	 * Either this entry came from the tree-ish we are
+	 * checking the paths out of, or we are checking out
+	 * of the index.
+	 *
+	 * If it comes from the tree-ish, we already know it
+	 * matches the pathspec and could just stamp
+	 * CE_MATCHED to it from update_some(). But we still
+	 * need ps_matched and read_tree_recursive (and
+	 * eventually tree_entry_interesting) cannot fill
+	 * ps_matched yet. Once it can, we can avoid calling
+	 * match_pathspec() for _all_ entries when
+	 * opts->source_tree != NULL.
+	 */
+	if (ce_path_match(&the_index, ce, &opts->pathspec, ps_matched))
+		ce->ce_flags |= CE_MATCHED;
+}
+
 static int checkout_paths(const struct checkout_opts *opts,
 			  const char *revision)
 {
@@ -297,37 +331,8 @@ static int checkout_paths(const struct checkout_opts *opts,
 	 * Make sure all pathspecs participated in locating the paths
 	 * to be checked out.
 	 */
-	for (pos = 0; pos < active_nr; pos++) {
-		struct cache_entry *ce = active_cache[pos];
-		ce->ce_flags &= ~CE_MATCHED;
-		if (!opts->ignore_skipworktree && ce_skip_worktree(ce))
-			continue;
-		if (opts->source_tree && !(ce->ce_flags & CE_UPDATE))
-			/*
-			 * "git checkout tree-ish -- path" and this entry
-			 * is in the original index, but is not in tree-ish
-			 * or does not match the pathspec; it will not be
-			 * checked out to the working tree.  We will not do
-			 * anything to this entry at all.
-			 */
-			continue;
-		/*
-		 * Either this entry came from the tree-ish we are
-		 * checking the paths out of, or we are checking out
-		 * of the index.
-		 *
-		 * If it comes from the tree-ish, we already know it
-		 * matches the pathspec and could just stamp
-		 * CE_MATCHED to it from update_some(). But we still
-		 * need ps_matched and read_tree_recursive (and
-		 * eventually tree_entry_interesting) cannot fill
-		 * ps_matched yet. Once it can, we can avoid calling
-		 * match_pathspec() for _all_ entries when
-		 * opts->source_tree != NULL.
-		 */
-		if (ce_path_match(&the_index, ce, &opts->pathspec, ps_matched))
-			ce->ce_flags |= CE_MATCHED;
-	}
+	for (pos = 0; pos < active_nr; pos++)
+		mark_ce_for_checkout(active_cache[pos], ps_matched, opts);
 
 	if (report_path_error(ps_matched, &opts->pathspec, opts->prefix)) {
 		free(ps_matched);
-- 
2.20.1.153.gd81d796ee0


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH v3 7/8] checkout: introduce --{,no-}overlay option
  2019-01-08 21:52   ` [PATCH v3 0/8] introduce no-overlay mode in git checkout Thomas Gummerer
                       ` (5 preceding siblings ...)
  2019-01-08 21:52     ` [PATCH v3 6/8] checkout: factor out mark_cache_entry_for_checkout function Thomas Gummerer
@ 2019-01-08 21:52     ` Thomas Gummerer
  2019-01-08 21:52     ` [PATCH v3 8/8] checkout: introduce checkout.overlayMode config Thomas Gummerer
  7 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2019-01-08 21:52 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Eric Sunshine, Thomas Gummerer

Currently 'git checkout' is defined as an overlay operation, which
means that if in 'git checkout <tree-ish> -- [<pathspec>]' we have an
entry in the index that matches <pathspec>, but that doesn't exist in
<tree-ish>, that entry will not be removed from the index or the
working tree.

Introduce a new --{,no-}overlay option, which allows using 'git
checkout' in non-overlay mode, thus removing files from the working
tree if they do not exist in <tree-ish> but match <pathspec>.

Note that 'git checkout -p <tree-ish> -- [<pathspec>]' already works
this way, so no changes are needed for the patch mode.  We disallow
'git checkout --overlay -p' to avoid confusing users who would expect
to be able to force overlay mode in 'git checkout -p' this way.

Untracked files are not affected by this change, so 'git checkout
--no-overlay HEAD -- untracked' will not remove untracked from the
working tree.  This is so e.g. 'git checkout --no-overlay HEAD -- dir/'
doesn't delete all untracked files in dir/, but rather just resets the
state of files that are known to git.

Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 Documentation/git-checkout.txt | 10 ++++++
 builtin/checkout.c             | 66 +++++++++++++++++++++++++++++-----
 t/t2025-checkout-no-overlay.sh | 47 ++++++++++++++++++++++++
 t/t9902-completion.sh          |  1 +
 4 files changed, 116 insertions(+), 8 deletions(-)
 create mode 100755 t/t2025-checkout-no-overlay.sh

diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
index 801de2f764..24e52b01e1 100644
--- a/Documentation/git-checkout.txt
+++ b/Documentation/git-checkout.txt
@@ -260,6 +260,9 @@ the conflicted merge in the specified paths.
 This means that you can use `git checkout -p` to selectively discard
 edits from your current working tree. See the ``Interactive Mode''
 section of linkgit:git-add[1] to learn how to operate the `--patch` mode.
++
+Note that this option uses the no overlay mode by default (see also
+`--[no-]overlay`), and currently doesn't support overlay mode.
 
 --ignore-other-worktrees::
 	`git checkout` refuses when the wanted ref is already checked
@@ -276,6 +279,13 @@ section of linkgit:git-add[1] to learn how to operate the `--patch` mode.
 	Just like linkgit:git-submodule[1], this will detach the
 	submodules HEAD.
 
+--[no-]overlay::
+	In the default overlay mode, `git checkout` never
+	removes files from the index or the working tree.  When
+	specifying `--no-overlay`, files that appear in the index and
+	working tree, but not in <tree-ish> are removed, to make them
+	match <tree-ish> exactly.
+
 <branch>::
 	Branch to checkout; if it refers to a branch (i.e., a name that,
 	when prepended with "refs/heads/", is a valid ref), then that
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 32c4b7f897..0c5fe948ef 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -44,6 +44,7 @@ struct checkout_opts {
 	int ignore_skipworktree;
 	int ignore_other_worktrees;
 	int show_progress;
+	int overlay_mode;
 	/*
 	 * If new checkout options are added, skip_merge_working_tree
 	 * should be updated accordingly.
@@ -132,7 +133,8 @@ static int skip_same_name(const struct cache_entry *ce, int pos)
 	return pos;
 }
 
-static int check_stage(int stage, const struct cache_entry *ce, int pos)
+static int check_stage(int stage, const struct cache_entry *ce, int pos,
+		       int overlay_mode)
 {
 	while (pos < active_nr &&
 	       !strcmp(active_cache[pos]->name, ce->name)) {
@@ -140,6 +142,8 @@ static int check_stage(int stage, const struct cache_entry *ce, int pos)
 			return 0;
 		pos++;
 	}
+	if (!overlay_mode)
+		return 0;
 	if (stage == 2)
 		return error(_("path '%s' does not have our version"), ce->name);
 	else
@@ -165,7 +169,7 @@ static int check_stages(unsigned stages, const struct cache_entry *ce, int pos)
 }
 
 static int checkout_stage(int stage, const struct cache_entry *ce, int pos,
-			  const struct checkout *state)
+			  const struct checkout *state, int overlay_mode)
 {
 	while (pos < active_nr &&
 	       !strcmp(active_cache[pos]->name, ce->name)) {
@@ -173,6 +177,10 @@ static int checkout_stage(int stage, const struct cache_entry *ce, int pos,
 			return checkout_entry(active_cache[pos], state, NULL);
 		pos++;
 	}
+	if (!overlay_mode) {
+		unlink_entry(ce);
+		return 0;
+	}
 	if (stage == 2)
 		return error(_("path '%s' does not have our version"), ce->name);
 	else
@@ -247,9 +255,9 @@ static int checkout_merged(int pos, const struct checkout *state)
 	return status;
 }
 
-static void mark_ce_for_checkout(struct cache_entry *ce,
-				 char *ps_matched,
-				 const struct checkout_opts *opts)
+static void mark_ce_for_checkout_overlay(struct cache_entry *ce,
+					 char *ps_matched,
+					 const struct checkout_opts *opts)
 {
 	ce->ce_flags &= ~CE_MATCHED;
 	if (!opts->ignore_skipworktree && ce_skip_worktree(ce))
@@ -281,6 +289,25 @@ static void mark_ce_for_checkout(struct cache_entry *ce,
 		ce->ce_flags |= CE_MATCHED;
 }
 
+static void mark_ce_for_checkout_no_overlay(struct cache_entry *ce,
+					    char *ps_matched,
+					    const struct checkout_opts *opts)
+{
+	ce->ce_flags &= ~CE_MATCHED;
+	if (!opts->ignore_skipworktree && ce_skip_worktree(ce))
+		return;
+	if (ce_path_match(&the_index, ce, &opts->pathspec, ps_matched)) {
+		ce->ce_flags |= CE_MATCHED;
+		if (opts->source_tree && !(ce->ce_flags & CE_UPDATE))
+			/*
+			 * In overlay mode, but the path is not in
+			 * tree-ish, which means we should remove it
+			 * from the index and the working tree.
+			 */
+			ce->ce_flags |= CE_REMOVE | CE_WT_REMOVE;
+	}
+}
+
 static int checkout_paths(const struct checkout_opts *opts,
 			  const char *revision)
 {
@@ -332,7 +359,14 @@ static int checkout_paths(const struct checkout_opts *opts,
 	 * to be checked out.
 	 */
 	for (pos = 0; pos < active_nr; pos++)
-		mark_ce_for_checkout(active_cache[pos], ps_matched, opts);
+		if (opts->overlay_mode)
+			mark_ce_for_checkout_overlay(active_cache[pos],
+						     ps_matched,
+						     opts);
+		else
+			mark_ce_for_checkout_no_overlay(active_cache[pos],
+							ps_matched,
+							opts);
 
 	if (report_path_error(ps_matched, &opts->pathspec, opts->prefix)) {
 		free(ps_matched);
@@ -353,7 +387,7 @@ static int checkout_paths(const struct checkout_opts *opts,
 			if (opts->force) {
 				warning(_("path '%s' is unmerged"), ce->name);
 			} else if (opts->writeout_stage) {
-				errs |= check_stage(opts->writeout_stage, ce, pos);
+				errs |= check_stage(opts->writeout_stage, ce, pos, opts->overlay_mode);
 			} else if (opts->merge) {
 				errs |= check_stages((1<<2) | (1<<3), ce, pos);
 			} else {
@@ -380,12 +414,14 @@ static int checkout_paths(const struct checkout_opts *opts,
 				continue;
 			}
 			if (opts->writeout_stage)
-				errs |= checkout_stage(opts->writeout_stage, ce, pos, &state);
+				errs |= checkout_stage(opts->writeout_stage, ce, pos, &state, opts->overlay_mode);
 			else if (opts->merge)
 				errs |= checkout_merged(pos, &state);
 			pos = skip_same_name(ce, pos) - 1;
 		}
 	}
+	remove_marked_cache_entries(&the_index, 1);
+	remove_scheduled_dirs();
 	errs |= finish_delayed_checkout(&state);
 
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
@@ -547,6 +583,11 @@ static int skip_merge_working_tree(const struct checkout_opts *opts,
 	 * opts->show_progress only impacts output so doesn't require a merge
 	 */
 
+	/*
+	 * opts->overlay_mode cannot be used with switching branches so is
+	 * not tested here
+	 */
+
 	/*
 	 * If we aren't creating a new branch any changes or updates will
 	 * happen in the existing branch.  Since that could only be updating
@@ -1183,6 +1224,10 @@ static int checkout_branch(struct checkout_opts *opts,
 		die(_("'%s' cannot be used with switching branches"),
 		    "--patch");
 
+	if (!opts->overlay_mode)
+		die(_("'%s' cannot be used with switching branches"),
+		    "--no-overlay");
+
 	if (opts->writeout_stage)
 		die(_("'%s' cannot be used with switching branches"),
 		    "--ours/--theirs");
@@ -1271,6 +1316,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 			    "checkout", "control recursive updating of submodules",
 			    PARSE_OPT_OPTARG, option_parse_recurse_submodules_worktree_updater },
 		OPT_BOOL(0, "progress", &opts.show_progress, N_("force progress reporting")),
+		OPT_BOOL(0, "overlay", &opts.overlay_mode, N_("use overlay mode (default)")),
 		OPT_END(),
 	};
 
@@ -1279,6 +1325,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 	opts.overwrite_ignore = 1;
 	opts.prefix = prefix;
 	opts.show_progress = -1;
+	opts.overlay_mode = -1;
 
 	git_config(git_checkout_config, &opts);
 
@@ -1302,6 +1349,9 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 	if ((!!opts.new_branch + !!opts.new_branch_force + !!opts.new_orphan_branch) > 1)
 		die(_("-b, -B and --orphan are mutually exclusive"));
 
+	if (opts.overlay_mode == 1 && opts.patch_mode)
+		die(_("-p and --overlay are mutually exclusive"));
+
 	/*
 	 * From here on, new_branch will contain the branch to be checked out,
 	 * and new_branch_force and new_orphan_branch will tell us which one of
diff --git a/t/t2025-checkout-no-overlay.sh b/t/t2025-checkout-no-overlay.sh
new file mode 100755
index 0000000000..76330cb5ab
--- /dev/null
+++ b/t/t2025-checkout-no-overlay.sh
@@ -0,0 +1,47 @@
+#!/bin/sh
+
+test_description='checkout --no-overlay <tree-ish> -- <pathspec>'
+
+. ./test-lib.sh
+
+test_expect_success 'setup' '
+	git commit --allow-empty -m "initial"
+'
+
+test_expect_success 'checkout --no-overlay deletes files not in <tree-ish>' '
+	>file &&
+	mkdir dir &&
+	>dir/file1 &&
+	git add file dir/file1 &&
+	git checkout --no-overlay HEAD -- file &&
+	test_path_is_missing file &&
+	test_path_is_file dir/file1
+'
+
+test_expect_success 'checkout --no-overlay removing last file from directory' '
+	git checkout --no-overlay HEAD -- dir/file1 &&
+	test_path_is_missing dir
+'
+
+test_expect_success 'checkout -p --overlay is disallowed' '
+	test_must_fail git checkout -p --overlay HEAD 2>actual &&
+	test_i18ngrep "fatal: -p and --overlay are mutually exclusive" actual
+'
+
+test_expect_success '--no-overlay --theirs with D/F conflict deletes file' '
+	test_commit file1 file1 &&
+	test_commit file2 file2 &&
+	git rm --cached file1 &&
+	echo 1234 >file1 &&
+	F1=$(git rev-parse HEAD:file1) &&
+	F2=$(git rev-parse HEAD:file2) &&
+	{
+		echo "100644 $F1 1	file1" &&
+		echo "100644 $F2 2	file1"
+	} | git update-index --index-info &&
+	test_path_is_file file1 &&
+	git checkout --theirs --no-overlay -- file1 &&
+	test_path_is_missing file1
+'
+
+test_done
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index d01ad8eb25..5758fffa0d 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -1436,6 +1436,7 @@ test_expect_success 'double dash "git checkout"' '
 	--progress Z
 	--no-quiet Z
 	--no-... Z
+	--overlay Z
 	EOF
 '
 
-- 
2.20.1.153.gd81d796ee0


^ permalink raw reply	[flat|nested] 81+ messages in thread

* [PATCH v3 8/8] checkout: introduce checkout.overlayMode config
  2019-01-08 21:52   ` [PATCH v3 0/8] introduce no-overlay mode in git checkout Thomas Gummerer
                       ` (6 preceding siblings ...)
  2019-01-08 21:52     ` [PATCH v3 7/8] checkout: introduce --{,no-}overlay option Thomas Gummerer
@ 2019-01-08 21:52     ` Thomas Gummerer
  7 siblings, 0 replies; 81+ messages in thread
From: Thomas Gummerer @ 2019-01-08 21:52 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
	Elijah Newren, Eric Sunshine, Thomas Gummerer

In the previous patch we introduced a new no-overlay mode for git
checkout.  Some users (such as the author of this commit) may want to
have this mode turned on by default as it matches their mental model
more closely.  Make that possible by introducing a new config option
to that extend.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 Documentation/config/checkout.txt |  7 +++++++
 builtin/checkout.c                |  8 +++++++-
 t/t2025-checkout-no-overlay.sh    | 10 ++++++++++
 3 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/Documentation/config/checkout.txt b/Documentation/config/checkout.txt
index c4118fa196..73380a8d86 100644
--- a/Documentation/config/checkout.txt
+++ b/Documentation/config/checkout.txt
@@ -21,3 +21,10 @@ checkout.optimizeNewBranch::
 	will not update the skip-worktree bit in the index nor add/remove
 	files in the working directory to reflect the current sparse checkout
 	settings nor will it show the local changes.
+
+checkout.overlayMode::
+	In the default overlay mode, `git checkout` never
+	removes files from the index or the working tree.  When
+	setting `checkout.overlayMode` to false, files that appear in
+	the index and working tree, but not in <tree-ish> are removed,
+	to make them match <tree-ish> exactly.
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 0c5fe948ef..b5dfc45736 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -1019,13 +1019,19 @@ static int switch_branches(const struct checkout_opts *opts,
 
 static int git_checkout_config(const char *var, const char *value, void *cb)
 {
+	struct checkout_opts *opts = cb;
+
 	if (!strcmp(var, "checkout.optimizenewbranch")) {
 		checkout_optimize_new_branch = git_config_bool(var, value);
 		return 0;
 	}
 
+	if (!strcmp(var, "checkout.overlaymode")) {
+		opts->overlay_mode = git_config_bool(var, value);
+		return 0;
+	}
+
 	if (!strcmp(var, "diff.ignoresubmodules")) {
-		struct checkout_opts *opts = cb;
 		handle_ignore_submodules_arg(&opts->diff_options, value);
 		return 0;
 	}
diff --git a/t/t2025-checkout-no-overlay.sh b/t/t2025-checkout-no-overlay.sh
index 76330cb5ab..a4912e35cb 100755
--- a/t/t2025-checkout-no-overlay.sh
+++ b/t/t2025-checkout-no-overlay.sh
@@ -44,4 +44,14 @@ test_expect_success '--no-overlay --theirs with D/F conflict deletes file' '
 	test_path_is_missing file1
 '
 
+test_expect_success 'checkout with checkout.overlayMode=false deletes files not in <tree-ish>' '
+	>file &&
+	mkdir dir &&
+	>dir/file1 &&
+	git add file dir/file1 &&
+	git -c checkout.overlayMode=false checkout HEAD -- file &&
+	test_path_is_missing file &&
+	test_path_is_file dir/file1
+'
+
 test_done
-- 
2.20.1.153.gd81d796ee0


^ permalink raw reply	[flat|nested] 81+ messages in thread

end of thread, back to index

Thread overview: 81+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-12-09 20:04 [PATCH 0/8] introduce no-overlay and cached mode in git checkout Thomas Gummerer
2018-12-09 20:04 ` [PATCH 1/8] move worktree tests to t24* Thomas Gummerer
2018-12-10  3:48   ` Junio C Hamano
2018-12-10 15:32   ` Duy Nguyen
2018-12-11 21:50     ` Thomas Gummerer
2018-12-12 13:26       ` Eric Sunshine
2018-12-12 17:07         ` Duy Nguyen
2018-12-09 20:04 ` [PATCH 2/8] entry: factor out unlink_entry function Thomas Gummerer
2018-12-10 15:49   ` Duy Nguyen
2018-12-10 17:23     ` Elijah Newren
2018-12-10 17:27       ` Duy Nguyen
2018-12-11  2:23     ` Junio C Hamano
2018-12-20 13:36     ` Thomas Gummerer
2018-12-09 20:04 ` [PATCH 3/8] entry: support CE_WT_REMOVE flag in checkout_entry Thomas Gummerer
2018-12-10 15:58   ` Duy Nguyen
2018-12-11  2:28     ` Junio C Hamano
2018-12-12  6:16       ` Duy Nguyen
2018-12-12  7:36         ` Junio C Hamano
2018-12-10 17:49   ` Elijah Newren
2018-12-11 22:00     ` Thomas Gummerer
2018-12-09 20:04 ` [PATCH 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries Thomas Gummerer
2018-12-10 16:08   ` Duy Nguyen
2018-12-10 18:09     ` Elijah Newren
2018-12-10 18:19       ` Duy Nguyen
2018-12-10 18:25         ` Elijah Newren
2018-12-10 18:33           ` Duy Nguyen
2018-12-10 18:47             ` Elijah Newren
2018-12-11 21:59       ` Thomas Gummerer
2018-12-11  2:42     ` Junio C Hamano
2018-12-09 20:04 ` [PATCH 5/8] checkout: introduce --{,no-}overlay option Thomas Gummerer
2018-12-10 16:42   ` Duy Nguyen
2018-12-11 22:42     ` Thomas Gummerer
2018-12-10 18:19   ` Elijah Newren
2018-12-11  3:07     ` Junio C Hamano
2018-12-11  6:04       ` Elijah Newren
2018-12-11 22:07       ` Thomas Gummerer
2018-12-09 20:04 ` [PATCH 6/8] checkout: add --cached option Thomas Gummerer
2018-12-10 16:49   ` Duy Nguyen
2018-12-11  3:13     ` Junio C Hamano
2018-12-11  6:12       ` Elijah Newren
2018-12-11 19:23         ` Duy Nguyen
2018-12-10 18:42   ` Elijah Newren
2018-12-11 22:18     ` Thomas Gummerer
2018-12-09 20:04 ` [PATCH 7/8] checkout: allow ignoring unmatched pathspec Thomas Gummerer
2018-12-10 16:51   ` Duy Nguyen
2018-12-11 22:23     ` Thomas Gummerer
2018-12-10 20:25   ` Elijah Newren
2018-12-11 22:36     ` Thomas Gummerer
2018-12-09 20:04 ` [PATCH 8/8] stash: use git checkout --no-overlay Thomas Gummerer
2018-12-10 20:26   ` Elijah Newren
2018-12-10  3:47 ` [PATCH 0/8] introduce no-overlay and cached mode in git checkout Junio C Hamano
2018-12-20  8:43   ` Thomas Gummerer
2018-12-10 17:06 ` Duy Nguyen
2018-12-10 17:18 ` Elijah Newren
2018-12-10 18:37   ` Duy Nguyen
2018-12-11 22:52   ` Thomas Gummerer
2018-12-12  7:28     ` Junio C Hamano
2018-12-20 13:48 ` [PATCH v2 0/8] introduce no-overlay " Thomas Gummerer
2018-12-20 13:48   ` [PATCH v2 1/8] move worktree tests to t24* Thomas Gummerer
2018-12-20 13:48   ` [PATCH v2 2/8] entry: factor out unlink_entry function Thomas Gummerer
2018-12-20 13:48   ` [PATCH v2 3/8] entry: support CE_WT_REMOVE flag in checkout_entry Thomas Gummerer
2018-12-20 13:48   ` [PATCH v2 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries Thomas Gummerer
2018-12-20 13:48   ` [PATCH v2 5/8] checkout: clarify comment Thomas Gummerer
2018-12-20 13:48   ` [PATCH v2 6/8] checkout: factor out mark_cache_entry_for_checkout function Thomas Gummerer
2018-12-20 13:48   ` [PATCH v2 7/8] checkout: introduce --{,no-}overlay option Thomas Gummerer
2018-12-23  8:05     ` Duy Nguyen
2018-12-23  9:44       ` Eric Sunshine
2019-01-06 18:18         ` Thomas Gummerer
2018-12-20 13:48   ` [PATCH v2 8/8] checkout: introduce checkout.overlayMode config Thomas Gummerer
2019-01-02 23:39     ` Junio C Hamano
2019-01-06 18:32       ` Thomas Gummerer
2019-01-07 17:00         ` Junio C Hamano
2019-01-08 21:52   ` [PATCH v3 0/8] introduce no-overlay mode in git checkout Thomas Gummerer
2019-01-08 21:52     ` [PATCH v3 1/8] move worktree tests to t24* Thomas Gummerer
2019-01-08 21:52     ` [PATCH v3 2/8] entry: factor out unlink_entry function Thomas Gummerer
2019-01-08 21:52     ` [PATCH v3 3/8] entry: support CE_WT_REMOVE flag in checkout_entry Thomas Gummerer
2019-01-08 21:52     ` [PATCH v3 4/8] read-cache: add invalidate parameter to remove_marked_cache_entries Thomas Gummerer
2019-01-08 21:52     ` [PATCH v3 5/8] checkout: clarify comment Thomas Gummerer
2019-01-08 21:52     ` [PATCH v3 6/8] checkout: factor out mark_cache_entry_for_checkout function Thomas Gummerer
2019-01-08 21:52     ` [PATCH v3 7/8] checkout: introduce --{,no-}overlay option Thomas Gummerer
2019-01-08 21:52     ` [PATCH v3 8/8] checkout: introduce checkout.overlayMode config Thomas Gummerer

git@vger.kernel.org mailing list mirror (one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/
       or Tor2web: https://www.tor2web.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox