[PATCH 0/8] Avoid removing the current working directory, even if it becomes empty

[PATCH 0/8] Avoid removing the current working directory, even if it becomes empty

[Elijah Newren via GitGitGadget]

Traditionally, if folks run git commands such as checkout or rebase from a
subdirectory, that git command could remove their current working directory
and result in subsequent git and non-git commands either getting confused or
printing messages that confuse the user (e.g. "fatal: Unable to read current
working directory: No such file or directory"). We already refuse to remove
directories that have untracked files within them[1], preferring to show an
error; with this series, we tweak that rule a bit to also refuse to remove
the current working directory even if it has no untracked files within it.

Peff endorsed the idea behind this series at [2] and even pointed out a
corner case that he suggested (and I agree) would probably be more
problematic to attempt to address than to leave be.

This series has a minor textual conflict with so/stash-staged (in next),
because both series add a new parameter to the do_push_stash() function. The
correct resolution is simple -- just to take both new parameters. However,
let me know if you'd rather I rebased onto so/stash-staged.

(Sorry for the delay; I know I promised this months ago in response to [3],
but it was blocked by some preliminary series and then by my absence during
a migration at $DAYJOB).

[1] well, with a few exceptions; see
https://lore.kernel.org/git/pull.1036.v3.git.1632760428.gitgitgadget@gmail.com/
[2] https://lore.kernel.org/git/YS8eEtwQvF7TaLCb@coredump.intra.peff.net/
[3]
https://lore.kernel.org/git/c0557284-8f82-76cc-8c47-0b1bc9f2ce5a@rawbw.com/

Elijah Newren (8):
  t2501: add various tests for removing the current working directory
  repository, setup: introduce the_cwd
  unpack-trees: refuse to remove the current working directory
  unpack-trees: add special cwd handling
  symlinks: do not include current working directory in dir removal
  clean: do not attempt to remove current working directory
  stash: do not attempt to remove current working directory
  dir: avoid removing the current working directory

 builtin/clean.c      |  29 +++--
 builtin/stash.c      |  13 ++-
 dir.c                |  11 +-
 repository.c         |   1 +
 repository.h         |   1 +
 setup.c              |   2 +
 symlinks.c           |  12 +-
 t/t2501-cwd-empty.sh | 255 +++++++++++++++++++++++++++++++++++++++++++
 unpack-trees.c       |  28 ++++-
 unpack-trees.h       |   1 +
 10 files changed, 329 insertions(+), 24 deletions(-)
 create mode 100755 t/t2501-cwd-empty.sh


base-commit: 88d915a634b449147855041d44875322de2b286d
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1140%2Fnewren%2Fcwd_removal-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1140/newren/cwd_removal-v1
Pull-Request: https://github.com/git/git/pull/1140
-- 
gitgitgadget

[PATCH 1/8] t2501: add various tests for removing the current working directory

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Numerous commands will remove empty working directories, especially if
they are in the way of placing needed files.  That is normally fine, but
removing the current working directory can cause confusion for the user
when they run subsequent commands.  Add some tests checking for such
problems.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh | 255 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 255 insertions(+)
 create mode 100755 t/t2501-cwd-empty.sh

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
new file mode 100755
index 00000000000..5dfb456a691
--- /dev/null
+++ b/t/t2501-cwd-empty.sh
@@ -0,0 +1,255 @@
+#!/bin/sh
+
+test_description='Test handling of the current working directory becoming empty'
+
+. ./test-lib.sh
+
+test_expect_success setup '
+	test_commit init &&
+	mkdir subdir &&
+	test_commit subdir/file &&
+
+	git branch fd_conflict &&
+
+	mkdir -p foo/bar &&
+	test_commit foo/bar/baz &&
+
+	git revert HEAD &&
+	git tag reverted &&
+
+	git checkout fd_conflict &&
+	git rm subdir/file.t &&
+	echo not-a-directory >dirORfile &&
+	git add dirORfile &&
+	git commit -m dirORfile
+'
+
+test_expect_failure 'checkout does not clean cwd incidentally' '
+	git checkout foo/bar/baz &&
+	test_path_is_dir foo/bar &&
+
+	(
+		cd foo &&
+		git checkout init &&
+		cd ..
+	) &&
+	test_path_is_missing foo/bar/baz &&
+	test_path_is_missing foo/bar &&
+	test_path_is_dir foo
+'
+
+test_expect_failure 'checkout fails if cwd needs to be removed' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir dirORfile &&
+	(
+		cd dirORfile &&
+
+		test_must_fail git checkout fd_conflict 2>../error &&
+		grep "Refusing to remove the current working directory" ../error
+	) &&
+
+	test_path_is_dir dirORfile
+'
+
+test_expect_failure 'reset --hard does not clean cwd incidentally' '
+	git checkout foo/bar/baz &&
+	test_path_is_dir foo/bar &&
+
+	(
+		cd foo &&
+		git reset --hard init &&
+		cd ..
+	) &&
+	test_path_is_missing foo/bar/baz &&
+	test_path_is_missing foo/bar &&
+	test_path_is_dir foo
+'
+
+test_expect_failure 'reset --hard fails if cwd needs to be removed' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir dirORfile &&
+	(
+		cd dirORfile &&
+
+		test_must_fail git reset --hard fd_conflict 2>../error &&
+		grep "Refusing to remove.*the current working directory" ../error
+	) &&
+
+	test_path_is_dir dirORfile
+'
+
+test_expect_failure 'merge does not remove cwd incidentally' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	(
+		cd subdir &&
+		git merge fd_conflict
+	) &&
+
+	test_path_is_missing subdir/file.t &&
+	test_path_is_dir subdir
+'
+
+test_expect_failure 'merge fails if cwd needs to be removed' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir dirORfile &&
+	(
+		cd dirORfile &&
+		test_must_fail git merge fd_conflict 2>../error &&
+		grep "Refusing to remove the current working directory" ../error
+	) &&
+
+	test_path_is_dir dirORfile
+'
+
+test_expect_failure 'cherry-pick does not remove cwd incidentally' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	(
+		cd subdir &&
+		git cherry-pick fd_conflict
+	) &&
+
+	test_path_is_missing subdir/file.t &&
+	test_path_is_dir subdir
+'
+
+test_expect_failure 'cherry-pick fails if cwd needs to be removed' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir dirORfile &&
+	(
+		cd dirORfile &&
+		test_must_fail git cherry-pick fd_conflict 2>../error &&
+		grep "Refusing to remove the current working directory" ../error
+	) &&
+
+	test_path_is_dir dirORfile
+'
+
+test_expect_failure 'rebase does not remove cwd incidentally' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	(
+		cd subdir &&
+		git rebase foo/bar/baz fd_conflict
+	) &&
+
+	test_path_is_missing subdir/file.t &&
+	test_path_is_dir subdir
+'
+
+test_expect_failure 'rebase fails if cwd needs to be removed' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir dirORfile &&
+	(
+		cd dirORfile &&
+		test_must_fail git rebase foo/bar/baz fd_conflict 2>../error &&
+		grep "Refusing to remove the current working directory" ../error
+	) &&
+
+	test_path_is_dir dirORfile
+'
+
+test_expect_failure 'revert does not remove cwd incidentally' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	(
+		cd subdir &&
+		git revert subdir/file
+	) &&
+
+	test_path_is_missing subdir/file.t &&
+	test_path_is_dir subdir
+'
+
+test_expect_failure 'revert fails if cwd needs to be removed' '
+	git checkout fd_conflict &&
+	git revert HEAD &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir dirORfile &&
+	(
+		cd dirORfile &&
+		test_must_fail git revert HEAD 2>../error &&
+		grep "Refusing to remove the current working directory" ../error
+	) &&
+
+	test_path_is_dir dirORfile
+'
+
+test_expect_failure 'rm does not remove cwd incidentally' '
+	test_when_finished "git reset --hard" &&
+	git checkout foo/bar/baz &&
+
+	(
+		cd foo &&
+		git rm bar/baz.t
+	) &&
+
+	test_path_is_missing foo/bar/baz &&
+	test_path_is_missing foo/bar &&
+	test_path_is_dir foo
+'
+
+test_expect_failure 'apply does not remove cwd incidentally' '
+	test_when_finished "git reset --hard" &&
+	git checkout foo/bar/baz &&
+
+	(
+		cd subdir &&
+		git diff subdir/file init | git apply
+	) &&
+
+	test_path_is_missing subdir/file.t &&
+	test_path_is_dir subdir
+'
+
+test_expect_failure 'clean does not remove cwd incidentally' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir empty &&
+	mkdir untracked &&
+	>untracked/random &&
+	(
+		cd untracked &&
+		git clean -fd -e warnings :/ >../warnings &&
+		grep "Refusing to remove current working directory" ../warnings
+	) &&
+
+	test_path_is_missing empty &&
+	test_path_is_missing untracked/random &&
+	test_path_is_dir untracked
+'
+
+test_expect_failure 'stash does not remove cwd incidentally' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir untracked &&
+	>untracked/random &&
+	(
+		cd untracked &&
+		git stash --include-untracked &&
+		git status
+	) &&
+
+	test_path_is_missing untracked/random &&
+	test_path_is_dir untracked
+'
+
+test_done
-- 
gitgitgadget

[PATCH 2/8] repository, setup: introduce the_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Removing the current working directory causes all subsequent git
commands (and likely a number of non-git commands) run from that
directory to get confused and fail with a message about being unable to
read the current working directory.  That confuses end users,
particularly since the command they get the error from is not the one
that caused the problem; the problem came from the side-effect of some
previous command.

We would like to avoid removing the current working directory; towards
this end, introduce a new the_cwd variable that tracks the current
working directory.  Subsequent commits will make use of this new
variable.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 repository.c | 1 +
 repository.h | 1 +
 setup.c      | 2 ++
 3 files changed, 4 insertions(+)

diff --git a/repository.c b/repository.c
index c5b90ba93ea..69a106c553c 100644
--- a/repository.c
+++ b/repository.c
@@ -17,6 +17,7 @@
 static struct repository the_repo;
 struct repository *the_repository;
 struct index_state the_index;
+char *the_cwd;
 
 void initialize_the_repository(void)
 {
diff --git a/repository.h b/repository.h
index a057653981c..45de85d18ef 100644
--- a/repository.h
+++ b/repository.h
@@ -147,6 +147,7 @@ struct repository {
 };
 
 extern struct repository *the_repository;
+extern char *the_cwd;
 
 /*
  * Define a custom repository layout. Any field can be NULL, which
diff --git a/setup.c b/setup.c
index 347d7181ae9..4466fa55af3 100644
--- a/setup.c
+++ b/setup.c
@@ -887,6 +887,7 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 		set_git_dir(gitdirenv, 1);
 		if (chdir(worktree))
 			die_errno(_("cannot chdir to '%s'"), worktree);
+		the_cwd = xstrdup(cwd->buf + offset);
 		strbuf_addch(cwd, '/');
 		free(gitfile);
 		return cwd->buf + offset;
@@ -940,6 +941,7 @@ static const char *setup_discovered_git_dir(const char *gitdir,
 	/* Make "offset" point past the '/' (already the case for root dirs) */
 	if (offset != offset_1st_component(cwd->buf))
 		offset++;
+	the_cwd = xstrdup(cwd->buf + offset);
 	/* Add a '/' at the end */
 	strbuf_addch(cwd, '/');
 	return cwd->buf + offset;
-- 
gitgitgadget

[PATCH 3/8] unpack-trees: refuse to remove the current working directory

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

In the past, when a directory needs to be removed to make room for a
file, we have always errored out when that directory contains any
untracked (but not ignored) files.  Add an extra condition on that: also
error out if the directory is the current working directory.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh | 10 +++++-----
 unpack-trees.c       | 16 ++++++++++++----
 unpack-trees.h       |  1 +
 3 files changed, 18 insertions(+), 9 deletions(-)

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 5dfb456a691..212676d71c3 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -38,7 +38,7 @@ test_expect_failure 'checkout does not clean cwd incidentally' '
 	test_path_is_dir foo
 '
 
-test_expect_failure 'checkout fails if cwd needs to be removed' '
+test_expect_success 'checkout fails if cwd needs to be removed' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -95,7 +95,7 @@ test_expect_failure 'merge does not remove cwd incidentally' '
 	test_path_is_dir subdir
 '
 
-test_expect_failure 'merge fails if cwd needs to be removed' '
+test_expect_success 'merge fails if cwd needs to be removed' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -122,7 +122,7 @@ test_expect_failure 'cherry-pick does not remove cwd incidentally' '
 	test_path_is_dir subdir
 '
 
-test_expect_failure 'cherry-pick fails if cwd needs to be removed' '
+test_expect_success 'cherry-pick fails if cwd needs to be removed' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -149,7 +149,7 @@ test_expect_failure 'rebase does not remove cwd incidentally' '
 	test_path_is_dir subdir
 '
 
-test_expect_failure 'rebase fails if cwd needs to be removed' '
+test_expect_success 'rebase fails if cwd needs to be removed' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -176,7 +176,7 @@ test_expect_failure 'revert does not remove cwd incidentally' '
 	test_path_is_dir subdir
 '
 
-test_expect_failure 'revert fails if cwd needs to be removed' '
+test_expect_success 'revert fails if cwd needs to be removed' '
 	git checkout fd_conflict &&
 	git revert HEAD &&
 	test_when_finished "git clean -fdx" &&
diff --git a/unpack-trees.c b/unpack-trees.c
index 89ca95ce90b..8879b8d6c8c 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -36,6 +36,9 @@ static const char *unpack_plumbing_errors[NB_UNPACK_TREES_WARNING_TYPES] = {
 	/* ERROR_NOT_UPTODATE_DIR */
 	"Updating '%s' would lose untracked files in it",
 
+	/* ERROR_CWD_IN_THE_WAY */
+	"Refusing to remove '%s' since it is the current working directory.",
+
 	/* ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN */
 	"Untracked working tree file '%s' would be overwritten by merge.",
 
@@ -131,6 +134,9 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 	msgs[ERROR_NOT_UPTODATE_DIR] =
 		_("Updating the following directories would lose untracked files in them:\n%s");
 
+	msgs[ERROR_CWD_IN_THE_WAY] =
+		_("Refusing to remove the current working directory:\n%s");
+
 	if (!strcmp(cmd, "checkout"))
 		msg = advice_enabled(ADVICE_COMMIT_BEFORE_MERGE)
 		      ? _("The following untracked working tree files would be removed by checkout:\n%%s"
@@ -2146,10 +2152,7 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,
 		cnt++;
 	}
 
-	/*
-	 * Then we need to make sure that we do not lose a locally
-	 * present file that is not ignored.
-	 */
+	/* Do not lose a locally present file that is not ignored. */
 	pathbuf = xstrfmt("%.*s/", namelen, ce->name);
 
 	memset(&d, 0, sizeof(d));
@@ -2160,6 +2163,11 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,
 	free(pathbuf);
 	if (i)
 		return add_rejected_path(o, ERROR_NOT_UPTODATE_DIR, ce->name);
+
+	/* Do not lose the current working directory. */
+	if (the_cwd && !strcmp(the_cwd, ce->name))
+		return add_rejected_path(o, ERROR_CWD_IN_THE_WAY, ce->name);
+
 	return cnt;
 }
 
diff --git a/unpack-trees.h b/unpack-trees.h
index 71ffb7eeb0c..efb9edfbb27 100644
--- a/unpack-trees.h
+++ b/unpack-trees.h
@@ -19,6 +19,7 @@ enum unpack_trees_error_types {
 	ERROR_WOULD_OVERWRITE = 0,
 	ERROR_NOT_UPTODATE_FILE,
 	ERROR_NOT_UPTODATE_DIR,
+	ERROR_CWD_IN_THE_WAY,
 	ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN,
 	ERROR_WOULD_LOSE_UNTRACKED_REMOVED,
 	ERROR_BIND_OVERLAP,
-- 
gitgitgadget

[PATCH 4/8] unpack-trees: add special cwd handling

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

When running commands such as `git reset --hard` from a subdirectory, if
that subdirectory is in the way of adding needed files, bail with an
error message.

Note that this change looks kind of like it duplicates the new lines of
code from the previous commit in verify_clean_subdirectory().  However,
when we are preserving untracked files, we would rather any error
messages about untracked files being in the way take precedence over
error messages about a subdirectory that happens to be the current
working directory being in the way.  But in the
UNPACK_RESET_OVERWRITE_UNTRACKED case, there is no untracked checking to
be done, so we simply add a special case near the top of
verify_absent_1.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh |  2 +-
 unpack-trees.c       | 12 ++++++++++--
 2 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 212676d71c3..526d8ec2ee3 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -67,7 +67,7 @@ test_expect_failure 'reset --hard does not clean cwd incidentally' '
 	test_path_is_dir foo
 '
 
-test_expect_failure 'reset --hard fails if cwd needs to be removed' '
+test_expect_success 'reset --hard fails if cwd needs to be removed' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
diff --git a/unpack-trees.c b/unpack-trees.c
index 8879b8d6c8c..d1ce4f4a35b 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -2260,10 +2260,18 @@ static int verify_absent_1(const struct cache_entry *ce,
 	int len;
 	struct stat st;
 
-	if (o->index_only || !o->update ||
-	    o->reset == UNPACK_RESET_OVERWRITE_UNTRACKED)
+	if (o->index_only || !o->update)
 		return 0;
 
+	if (o->reset == UNPACK_RESET_OVERWRITE_UNTRACKED) {
+		/* Avoid nuking cwd... */
+		if (the_cwd && !strcmp(the_cwd, ce->name))
+			return add_rejected_path(o, ERROR_CWD_IN_THE_WAY,
+						 ce->name);
+		/* ...but nuke anything else. */
+		return 0;
+	}
+
 	len = check_leading_path(ce->name, ce_namelen(ce), 0);
 	if (!len)
 		return 0;
-- 
gitgitgadget

[PATCH 5/8] symlinks: do not include current working directory in dir removal

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

symlinks has a pair of schedule_dir_for_removal() and
remove_scheduled_dirs() functions that ensure that directories made
empty by removing other files also themselves get removed.  However, we
want to exclude the current working directory and leave it around so
that subsequent git commands (and non-git commands) that the user runs
afterwards don't cause the user to get confused.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 symlinks.c           | 12 +++++++++++-
 t/t2501-cwd-empty.sh | 12 ++++++------
 2 files changed, 17 insertions(+), 7 deletions(-)

diff --git a/symlinks.c b/symlinks.c
index 5232d02020c..84622bedcde 100644
--- a/symlinks.c
+++ b/symlinks.c
@@ -275,11 +275,18 @@ static int threaded_has_dirs_only_path(struct cache_def *cache, const char *name
 
 static struct strbuf removal = STRBUF_INIT;
 
+static int cant_remove(char *dirname)
+{
+	if (the_cwd && !strcmp(dirname, the_cwd))
+		return 1;
+	return rmdir(dirname);
+}
+
 static void do_remove_scheduled_dirs(int new_len)
 {
 	while (removal.len > new_len) {
 		removal.buf[removal.len] = '\0';
-		if (rmdir(removal.buf))
+		if (cant_remove(removal.buf))
 			break;
 		do {
 			removal.len--;
@@ -293,6 +300,9 @@ void schedule_dir_for_removal(const char *name, int len)
 {
 	int match_len, last_slash, i, previous_slash;
 
+	if (the_cwd && !strcmp(name, the_cwd))
+		return;	/* Do not remove the current working directory */
+
 	match_len = last_slash = i =
 		longest_path_match(name, len, removal.buf, removal.len,
 				   &previous_slash);
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 526d8ec2ee3..b92e1a9bb16 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -24,7 +24,7 @@ test_expect_success setup '
 	git commit -m dirORfile
 '
 
-test_expect_failure 'checkout does not clean cwd incidentally' '
+test_expect_success 'checkout does not clean cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_path_is_dir foo/bar &&
 
@@ -53,7 +53,7 @@ test_expect_success 'checkout fails if cwd needs to be removed' '
 	test_path_is_dir dirORfile
 '
 
-test_expect_failure 'reset --hard does not clean cwd incidentally' '
+test_expect_success 'reset --hard does not clean cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_path_is_dir foo/bar &&
 
@@ -82,7 +82,7 @@ test_expect_success 'reset --hard fails if cwd needs to be removed' '
 	test_path_is_dir dirORfile
 '
 
-test_expect_failure 'merge does not remove cwd incidentally' '
+test_expect_success 'merge does not remove cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -109,7 +109,7 @@ test_expect_success 'merge fails if cwd needs to be removed' '
 	test_path_is_dir dirORfile
 '
 
-test_expect_failure 'cherry-pick does not remove cwd incidentally' '
+test_expect_success 'cherry-pick does not remove cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -136,7 +136,7 @@ test_expect_success 'cherry-pick fails if cwd needs to be removed' '
 	test_path_is_dir dirORfile
 '
 
-test_expect_failure 'rebase does not remove cwd incidentally' '
+test_expect_success 'rebase does not remove cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -163,7 +163,7 @@ test_expect_success 'rebase fails if cwd needs to be removed' '
 	test_path_is_dir dirORfile
 '
 
-test_expect_failure 'revert does not remove cwd incidentally' '
+test_expect_success 'revert does not remove cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
-- 
gitgitgadget

[PATCH 6/8] clean: do not attempt to remove current working directory

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/clean.c      | 29 ++++++++++++++++++++---------
 t/t2501-cwd-empty.sh |  2 +-
 2 files changed, 21 insertions(+), 10 deletions(-)

diff --git a/builtin/clean.c b/builtin/clean.c
index 98a2860409b..17d54be57b7 100644
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@ -36,6 +36,8 @@ static const char *msg_skip_git_dir = N_("Skipping repository %s\n");
 static const char *msg_would_skip_git_dir = N_("Would skip repository %s\n");
 static const char *msg_warn_remove_failed = N_("failed to remove %s");
 static const char *msg_warn_lstat_failed = N_("could not lstat %s\n");
+static const char *msg_skip_cwd = N_("Refusing to remove current working directory\n");
+static const char *msg_would_skip_cwd = N_("Would refuse to remove current working directory\n");
 
 enum color_clean {
 	CLEAN_COLOR_RESET = 0,
@@ -231,16 +233,25 @@ static int remove_dirs(struct strbuf *path, const char *prefix, int force_flag,
 	strbuf_setlen(path, original_len);
 
 	if (*dir_gone) {
-		res = dry_run ? 0 : rmdir(path->buf);
-		if (!res)
-			*dir_gone = 1;
-		else {
-			int saved_errno = errno;
-			quote_path(path->buf, prefix, &quoted, 0);
-			errno = saved_errno;
-			warning_errno(_(msg_warn_remove_failed), quoted.buf);
+		int prefixlen = prefix ? strlen(prefix) : 0;
+		if (prefix &&
+		    path->len == prefixlen + 2 &&
+		    !strncmp(path->buf, prefix, prefixlen) &&
+		    !strcmp(path->buf + prefixlen, "./")) {
+			printf("%s", dry_run ? _(msg_would_skip_cwd) : _(msg_skip_cwd));
 			*dir_gone = 0;
-			ret = 1;
+		} else {
+			res = dry_run ? 0 : rmdir(path->buf);
+			if (!res)
+				*dir_gone = 1;
+			else {
+				int saved_errno = errno;
+				quote_path(path->buf, prefix, &quoted, 0);
+				errno = saved_errno;
+				warning_errno(_(msg_warn_remove_failed), quoted.buf);
+				*dir_gone = 0;
+				ret = 1;
+			}
 		}
 	}
 
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index b92e1a9bb16..20e1b6adede 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -218,7 +218,7 @@ test_expect_failure 'apply does not remove cwd incidentally' '
 	test_path_is_dir subdir
 '
 
-test_expect_failure 'clean does not remove cwd incidentally' '
+test_expect_success 'clean does not remove cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
-- 
gitgitgadget

[PATCH 7/8] stash: do not attempt to remove current working directory

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/stash.c      | 13 ++++++++-----
 t/t2501-cwd-empty.sh |  2 +-
 2 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/builtin/stash.c b/builtin/stash.c
index a0ccc8654df..556287cdb56 100644
--- a/builtin/stash.c
+++ b/builtin/stash.c
@@ -1404,7 +1404,8 @@ static int create_stash(int argc, const char **argv, const char *prefix)
 	return ret;
 }
 
-static int do_push_stash(const struct pathspec *ps, const char *stash_msg, int quiet,
+static int do_push_stash(const struct pathspec *ps, const char *stash_msg,
+			 const char *prefix, int quiet,
 			 int keep_index, int patch_mode, int include_untracked)
 {
 	int ret = 0;
@@ -1485,8 +1486,10 @@ static int do_push_stash(const struct pathspec *ps, const char *stash_msg, int q
 			struct child_process cp = CHILD_PROCESS_INIT;
 
 			cp.git_cmd = 1;
+			if (prefix)
+				strvec_pushl(&cp.args, "-C", prefix, NULL);
 			strvec_pushl(&cp.args, "clean", "--force",
-				     "--quiet", "-d", NULL);
+				     "--quiet", "-d", ":/", NULL);
 			if (include_untracked == INCLUDE_ALL_FILES)
 				strvec_push(&cp.args, "-x");
 			if (run_command(&cp)) {
@@ -1656,8 +1659,8 @@ static int push_stash(int argc, const char **argv, const char *prefix,
 		die(_("--pathspec-file-nul requires --pathspec-from-file"));
 	}
 
-	return do_push_stash(&ps, stash_msg, quiet, keep_index, patch_mode,
-			     include_untracked);
+	return do_push_stash(&ps, stash_msg, prefix, quiet, keep_index,
+			     patch_mode, include_untracked);
 }
 
 static int save_stash(int argc, const char **argv, const char *prefix)
@@ -1693,7 +1696,7 @@ static int save_stash(int argc, const char **argv, const char *prefix)
 		stash_msg = strbuf_join_argv(&stash_msg_buf, argc, argv, ' ');
 
 	memset(&ps, 0, sizeof(ps));
-	ret = do_push_stash(&ps, stash_msg, quiet, keep_index,
+	ret = do_push_stash(&ps, stash_msg, prefix, quiet, keep_index,
 			    patch_mode, include_untracked);
 
 	strbuf_release(&stash_msg_buf);
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 20e1b6adede..ff4e7cd89fa 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -236,7 +236,7 @@ test_expect_success 'clean does not remove cwd incidentally' '
 	test_path_is_dir untracked
 '
 
-test_expect_failure 'stash does not remove cwd incidentally' '
+test_expect_success 'stash does not remove cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
-- 
gitgitgadget

[PATCH 8/8] dir: avoid removing the current working directory

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

dir has a convenient remove_path() helper that will both remove a file
in a directory and remove its containing directory if it becomes empty
as a result of the removal, recursing all the way up.  However, we do
not want the current working directory to be removed, even if it becomes
empty.

dir also has a remove_dir_recursively() function which appears to mostly
be used to remove metadata directories or temporary directories or
submodules or worktrees.  I am not sure if it needs to be protected
against removing the current working directory, but did so for good
measure.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 dir.c                | 11 ++++++++---
 t/t2501-cwd-empty.sh |  4 ++--
 2 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/dir.c b/dir.c
index 94489298f4c..15e7639a158 100644
--- a/dir.c
+++ b/dir.c
@@ -3259,9 +3259,12 @@ static int remove_dir_recurse(struct strbuf *path, int flag, int *kept_up)
 	closedir(dir);
 
 	strbuf_setlen(path, original_len);
-	if (!ret && !keep_toplevel && !kept_down)
-		ret = (!rmdir(path->buf) || errno == ENOENT) ? 0 : -1;
-	else if (kept_up)
+	if (!ret && !keep_toplevel && !kept_down) {
+		if (the_cwd && !strcmp(the_cwd, path->buf))
+			ret = -1; /* Do not remove current working directory */
+		else
+			ret = (!rmdir(path->buf) || errno == ENOENT) ? 0 : -1;
+	} else if (kept_up)
 		/*
 		 * report the uplevel that it is not an error that we
 		 * did not rmdir() our directory.
@@ -3327,6 +3330,8 @@ int remove_path(const char *name)
 		slash = dirs + (slash - name);
 		do {
 			*slash = '\0';
+			if (the_cwd && !strcmp(the_cwd, dirs))
+				break;
 		} while (rmdir(dirs) == 0 && (slash = strrchr(dirs, '/')));
 		free(dirs);
 	}
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index ff4e7cd89fa..4362e7b15e5 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -191,7 +191,7 @@ test_expect_success 'revert fails if cwd needs to be removed' '
 	test_path_is_dir dirORfile
 '
 
-test_expect_failure 'rm does not remove cwd incidentally' '
+test_expect_success 'rm does not remove cwd incidentally' '
 	test_when_finished "git reset --hard" &&
 	git checkout foo/bar/baz &&
 
@@ -205,7 +205,7 @@ test_expect_failure 'rm does not remove cwd incidentally' '
 	test_path_is_dir foo
 '
 
-test_expect_failure 'apply does not remove cwd incidentally' '
+test_expect_success 'apply does not remove cwd incidentally' '
 	test_when_finished "git reset --hard" &&
 	git checkout foo/bar/baz &&
 
-- 
gitgitgadget

Re: [PATCH 2/8] repository, setup: introduce the_cwd

[Junio C Hamano]

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Elijah Newren <newren@gmail.com>
>
> Removing the current working directory causes all subsequent git
> commands (and likely a number of non-git commands) run from that
> directory to get confused and fail with a message about being unable to
> read the current working directory.  That confuses end users,
> particularly since the command they get the error from is not the one
> that caused the problem; the problem came from the side-effect of some
> previous command.
>
> We would like to avoid removing the current working directory; towards
> this end, introduce a new the_cwd variable that tracks the current
> working directory.  Subsequent commits will make use of this new
> variable.

Maybe a stupid question, but how is this different from doing getcwd()
and storing it away to the_cwd, or adding a check to see if the
directory we are about to rmdir() is the cwd, next to the existing
check that we do to see if that directory has some untracked files?

I am wondering how we are going to make sure that the_cwd is always
set to, and maintained to be, the correct value, even in the future
when these code paths change.  I also wonder if it might be safer to
learn what the value of cwd is very near the place where it will
become needed (i.e. the callsites of such rmdir() of a directory
inside working tree), instead of caching.

> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  repository.c | 1 +
>  repository.h | 1 +
>  setup.c      | 2 ++
>  3 files changed, 4 insertions(+)
>
> diff --git a/repository.c b/repository.c
> index c5b90ba93ea..69a106c553c 100644
> --- a/repository.c
> +++ b/repository.c
> @@ -17,6 +17,7 @@
>  static struct repository the_repo;
>  struct repository *the_repository;
>  struct index_state the_index;
> +char *the_cwd;
>  
>  void initialize_the_repository(void)
>  {
> diff --git a/repository.h b/repository.h
> index a057653981c..45de85d18ef 100644
> --- a/repository.h
> +++ b/repository.h
> @@ -147,6 +147,7 @@ struct repository {
>  };
>  
>  extern struct repository *the_repository;
> +extern char *the_cwd;
>  
>  /*
>   * Define a custom repository layout. Any field can be NULL, which
> diff --git a/setup.c b/setup.c
> index 347d7181ae9..4466fa55af3 100644
> --- a/setup.c
> +++ b/setup.c
> @@ -887,6 +887,7 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
>  		set_git_dir(gitdirenv, 1);
>  		if (chdir(worktree))
>  			die_errno(_("cannot chdir to '%s'"), worktree);
> +		the_cwd = xstrdup(cwd->buf + offset);
>  		strbuf_addch(cwd, '/');
>  		free(gitfile);
>  		return cwd->buf + offset;
> @@ -940,6 +941,7 @@ static const char *setup_discovered_git_dir(const char *gitdir,
>  	/* Make "offset" point past the '/' (already the case for root dirs) */
>  	if (offset != offset_1st_component(cwd->buf))
>  		offset++;
> +	the_cwd = xstrdup(cwd->buf + offset);
>  	/* Add a '/' at the end */
>  	strbuf_addch(cwd, '/');
>  	return cwd->buf + offset;

Re: [PATCH 0/8] Avoid removing the current working directory, even if it becomes empty

[Junio C Hamano]

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> Traditionally, if folks run git commands such as checkout or rebase from a
> subdirectory, that git command could remove their current working directory
> and result in subsequent git and non-git commands either getting confused or
> printing messages that confuse the user (e.g. "fatal: Unable to read current
> working directory: No such file or directory"). We already refuse to remove
> directories that have untracked files within them[1], preferring to show an
> error; with this series, we tweak that rule a bit to also refuse to remove
> the current working directory even if it has no untracked files within it.

The goal is roughly that we do not allow rmdir() of a directory at
"prefix" in the working tree, or any parent directory of it, for
functions that uses RUN_SETUP?  By the time we attempt to rmdir(),
most likely we'd chdir()'ed up to the top of the working tree
ourselves, so we need to remember where the original cwd was when we
started, and protecting the original cwd would mean that we protect
the directory in which the process that spawned us, like the user's
interactive shell, is still sitting, which makes sense.

Re: [PATCH 2/8] repository, setup: introduce the_cwd

[René Scharfe]

Am 21.11.21 um 01:46 schrieb Elijah Newren via GitGitGadget:
> From: Elijah Newren <newren@gmail.com>
>
> Removing the current working directory causes all subsequent git
> commands (and likely a number of non-git commands) run from that
> directory to get confused and fail with a message about being unable to
> read the current working directory.  That confuses end users,
> particularly since the command they get the error from is not the one
> that caused the problem; the problem came from the side-effect of some
> previous command.
>
> We would like to avoid removing the current working directory;

A worthy goal.

> towards
> this end, introduce a new the_cwd variable that tracks the current
> working directory.  Subsequent commits will make use of this new
> variable.

Why make it a global variable instead of getting the working directory
in the places that try to delete directories?  (Honest question, not a
suggestion.)

>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  repository.c | 1 +
>  repository.h | 1 +
>  setup.c      | 2 ++
>  3 files changed, 4 insertions(+)
>
> diff --git a/repository.c b/repository.c
> index c5b90ba93ea..69a106c553c 100644
> --- a/repository.c
> +++ b/repository.c
> @@ -17,6 +17,7 @@
>  static struct repository the_repo;
>  struct repository *the_repository;
>  struct index_state the_index;
> +char *the_cwd;
>
>  void initialize_the_repository(void)
>  {
> diff --git a/repository.h b/repository.h
> index a057653981c..45de85d18ef 100644
> --- a/repository.h
> +++ b/repository.h
> @@ -147,6 +147,7 @@ struct repository {
>  };
>
>  extern struct repository *the_repository;
> +extern char *the_cwd;
>
>  /*
>   * Define a custom repository layout. Any field can be NULL, which
> diff --git a/setup.c b/setup.c
> index 347d7181ae9..4466fa55af3 100644
> --- a/setup.c
> +++ b/setup.c
> @@ -887,6 +887,7 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
>  		set_git_dir(gitdirenv, 1);
>  		if (chdir(worktree))
>  			die_errno(_("cannot chdir to '%s'"), worktree);
> +		the_cwd = xstrdup(cwd->buf + offset);
>  		strbuf_addch(cwd, '/');
>  		free(gitfile);
>  		return cwd->buf + offset;
> @@ -940,6 +941,7 @@ static const char *setup_discovered_git_dir(const char *gitdir,
>  	/* Make "offset" point past the '/' (already the case for root dirs) */
>  	if (offset != offset_1st_component(cwd->buf))
>  		offset++;
> +	the_cwd = xstrdup(cwd->buf + offset);
>  	/* Add a '/' at the end */
>  	strbuf_addch(cwd, '/');
>  	return cwd->buf + offset;
>

Re: [PATCH 5/8] symlinks: do not include current working directory in dir removal

[René Scharfe]

Am 21.11.21 um 01:46 schrieb Elijah Newren via GitGitGadget:
> From: Elijah Newren <newren@gmail.com>
>
> symlinks has a pair of schedule_dir_for_removal() and
> remove_scheduled_dirs() functions that ensure that directories made
> empty by removing other files also themselves get removed.  However, we
> want to exclude the current working directory and leave it around so
> that subsequent git commands (and non-git commands) that the user runs
> afterwards don't cause the user to get confused.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  symlinks.c           | 12 +++++++++++-
>  t/t2501-cwd-empty.sh | 12 ++++++------
>  2 files changed, 17 insertions(+), 7 deletions(-)
>
> diff --git a/symlinks.c b/symlinks.c
> index 5232d02020c..84622bedcde 100644
> --- a/symlinks.c
> +++ b/symlinks.c
> @@ -275,11 +275,18 @@ static int threaded_has_dirs_only_path(struct cache_def *cache, const char *name
>
>  static struct strbuf removal = STRBUF_INIT;
>
> +static int cant_remove(char *dirname)
> +{
> +	if (the_cwd && !strcmp(dirname, the_cwd))

Initializing the_cwd to an empty string would allow removing the NULL check
everywhere.

Is strcmp() sufficient or do we need fspathcmp() in these kinds of checks?
Do we need to worry about normalizing directory separators?

> +		return 1;
> +	return rmdir(dirname);
> +}

I wouldn't expect a function of that name to actually try to remove
the directory.  Or with that body to require a non-const dirname.
It's used only once, perhaps inline it?

> +
>  static void do_remove_scheduled_dirs(int new_len)
>  {
>  	while (removal.len > new_len) {
>  		removal.buf[removal.len] = '\0';
> -		if (rmdir(removal.buf))
> +		if (cant_remove(removal.buf))
>  			break;
>  		do {
>  			removal.len--;
> @@ -293,6 +300,9 @@ void schedule_dir_for_removal(const char *name, int len)
>  {
>  	int match_len, last_slash, i, previous_slash;
>
> +	if (the_cwd && !strcmp(name, the_cwd))
> +		return;	/* Do not remove the current working directory */
> +
>  	match_len = last_slash = i =
>  		longest_path_match(name, len, removal.buf, removal.len,
>  				   &previous_slash);

Re: [PATCH 6/8] clean: do not attempt to remove current working directory

[Ævar Arnfjörð Bjarmason]

On Sun, Nov 21 2021, Elijah Newren via GitGitGadget wrote:

> +		int prefixlen = prefix ? strlen(prefix) : 0;

nit: s/int/size_t/

Re: [PATCH 1/8] t2501: add various tests for removing the current working directory

[Ævar Arnfjörð Bjarmason]

On Sun, Nov 21 2021, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>

> +test_expect_failure 'checkout fails if cwd needs to be removed' '
> +	git checkout foo/bar/baz &&
> +	test_when_finished "git clean -fdx" &&
> +
> +	mkdir dirORfile &&
> +	(
> +		cd dirORfile &&
> +
> +		test_must_fail git checkout fd_conflict 2>../error &&
> +		grep "Refusing to remove the current working directory" ../error
> +	) &&
> +
> +	test_path_is_dir dirORfile


I'd find this & the rest of this series much easier to understand if we
started out by positively asserting the current behavior here, and
didn't test_cmp/grep for erro r messages that don't exist anymore.

It would also help to show how exactly operations that currently "work"
behave, e.g. if you git checkout a revision within "t/" which is a
subdir, and it turns into "t" the file etc.

I'm also generally knee-jerk reactionary to test_expect_failure for its
fragility[1], but in this case more because it makes explaining this
step-by-step harder & not as obvious.

1. https://lore.kernel.org/git/87tuhmk19c.fsf@evledraar.gmail.com/

Re: [PATCH 2/8] repository, setup: introduce the_cwd

[Elijah Newren]

On Sun, Nov 21, 2021 at 12:00 AM Junio C Hamano <gitster@pobox.com> wrote:
>
> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > Removing the current working directory causes all subsequent git
> > commands (and likely a number of non-git commands) run from that
> > directory to get confused and fail with a message about being unable to
> > read the current working directory.  That confuses end users,
> > particularly since the command they get the error from is not the one
> > that caused the problem; the problem came from the side-effect of some
> > previous command.
> >
> > We would like to avoid removing the current working directory; towards
> > this end, introduce a new the_cwd variable that tracks the current
> > working directory.  Subsequent commits will make use of this new
> > variable.
>
> Maybe a stupid question, but how is this different from doing getcwd()
> and storing it away to the_cwd

The exact output of getcwd() is an absolute path, whereas it's easier
to perform later checks if we have a path that is relative to the
toplevel working directory.  Also, setup.c already calls getcwd() (via
strbuf_getcwd()) and then massages it, and I'd have to do the exact
same type of massaging.  So, instead of calling getcwd() again and
re-massaging its output into the format I want, I just reuse the
existing call of getcwd() and its massaged output.

> or adding a check to see if the
> directory we are about to rmdir() is the cwd, next to the existing
> check that we do to see if that directory has some untracked files?
>
> I am wondering how we are going to make sure that the_cwd is always
> set to, and maintained to be, the correct value, even in the future
> when these code paths change.  I also wonder if it might be safer to
> learn what the value of cwd is very near the place where it will
> become needed (i.e. the callsites of such rmdir() of a directory
> inside working tree), instead of caching.

I need to clean up my wording a bit, to clarify this.  Doing as you
suggest here would protect the wrong thing.  git has long done an
automatic chdir() at startup to the toplevel working tree, whereas
what we want to protect is the original current working directory as
of the time git started.  The original current working directory is
likely also still the current working directory of the parent process
that spawned us, and if that parent process is a shell, any subsequent
commands executed from there can be somewhat confusing when its
current working directory is removed.

> > Signed-off-by: Elijah Newren <newren@gmail.com>
> > ---
> >  repository.c | 1 +
> >  repository.h | 1 +
> >  setup.c      | 2 ++
> >  3 files changed, 4 insertions(+)
> >
> > diff --git a/repository.c b/repository.c
> > index c5b90ba93ea..69a106c553c 100644
> > --- a/repository.c
> > +++ b/repository.c
> > @@ -17,6 +17,7 @@
> >  static struct repository the_repo;
> >  struct repository *the_repository;
> >  struct index_state the_index;
> > +char *the_cwd;
> >
> >  void initialize_the_repository(void)
> >  {
> > diff --git a/repository.h b/repository.h
> > index a057653981c..45de85d18ef 100644
> > --- a/repository.h
> > +++ b/repository.h
> > @@ -147,6 +147,7 @@ struct repository {
> >  };
> >
> >  extern struct repository *the_repository;
> > +extern char *the_cwd;
> >
> >  /*
> >   * Define a custom repository layout. Any field can be NULL, which
> > diff --git a/setup.c b/setup.c
> > index 347d7181ae9..4466fa55af3 100644
> > --- a/setup.c
> > +++ b/setup.c
> > @@ -887,6 +887,7 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
> >               set_git_dir(gitdirenv, 1);
> >               if (chdir(worktree))
> >                       die_errno(_("cannot chdir to '%s'"), worktree);
> > +             the_cwd = xstrdup(cwd->buf + offset);
> >               strbuf_addch(cwd, '/');
> >               free(gitfile);
> >               return cwd->buf + offset;
> > @@ -940,6 +941,7 @@ static const char *setup_discovered_git_dir(const char *gitdir,
> >       /* Make "offset" point past the '/' (already the case for root dirs) */
> >       if (offset != offset_1st_component(cwd->buf))
> >               offset++;
> > +     the_cwd = xstrdup(cwd->buf + offset);
> >       /* Add a '/' at the end */
> >       strbuf_addch(cwd, '/');
> >       return cwd->buf + offset;

Re: [PATCH 2/8] repository, setup: introduce the_cwd

[Elijah Newren]

On Sun, Nov 21, 2021 at 12:56 AM René Scharfe <l.s.r@web.de> wrote:
>
> Am 21.11.21 um 01:46 schrieb Elijah Newren via GitGitGadget:
> > From: Elijah Newren <newren@gmail.com>
> >
> > Removing the current working directory causes all subsequent git
> > commands (and likely a number of non-git commands) run from that
> > directory to get confused and fail with a message about being unable to
> > read the current working directory.  That confuses end users,
> > particularly since the command they get the error from is not the one
> > that caused the problem; the problem came from the side-effect of some
> > previous command.
> >
> > We would like to avoid removing the current working directory;
>
> A worthy goal.
>
> > towards
> > this end, introduce a new the_cwd variable that tracks the current
> > working directory.  Subsequent commits will make use of this new
> > variable.
>
> Why make it a global variable instead of getting the working directory
> in the places that try to delete directories?  (Honest question, not a
> suggestion.)

As I mentioned in my response to Junio, I need to be clearer that what
I want to protect is the current working directory as of the startup
of the git process, as a proxy for the current working directory of
the parent process, so that subsequent commands started from the
parent process don't get confused.

As such, we don't want to get the working directory again later,
because that'd give us the wrong thing.  Also, setup.c has some nice
massaging of getcwd() from an absolute path into a relative path that
is much more convenient for us to do our later comparisons, and we'd
rather not do those additional tweaks with every check.  Since the
relative path from the project root to the _original_ current working
directory is a single value, a global seemed to make sense for saving
it.  Perhaps I should tweak it to be const as well.

>
> >
> > Signed-off-by: Elijah Newren <newren@gmail.com>
> > ---
> >  repository.c | 1 +
> >  repository.h | 1 +
> >  setup.c      | 2 ++
> >  3 files changed, 4 insertions(+)
> >
> > diff --git a/repository.c b/repository.c
> > index c5b90ba93ea..69a106c553c 100644
> > --- a/repository.c
> > +++ b/repository.c
> > @@ -17,6 +17,7 @@
> >  static struct repository the_repo;
> >  struct repository *the_repository;
> >  struct index_state the_index;
> > +char *the_cwd;
> >
> >  void initialize_the_repository(void)
> >  {
> > diff --git a/repository.h b/repository.h
> > index a057653981c..45de85d18ef 100644
> > --- a/repository.h
> > +++ b/repository.h
> > @@ -147,6 +147,7 @@ struct repository {
> >  };
> >
> >  extern struct repository *the_repository;
> > +extern char *the_cwd;
> >
> >  /*
> >   * Define a custom repository layout. Any field can be NULL, which
> > diff --git a/setup.c b/setup.c
> > index 347d7181ae9..4466fa55af3 100644
> > --- a/setup.c
> > +++ b/setup.c
> > @@ -887,6 +887,7 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
> >               set_git_dir(gitdirenv, 1);
> >               if (chdir(worktree))
> >                       die_errno(_("cannot chdir to '%s'"), worktree);
> > +             the_cwd = xstrdup(cwd->buf + offset);
> >               strbuf_addch(cwd, '/');
> >               free(gitfile);
> >               return cwd->buf + offset;
> > @@ -940,6 +941,7 @@ static const char *setup_discovered_git_dir(const char *gitdir,
> >       /* Make "offset" point past the '/' (already the case for root dirs) */
> >       if (offset != offset_1st_component(cwd->buf))
> >               offset++;
> > +     the_cwd = xstrdup(cwd->buf + offset);
> >       /* Add a '/' at the end */
> >       strbuf_addch(cwd, '/');
> >       return cwd->buf + offset;
> >
>

Re: [PATCH 5/8] symlinks: do not include current working directory in dir removal

[Elijah Newren]

On Sun, Nov 21, 2021 at 12:56 AM René Scharfe <l.s.r@web.de> wrote:
>
> Am 21.11.21 um 01:46 schrieb Elijah Newren via GitGitGadget:
> > From: Elijah Newren <newren@gmail.com>
> >
> > symlinks has a pair of schedule_dir_for_removal() and
> > remove_scheduled_dirs() functions that ensure that directories made
> > empty by removing other files also themselves get removed.  However, we
> > want to exclude the current working directory and leave it around so
> > that subsequent git commands (and non-git commands) that the user runs
> > afterwards don't cause the user to get confused.
> >
> > Signed-off-by: Elijah Newren <newren@gmail.com>
> > ---
> >  symlinks.c           | 12 +++++++++++-
> >  t/t2501-cwd-empty.sh | 12 ++++++------
> >  2 files changed, 17 insertions(+), 7 deletions(-)
> >
> > diff --git a/symlinks.c b/symlinks.c
> > index 5232d02020c..84622bedcde 100644
> > --- a/symlinks.c
> > +++ b/symlinks.c
> > @@ -275,11 +275,18 @@ static int threaded_has_dirs_only_path(struct cache_def *cache, const char *name
> >
> >  static struct strbuf removal = STRBUF_INIT;
> >
> > +static int cant_remove(char *dirname)
> > +{
> > +     if (the_cwd && !strcmp(dirname, the_cwd))
>
> Initializing the_cwd to an empty string would allow removing the NULL check
> everywhere.

I actually went and made this change, because although it'd add
protection to the toplevel directory (the_cwd is relative to the
toplevel), we usually want it protected.  However, dir.c's
remove_dir_recursively() can be used to remove the toplevel directory
as well and has an explicit flag for that, so I need to be able to
distinguish between uninitialized and explicitly set to the toplevel
directory.

> Is strcmp() sufficient or do we need fspathcmp() in these kinds of checks?
> Do we need to worry about normalizing directory separators?

Good catch, I should normalize the_cwd when I create it; I was
essentially creating it to match "prefix", but it appears that isn't
pre-emptively normalized, and instead later callers normalize any
combination of prefix plus another path.

> > +             return 1;
> > +     return rmdir(dirname);
> > +}
>
> I wouldn't expect a function of that name to actually try to remove
> the directory.  Or with that body to require a non-const dirname.
> It's used only once, perhaps inline it?

Sure.

> > +
> >  static void do_remove_scheduled_dirs(int new_len)
> >  {
> >       while (removal.len > new_len) {
> >               removal.buf[removal.len] = '\0';
> > -             if (rmdir(removal.buf))
> > +             if (cant_remove(removal.buf))
> >                       break;
> >               do {
> >                       removal.len--;
> > @@ -293,6 +300,9 @@ void schedule_dir_for_removal(const char *name, int len)
> >  {
> >       int match_len, last_slash, i, previous_slash;
> >
> > +     if (the_cwd && !strcmp(name, the_cwd))
> > +             return; /* Do not remove the current working directory */
> > +
> >       match_len = last_slash = i =
> >               longest_path_match(name, len, removal.buf, removal.len,
> >                                  &previous_slash);

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Glen Choo]

> @@ -3259,9 +3259,12 @@ static int remove_dir_recurse(struct strbuf *path, int flag, int *kept_up)
>  	closedir(dir);
>  
>  	strbuf_setlen(path, original_len);
> -	if (!ret && !keep_toplevel && !kept_down)
> -		ret = (!rmdir(path->buf) || errno == ENOENT) ? 0 : -1;
> -	else if (kept_up)
> +	if (!ret && !keep_toplevel && !kept_down) {
> +		if (the_cwd && !strcmp(the_cwd, path->buf))
> +			ret = -1; /* Do not remove current working directory */
> +		else
> +			ret = (!rmdir(path->buf) || errno == ENOENT) ? 0 : -1;
> +	} else if (kept_up)
>  		/*
>  		 * report the uplevel that it is not an error that we
>  		 * did not rmdir() our directory.
> @@ -3327,6 +3330,8 @@ int remove_path(const char *name)
>  		slash = dirs + (slash - name);
>  		do {
>  			*slash = '\0';
> +			if (the_cwd && !strcmp(the_cwd, dirs))
> +				break;
>  		} while (rmdir(dirs) == 0 && (slash = strrchr(dirs, '/')));
>  		free(dirs);
>  	}

I don't think it's appropriate to implement user-facing concern (don't remove 
cwd because this will confuse users) in library functions like remove_path().
remove_path() has other callers e.g. midx.c, and possible future callers e.g. 
we're working on adding a command to delete corrupted commit-graphs and this 
library function would be extremely handy.

It seems more appropriate to check the_cwd from builtin/add.c and builtin/rm.c 
instead.

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Elijah Newren]

On Mon, Nov 22, 2021 at 4:40 PM Glen Choo <chooglen@google.com> wrote:
>
> > @@ -3259,9 +3259,12 @@ static int remove_dir_recurse(struct strbuf *path, int flag, int *kept_up)
> >       closedir(dir);
> >
> >       strbuf_setlen(path, original_len);
> > -     if (!ret && !keep_toplevel && !kept_down)
> > -             ret = (!rmdir(path->buf) || errno == ENOENT) ? 0 : -1;
> > -     else if (kept_up)
> > +     if (!ret && !keep_toplevel && !kept_down) {
> > +             if (the_cwd && !strcmp(the_cwd, path->buf))
> > +                     ret = -1; /* Do not remove current working directory */
> > +             else
> > +                     ret = (!rmdir(path->buf) || errno == ENOENT) ? 0 : -1;
> > +     } else if (kept_up)
> >               /*
> >                * report the uplevel that it is not an error that we
> >                * did not rmdir() our directory.
> > @@ -3327,6 +3330,8 @@ int remove_path(const char *name)
> >               slash = dirs + (slash - name);
> >               do {
> >                       *slash = '\0';
> > +                     if (the_cwd && !strcmp(the_cwd, dirs))
> > +                             break;
> >               } while (rmdir(dirs) == 0 && (slash = strrchr(dirs, '/')));
> >               free(dirs);
> >       }
>
> I don't think it's appropriate to implement user-facing concern (don't remove
> cwd because this will confuse users) in library functions like remove_path().
> remove_path() has other callers e.g. midx.c, and possible future callers e.g.
> we're working on adding a command to delete corrupted commit-graphs and this
> library function would be extremely handy.

I think we'd want this code change for those cases too.  Said another
way, why wouldn't these callers want to avoid deleting the original
current working directory of the git process (which is likely still
the current working directory of the parent process)?  Deleting that
directory causes problems regardless of whether it's a user-facing
command (rm, merge, stash, etc.) or something more internal (midx or
commit-graphs stuff being called by gc) that is doing the deleting.

Putting it in this helper function means we protect all current and
future callers without developers having to remember which
"remove_path()" variant they need and why.

> It seems more appropriate to check the_cwd from builtin/add.c and builtin/rm.c
> instead.

Not sure how you determined that those two paths are affected or that
those are the only two.

Re: [PATCH 6/8] clean: do not attempt to remove current working directory

[Elijah Newren]

On Sun, Nov 21, 2021 at 9:51 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
>
> On Sun, Nov 21 2021, Elijah Newren via GitGitGadget wrote:
>
> > +             int prefixlen = prefix ? strlen(prefix) : 0;
>
> nit: s/int/size_t/

Thanks; will fix.

Re: [PATCH 1/8] t2501: add various tests for removing the current working directory

[Elijah Newren]

On Sun, Nov 21, 2021 at 9:59 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Sun, Nov 21 2021, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
>
> > +test_expect_failure 'checkout fails if cwd needs to be removed' '
> > +     git checkout foo/bar/baz &&
> > +     test_when_finished "git clean -fdx" &&
> > +
> > +     mkdir dirORfile &&
> > +     (
> > +             cd dirORfile &&
> > +
> > +             test_must_fail git checkout fd_conflict 2>../error &&
> > +             grep "Refusing to remove the current working directory" ../error
> > +     ) &&
> > +
> > +     test_path_is_dir dirORfile
>
>
> I'd find this & the rest of this series much easier to understand if we
> started out by positively asserting the current behavior here, and
> didn't test_cmp/grep for erro r messages that don't exist anymore.

Yeah, this is my fault for my bad commit message.  I stated I was
adding tests checking for the problems of interest, making it sound
like I was testing existing behavior, but I should have stated I was
adding tests with the behavior we'd prefer to have (i.e. basically a
test-driven-development) setup.

Also, there really wouldn't need to be so many tests for describing
the existing behavior.  It's basically just `git
$OPERATION_THAT_REMOVES_CWD_AS_SIDE_EFFECT` followed by nearly any
other git command will cause the second and later commands to fail
with:

```
shell-init: error retrieving current directory: getcwd: cannot access
parent directories: No such file or directory
fatal: Unable to read current working directory: No such file or directory
```

However, we do need a lot of tests for corrected behavior, because
there are so many different codepaths we can follow which will lead to
deletion of the current working directory.

Re: [PATCH 1/8] t2501: add various tests for removing the current working directory

[Ævar Arnfjörð Bjarmason]

On Mon, Nov 22 2021, Elijah Newren wrote:

> On Sun, Nov 21, 2021 at 9:59 AM Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>>
>> On Sun, Nov 21 2021, Elijah Newren via GitGitGadget wrote:
>>
>> > From: Elijah Newren <newren@gmail.com>
>>
>> > +test_expect_failure 'checkout fails if cwd needs to be removed' '
>> > +     git checkout foo/bar/baz &&
>> > +     test_when_finished "git clean -fdx" &&
>> > +
>> > +     mkdir dirORfile &&
>> > +     (
>> > +             cd dirORfile &&
>> > +
>> > +             test_must_fail git checkout fd_conflict 2>../error &&
>> > +             grep "Refusing to remove the current working directory" ../error
>> > +     ) &&
>> > +
>> > +     test_path_is_dir dirORfile
>>
>>
>> I'd find this & the rest of this series much easier to understand if we
>> started out by positively asserting the current behavior here, and
>> didn't test_cmp/grep for erro r messages that don't exist anymore.
>
> Yeah, this is my fault for my bad commit message.  I stated I was
> adding tests checking for the problems of interest, making it sound
> like I was testing existing behavior, but I should have stated I was
> adding tests with the behavior we'd prefer to have (i.e. basically a
> test-driven-development) setup.
>
> Also, there really wouldn't need to be so many tests for describing
> the existing behavior.  It's basically just `git
> $OPERATION_THAT_REMOVES_CWD_AS_SIDE_EFFECT` followed by nearly any
> other git command will cause the second and later commands to fail
> with:
>
> ```
> shell-init: error retrieving current directory: getcwd: cannot access
> parent directories: No such file or directory
> fatal: Unable to read current working directory: No such file or directory
> ```
>
> However, we do need a lot of tests for corrected behavior, because
> there are so many different codepaths we can follow which will lead to
> deletion of the current working directory.

Currently if I do e.g.:

    git checkout master
    git clean -dxf
    cd perl
    git checkout v0.99
    cd ../
    git clean -dxfn

Nothing breaks and I don't end up with an empty perl/ to remove. With
these patches we'd either die on the "checkout" (I think) keep the
"perl" and have an empty perl/ to report in the "git clean -dxfn" at the
end (I'm not sure which, I forgot and haven't re-read this series just
now).

I think changing it anyway might be justifiable, but changing the
behavior of things like that tickles my spidey sense a bit. I.e. I can
see people having written scripts like that which would break (it's
often easier to cd around after globbing than staying at the top-level,
then jump back).

So I wonder (especially with Glen's comment in
<20211123003958.3978-1-chooglen@google.com>) if this is being done at
the right API level. E.g. maybe it would be better for some commands to
ease into this with an advise() or warning() and not a die() or error(),
or have the die() be in the likes of "git switch" but not "reset
--hard".

Or maybe not, just food for thought...

Re: [PATCH 1/8] t2501: add various tests for removing the current working directory

[Elijah Newren]

On Mon, Nov 22, 2021 at 6:27 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Mon, Nov 22 2021, Elijah Newren wrote:
>
> > On Sun, Nov 21, 2021 at 9:59 AM Ævar Arnfjörð Bjarmason
> > <avarab@gmail.com> wrote:
> >>
> >> On Sun, Nov 21 2021, Elijah Newren via GitGitGadget wrote:
> >>
> >> > From: Elijah Newren <newren@gmail.com>
> >>
> >> > +test_expect_failure 'checkout fails if cwd needs to be removed' '
> >> > +     git checkout foo/bar/baz &&
> >> > +     test_when_finished "git clean -fdx" &&
> >> > +
> >> > +     mkdir dirORfile &&
> >> > +     (
> >> > +             cd dirORfile &&
> >> > +
> >> > +             test_must_fail git checkout fd_conflict 2>../error &&
> >> > +             grep "Refusing to remove the current working directory" ../error
> >> > +     ) &&
> >> > +
> >> > +     test_path_is_dir dirORfile
> >>
> >>
> >> I'd find this & the rest of this series much easier to understand if we
> >> started out by positively asserting the current behavior here, and
> >> didn't test_cmp/grep for erro r messages that don't exist anymore.
> >
> > Yeah, this is my fault for my bad commit message.  I stated I was
> > adding tests checking for the problems of interest, making it sound
> > like I was testing existing behavior, but I should have stated I was
> > adding tests with the behavior we'd prefer to have (i.e. basically a
> > test-driven-development) setup.
> >
> > Also, there really wouldn't need to be so many tests for describing
> > the existing behavior.  It's basically just `git
> > $OPERATION_THAT_REMOVES_CWD_AS_SIDE_EFFECT` followed by nearly any
> > other git command will cause the second and later commands to fail
> > with:
> >
> > ```
> > shell-init: error retrieving current directory: getcwd: cannot access
> > parent directories: No such file or directory
> > fatal: Unable to read current working directory: No such file or directory
> > ```
> >
> > However, we do need a lot of tests for corrected behavior, because
> > there are so many different codepaths we can follow which will lead to
> > deletion of the current working directory.
>
> Currently if I do e.g.:
>
>     git checkout master
>     git clean -dxf
>     cd perl
>     git checkout v0.99
>     cd ../
>     git clean -dxfn
>
> Nothing breaks and I don't end up with an empty perl/ to remove. With
> these patches we'd either die on the "checkout" (I think) keep the
> "perl" and have an empty perl/ to report in the "git clean -dxfn" at the
> end (I'm not sure which, I forgot and haven't re-read this series just
> now).

You'd have an empty perl/ left after the checkout, which would be
cleaned up by your final git clean command.

> I think changing it anyway might be justifiable, but changing the
> behavior of things like that tickles my spidey sense a bit. I.e. I can
> see people having written scripts like that which would break (it's
> often easier to cd around after globbing than staying at the top-level,
> then jump back).

I disagree this would break any user scripts.  If people expect a 'git
checkout' or 'git rebase' to always work, their script is _already_
broken.  The presence of any untracked files within the directory
already results in a hard error -- we refuse to remove non-empty
directories (unless all files are tracked and unmodified).  This rule
deserves a clarification: treat the current working directory as
non-empty since the parent process is likely still parked there..

Further, our own commands are broken/misbehaving due to us not
erroring out; see e.g.
https://lore.kernel.org/git/xmqqv93n7q1v.fsf@gitster.g/ and its
grandparent.  User scripts likely have lurking problems too.

> So I wonder (especially with Glen's comment in
> <20211123003958.3978-1-chooglen@google.com>) if this is being done at
> the right API level.

Glen's comment was interesting, but provided no specifics about how
the changes I made could cause any possible harm in his case.
Further, the fact that others are adding extra places doing cleanup
sound like additional codepaths that should be protected for the exact
same reasons.  I think we absolutely want my changes affecting his new
codepaths.

> E.g. maybe it would be better for some commands to
> ease into this with an advise() or warning() and not a die() or error(),
> or have the die() be in the likes of "git switch" but not "reset
> --hard".

The commands that don't need to remove the current working directory
but just were as a convenience, no longer do and continue on just
fine.  Commands that need to remove the current working directory in
order to place a file there will error out, just as they would have
when trying to remove a directory with untracked files.  I see no need
to ease into anything here.

> Or maybe not, just food for thought...

You may also be interested in reading more of the other thread I
linked to from my cover letter; all these cases were discussed in good
detail over there.  For example, look at
https://lore.kernel.org/git/CABPp-BFmU+RaAjq4_0-PSfRgH1Jc63nN0fMuDWk2+iDbdz7CCA@mail.gmail.com/.
Peff's previous suggestion was to just make the commands error out if
they'd normally remove the current working directory and require the
user to run from a different directory instead.  My version lightened
that requirement so it only errors out if the current working
directory needed to be removed in order to place something else there
(and if nothing else was needed to be placed there, then just leaving
the directory around).

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Glen Choo]

Elijah Newren <newren@gmail.com> writes:

>> I don't think it's appropriate to implement user-facing concern (don't remove
>> cwd because this will confuse users) in library functions like remove_path().
>> remove_path() has other callers e.g. midx.c, and possible future callers e.g.
>> we're working on adding a command to delete corrupted commit-graphs and this
>> library function would be extremely handy.
>
> I think we'd want this code change for those cases too.  Said another
> way, why wouldn't these callers want to avoid deleting the original
> current working directory of the git process (which is likely still
> the current working directory of the parent process)?  Deleting that
> directory causes problems regardless of whether it's a user-facing
> command (rm, merge, stash, etc.) or something more internal (midx or
> commit-graphs stuff being called by gc) that is doing the deleting.

I agree that most, possibly all, of our commands should prefer to die
than to remove the cwd, but that doesn't justify adding
application-level concerns to a general-purpose utility function. Even
if it sounds overly defensive, having an obviously correct utility
function makes it easier for future authors to know exactly what their
code is doing and why. And surely if we're imaginative enough, we can
definitely dream up some possible use cases for remove_path() that don't
want this dying behavior e.g. other applications that link to our
libraries, or some new merge strategy that may need to remove + restore
the cwd.

I'm not going to say that we'll *definitely* need remove_path()
in its current form, but mixing concerns like this is an invitation to
unexpected behavior. An (imperfect) example that demonstrates this
principle is https://lore.kernel.org/git/24bffdab139435173712101aaf72f7277298c99d.1632497954.git.gitgitgadget@gmail.com/,
where we made a change to a generic path matching function in order to
speed up unpack_trees(), but accidentally ended up breaking gitignore.

> Putting it in this helper function means we protect all current and
> future callers without developers having to remember which
> "remove_path()" variant they need and why.

Code comments sound like adequate protection to me. Fudging the names a
little..

  /*
   * Pretend we have utility function that generalizes
   * check-then-delete (though we probably won't need it).
   */
  static int remove_path_conditionally(const char *name, check_path_fn can_delete_path);

  /**
   * This is identical to remove_path(), except that it will die if
   * attempting to remove the_cwd. When writing Git commands, prefer
   * using this over remove_path() so that we don't delete the cwd and
   * leave the user in a confusing state.
   */
  int remove_path_except_cwd(const char *name)
  {
    return remove_path_conditionally(name, die_on_cwd);
  }

  /*
   * Tries to remove the path with empty directories along it, ignores
   * ENOENT. Unless you really need to remove the path unconditionally,
   * consider using remove_path_except_cwd() instead.
   */
  int remove_path(const char *name);

>> It seems more appropriate to check the_cwd from builtin/add.c and builtin/rm.c
>> instead.
>
> Not sure how you determined that those two paths are affected or that
> those are the only two.

Typo: s/add/apply.

I took the example from your test cases:

  diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
  index ff4e7cd89fa..4362e7b15e5 100755
  --- a/t/t2501-cwd-empty.sh
  +++ b/t/t2501-cwd-empty.sh
  @@ -191,7 +191,7 @@ test_expect_success 'revert fails if cwd needs to be removed' '
    test_path_is_dir dirORfile
   '
 
  -test_expect_failure 'rm does not remove cwd incidentally' '
  +test_expect_success 'rm does not remove cwd incidentally' '
    test_when_finished "git reset --hard" &&
    git checkout foo/bar/baz &&
 
  @@ -205,7 +205,7 @@ test_expect_failure 'rm does not remove cwd incidentally' '
    test_path_is_dir foo
   '
 
  -test_expect_failure 'apply does not remove cwd incidentally' '
  +test_expect_success 'apply does not remove cwd incidentally' '
    test_when_finished "git reset --hard" &&
    git checkout foo/bar/baz &&

I read this as "I made these changes in order to make these tests pass".
I really like the 'TDD-ish' approach you used in this series; as a
reader, it gave me a clear idea of the expected outcome of your changes.
From that perspective, the fact that there are certainly untested paths
which are affected takes away some of the benefits of this approach.

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Elijah Newren]

On Tue, Nov 23, 2021 at 10:19 AM Glen Choo <chooglen@google.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> >> I don't think it's appropriate to implement user-facing concern (don't remove
> >> cwd because this will confuse users) in library functions like remove_path().
> >> remove_path() has other callers e.g. midx.c, and possible future callers e.g.
> >> we're working on adding a command to delete corrupted commit-graphs and this
> >> library function would be extremely handy.
> >
> > I think we'd want this code change for those cases too.  Said another
> > way, why wouldn't these callers want to avoid deleting the original
> > current working directory of the git process (which is likely still
> > the current working directory of the parent process)?  Deleting that
> > directory causes problems regardless of whether it's a user-facing
> > command (rm, merge, stash, etc.) or something more internal (midx or
> > commit-graphs stuff being called by gc) that is doing the deleting.
>
> I agree that most, possibly all, of our commands should prefer to die
> than to remove the cwd, but that doesn't justify adding
> application-level concerns to a general-purpose utility function. Even
> if it sounds overly defensive, having an obviously correct utility
> function makes it easier for future authors to know exactly what their
> code is doing and why. And surely if we're imaginative enough, we can
> definitely dream up some possible use cases for remove_path() that don't
> want this dying behavior e.g. other applications that link to our
> libraries, or some new merge strategy that may need to remove + restore
> the cwd.

Sounds like your objections here are based on a misunderstanding.  I
totally agree with you that adding dying behavior to these functions
would be wrong.

My patch doesn't do that.

> I'm not going to say that we'll *definitely* need remove_path()
> in its current form, but mixing concerns like this is an invitation to
> unexpected behavior. An (imperfect) example that demonstrates this
> principle is https://lore.kernel.org/git/24bffdab139435173712101aaf72f7277298c99d.1632497954.git.gitgitgadget@gmail.com/,
> where we made a change to a generic path matching function in order to
> speed up unpack_trees(), but accidentally ended up breaking gitignore.

There's no mixture of concerns; my patch is correcting this library
function to more fully match its documented intent; from dir.h:

    /* tries to remove the path with empty directories along it,
ignores ENOENT */
    int remove_path(const char *path);

Since the parent process's current working directory is still likely
parked in that directory, there is a good reason to treat it as
non-empty.  Thus the cwd should not be one of those directories
removed along with the specified path.  No need to die, just stop
removing the leading directories once it hits the cwd (much like it'd
stop once it hit a directory that had files left in it).

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Glen Choo]

Elijah Newren <newren@gmail.com> writes:

>> I agree that most, possibly all, of our commands should prefer to die
>> than to remove the cwd, but that doesn't justify adding
>> application-level concerns to a general-purpose utility function. Even
>> if it sounds overly defensive, having an obviously correct utility
>> function makes it easier for future authors to know exactly what their
>> code is doing and why. And surely if we're imaginative enough, we can
>> definitely dream up some possible use cases for remove_path() that don't
>> want this dying behavior e.g. other applications that link to our
>> libraries, or some new merge strategy that may need to remove + restore
>> the cwd.
>
> Sounds like your objections here are based on a misunderstanding.  I
> totally agree with you that adding dying behavior to these functions
> would be wrong.
>
> My patch doesn't do that.

Ah my mistake, that should be s/die/'stop gently'. Even so, that is not
at the core of my objection, mixing of concerns is.

>> I'm not going to say that we'll *definitely* need remove_path()
>> in its current form, but mixing concerns like this is an invitation to
>> unexpected behavior. An (imperfect) example that demonstrates this
>> principle is https://lore.kernel.org/git/24bffdab139435173712101aaf72f7277298c99d.1632497954.git.gitgitgadget@gmail.com/,
>> where we made a change to a generic path matching function in order to
>> speed up unpack_trees(), but accidentally ended up breaking gitignore.
>
> There's no mixture of concerns; my patch is correcting this library
> function to more fully match its documented intent; from dir.h:
>
>     /* tries to remove the path with empty directories along it,
> ignores ENOENT */
>     int remove_path(const char *path);

I don't think that there is a mismatch; reading the implementation +
documented intent seems to make it clear that 'emptiness' is defined by
directory contents, not the presence of any processes using it as its
current working directory.

> Since the parent process's current working directory is still likely
> parked in that directory, there is a good reason to treat it as
> non-empty.  Thus the cwd should not be one of those directories
> removed along with the specified path.  No need to die, just stop
> removing the leading directories once it hits the cwd (much like it'd
> stop once it hit a directory that had files left in it).

This doesn't sound like a typical definition of 'emptiness' to me, but I
can accept it if others also find it compelling. IOW if your definition
of 'emptiness' is compelling enough, then I'll be convinced that there
is no mixing of concerns and there would be no objection.

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Junio C Hamano]

Glen Choo <chooglen@google.com> writes:

> This doesn't sound like a typical definition of 'emptiness' to me, but I
> can accept it if others also find it compelling. IOW if your definition
> of 'emptiness' is compelling enough, then I'll be convinced that there
> is no mixing of concerns and there would be no objection.

FWIW, I do not find it compelling.  I can grant that it might be
convenient, but I do not think it is a good idea to explain the
reason why the directory is protected is because it is "not empty".

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Elijah Newren]

On Tue, Nov 23, 2021 at 12:33 PM Glen Choo <chooglen@google.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> >> I agree that most, possibly all, of our commands should prefer to die
> >> than to remove the cwd, but that doesn't justify adding
> >> application-level concerns to a general-purpose utility function. Even
> >> if it sounds overly defensive, having an obviously correct utility
> >> function makes it easier for future authors to know exactly what their
> >> code is doing and why. And surely if we're imaginative enough, we can
> >> definitely dream up some possible use cases for remove_path() that don't
> >> want this dying behavior e.g. other applications that link to our
> >> libraries, or some new merge strategy that may need to remove + restore
> >> the cwd.
> >
> > Sounds like your objections here are based on a misunderstanding.  I
> > totally agree with you that adding dying behavior to these functions
> > would be wrong.
> >
> > My patch doesn't do that.
>
> Ah my mistake, that should be s/die/'stop gently'. Even so, that is not
> at the core of my objection, mixing of concerns is.

If I were to introduce a new function, say remove_path_not_cwd(), to
avoid this claimed mixing of concerns, what would that buy us?

I've looked at every single caller of remove_path() in the git
codebase.  If I did introduce a new function, as you seem to want, my
series would include two more commits: one that would replace _every_
call of remove_path() in the codebase with a call to the new function,
and one that would delete the remove_path() declaration and definition
in dir.[ch] since they would be unused.  The net effect would be
merely forcing git developers to learn a different function name.
(I'd probably also follow it up later with another commit to rename
remove_path_not_cwd() to remove_path(), for simplicity, and so I don't
have to remember the longer name anymore.)


I haven't yet found or heard of any potential callers, even
hypothetical, that would be harmed by the modified behavior.  Every
case suggested so far actually sounds like a good candidate for the
modified behavior.

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Elijah Newren]

On Tue, Nov 23, 2021 at 1:57 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Glen Choo <chooglen@google.com> writes:
>
> > This doesn't sound like a typical definition of 'emptiness' to me, but I
> > can accept it if others also find it compelling. IOW if your definition
> > of 'emptiness' is compelling enough, then I'll be convinced that there
> > is no mixing of concerns and there would be no objection.
>
> FWIW, I do not find it compelling.  I can grant that it might be
> convenient, but I do not think it is a good idea to explain the
> reason why the directory is protected is because it is "not empty".

Is the objection to my hand-wavy explanation?  If so, point taken.
However, I'm curious if you're also objecting to my commit message
and/or the patch as well.

If your objection also includes my commit message, but not the patch,
would the following suit your taste better? :

"""
remove_path() was added in 4a92d1bfb784 (Add remove_path: a function to
remove as much as possible of a path, 2008-09-27) to, as it says, remove
as much of a path as possible.  Why remove as much as possible?  Well,
at the time we probably would have said something like:

  * removing leading directories makes things feel tidy
  * removing leading directories doesn't hurt anything so long as they
    had no files in them.

However, the second reason (and perhaps also the first) do not hold when
that empty directory was the current working directory we inherited from
our parent process.  Leaving the parent process in a deleted directory
can cause user confusion when subsequent processes fail: any git
command, for example, will immediately fail with

    fatal: Unable to read current working directory: No such file or directory

Modify remove_path() so that the empty leading directories it also
deletes does not include the current working directory we inherited from
our parent process.  Also do the same for remove_dir_recursively().
"""

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Glen Choo]

Elijah Newren <newren@gmail.com> writes:

>> >> I agree that most, possibly all, of our commands should prefer to die
>> >> than to remove the cwd, but that doesn't justify adding
>> >> application-level concerns to a general-purpose utility function. Even
>> >> if it sounds overly defensive, having an obviously correct utility
>> >> function makes it easier for future authors to know exactly what their
>> >> code is doing and why. And surely if we're imaginative enough, we can
>> >> definitely dream up some possible use cases for remove_path() that don't
>> >> want this dying behavior e.g. other applications that link to our
>> >> libraries, or some new merge strategy that may need to remove + restore
>> >> the cwd.
>> >
>> > Sounds like your objections here are based on a misunderstanding.  I
>> > totally agree with you that adding dying behavior to these functions
>> > would be wrong.
>> >
>> > My patch doesn't do that.
>>
>> Ah my mistake, that should be s/die/'stop gently'. Even so, that is not
>> at the core of my objection, mixing of concerns is.
>
> If I were to introduce a new function, say remove_path_not_cwd(), to
> avoid this claimed mixing of concerns, what would that buy us?
> I've looked at every single caller of remove_path() in the git
> codebase.  If I did introduce a new function, as you seem to want, my
> series would include two more commits: one that would replace _every_
> call of remove_path() in the codebase with a call to the new function,
> and one that would delete the remove_path() declaration and definition
> in dir.[ch] since they would be unused.

There is at least one other possible outcome, which is that
remove_path() is replaced with remove_path_not_cwd() in all callers that
obviously want it e.g. builtins, but not replaced in other callers. My
mental model of this is that the two functions serve two different use
cases:

1) remove_path(): Remove a path and all empty directories
2) remove_path_not_cwd(): Remove a path and all empty directories,
   except cwd

> I haven't yet found or heard of any potential callers, even
> hypothetical, that would be harmed by the modified behavior.  Every
> case suggested so far actually sounds like a good candidate for the
> modified behavior.

I trust that you have considered this change carefully, so I will
downgrade my objection to a nitpick. remove_path() seems nice to have as
a low-level function but I certainly can't imagine any non-contrived
use cases that *wouldn't* benefit.

To me, a more compelling argument is that protecting cwd is important
in order to ensure correctness, and user experience is an incidental
benefit. AFAICT that is not the argument you are making, but perhaps
there is some correctness benefit as well?

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Ævar Arnfjörð Bjarmason]

On Tue, Nov 23 2021, Elijah Newren wrote:

> On Tue, Nov 23, 2021 at 10:19 AM Glen Choo <chooglen@google.com> wrote:
>>
>> Elijah Newren <newren@gmail.com> writes:
>> [...]
>> I'm not going to say that we'll *definitely* need remove_path()
>> in its current form, but mixing concerns like this is an invitation to
>> unexpected behavior. An (imperfect) example that demonstrates this
>> principle is https://lore.kernel.org/git/24bffdab139435173712101aaf72f7277298c99d.1632497954.git.gitgitgadget@gmail.com/,
>> where we made a change to a generic path matching function in order to
>> speed up unpack_trees(), but accidentally ended up breaking gitignore.
>
> There's no mixture of concerns; my patch is correcting this library
> function to more fully match its documented intent; from dir.h:
>
>     /* tries to remove the path with empty directories along it,
> ignores ENOENT */
>     int remove_path(const char *path);
>
> Since the parent process's current working directory is still likely
> parked in that directory, there is a good reason to treat it as
> non-empty.  Thus the cwd should not be one of those directories
> removed along with the specified path.  No need to die, just stop
> removing the leading directories once it hits the cwd (much like it'd
> stop once it hit a directory that had files left in it).

I can buy that we'd pick this new behavior as a worthwhile trade-off,
but not that anyone intended for this to be the behavior all along.

I don't think "a process is sitting in it" has ever been anyone's idea
of a "non-empty directory". Rather it's what rmdir() returning EEXIST or
ENOTEMPTY maps to.

Doesn't this series also change the behavior of e.g.:

    cd contrib/subtree
    git rm -r ../subtree

If so then re the "spidey sense" comment I had earlier: There's no rm
codepaths or tests changed by this series, so the implementation of
doing it at this lower level might be casting too wide a net. Wouldn't
e.g. changing callers that use "remove_dir_recursively()" to use a
"remove_dir_recursively_not_cwd()" (or whatever) be a gentler way of
introducing this, and make sure that each step of the way we grok what's
being changed, that there's test coverage etc.

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Elijah Newren]

On Tue, Nov 23, 2021 at 5:19 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Tue, Nov 23 2021, Elijah Newren wrote:
>
> > On Tue, Nov 23, 2021 at 10:19 AM Glen Choo <chooglen@google.com> wrote:
> >>
> >> Elijah Newren <newren@gmail.com> writes:
> >> [...]
> >> I'm not going to say that we'll *definitely* need remove_path()
> >> in its current form, but mixing concerns like this is an invitation to
> >> unexpected behavior. An (imperfect) example that demonstrates this
> >> principle is https://lore.kernel.org/git/24bffdab139435173712101aaf72f7277298c99d.1632497954.git.gitgitgadget@gmail.com/,
> >> where we made a change to a generic path matching function in order to
> >> speed up unpack_trees(), but accidentally ended up breaking gitignore.
> >
> > There's no mixture of concerns; my patch is correcting this library
> > function to more fully match its documented intent; from dir.h:
> >
> >     /* tries to remove the path with empty directories along it,
> > ignores ENOENT */
> >     int remove_path(const char *path);
> >
> > Since the parent process's current working directory is still likely
> > parked in that directory, there is a good reason to treat it as
> > non-empty.  Thus the cwd should not be one of those directories
> > removed along with the specified path.  No need to die, just stop
> > removing the leading directories once it hits the cwd (much like it'd
> > stop once it hit a directory that had files left in it).
>
> I can buy that we'd pick this new behavior as a worthwhile trade-off,
> but not that anyone intended for this to be the behavior all along.
>
> I don't think "a process is sitting in it" has ever been anyone's idea
> of a "non-empty directory". Rather it's what rmdir() returning EEXIST or
> ENOTEMPTY maps to.

Yeah, Junio commented on my reasoning in that same paragraph of mine.
Bad reasoning on my part, and you were both right to call it out.

But that reasoning wasn't the underlying motivation for Peff to
suggest the behavior behind this series[1] and this patch, nor the
rationale Junio used to say that the overall behavioral change behind
this series "makes sense".[2]

[1] https://lore.kernel.org/git/YS8eEtwQvF7TaLCb@coredump.intra.peff.net/
[2] https://lore.kernel.org/git/xmqqo86elyht.fsf@gitster.g/

> Doesn't this series also change the behavior of e.g.:
>
>     cd contrib/subtree
>     git rm -r ../subtree

Yes, of course!

Before:

    $ cd contrib/subtree
    $ git rm -r -q ../subtree/
    $ ls -ld ../subtree/
    ls: cannot access '../subtree/': No such file or directory
    $ git status --porcelain
    fatal: Unable to read current working directory: No such file or directory
    $ git checkout HEAD ../subtree/
    fatal: Unable to read current working directory: No such file or directory

After:

    $ cd contrib/subtree
    $ git rm -r -q ../subtree/
    $ ls -ld ../subtree/
    drwxrwxr-x. 1 newren newren 0 Nov 23 19:18 ../subtree/
    $ git status --porcelain
    D  contrib/subtree/.gitignore
    D  contrib/subtree/COPYING
    D  contrib/subtree/INSTALL
    D  contrib/subtree/Makefile
    D  contrib/subtree/README
    D  contrib/subtree/git-subtree.sh
    D  contrib/subtree/git-subtree.txt
    D  contrib/subtree/t/Makefile
    D  contrib/subtree/t/t7900-subtree.sh
    D  contrib/subtree/todo
    $ git checkout HEAD ../subtree/
    Updated 10 paths from c557be478e

Very nice fix, don't you think?


> If so then re the "spidey sense" comment I had earlier: There's no rm
> codepaths or tests changed by this series,

That's not correct; I explicitly added a new rm test in the first
patch in my series.  Further, that same test was modified to mark it
as passing by this particular patch you are commenting on.

> so the implementation of
> doing it at this lower level might be casting too wide a net.

I'm getting the vibe that you are assuming I'm changing these two
functions without realizing what places might be calling them;
basically, that I'm just flippantly changing them.  Ignoring the
ramifications of such an assumption, if this vibe is correct, then let
me inform you that I've read over each and every caller (as well as
searched for other callers of unlink() and rmdir() throughout the tree
to see if they needed similar changes).  In my opinion, *each* *and*
*every* *single* *one* of the calls to remove_path() and
remove_dir_recursively() should take the behavioral change suggested
in this patch.

It's also not clear to me that you understand the point of the change
behind the series.  Clearly, I'm not doing well explaining it, but
have you read Peff's or Junio's comments on why they thought
protecting the_original_cwd makes sense?  Again, see the links [1] and
[2] above.  I think it'd help me understand how to respond to you
better if you could clarify to me whether you disagree with them, or
whether you agree with them but think I've gone wrong in the
implementation of their high level explanation somehow in this
particular patch.

> e.g. changing callers that use "remove_dir_recursively()" to use a
> "remove_dir_recursively_not_cwd()" (or whatever) be a gentler way of
> introducing this, and make sure that each step of the way we grok what's
> being changed, that there's test coverage etc.

I'll ask you the same two questions I asked Glen when he suggested
basically the same thing; if you can provide an answer to either one
of my questions that is compelling to me, I'd be totally onboard with
your suggested change:

(1) Can you point to any concrete example caller anywhere in the code
tree (or even provide a future hypothetical caller) that would in fact
be harmed by the change in this patch?

...and no, I'm not asking you to do my work for me; I've done that
same work -- in fact looking at all callsites -- and came up
empty-handed.  Since this and your previous email are essentially
claiming that I've probably missed things, I think it's a fair
question.

(2) What benefit would there be to introducing these new functions?

In particular, if I were to introduce these new functions, it would
look like this:

  * add new remove_path_not_cwd() and remove_dir_recursively_not_cwd()
function in one patch
  * convert all relevant callsites to use these new functions in the
subsequent patch(es)
  * delete the existing remove_path() and remove_dir_recursively()
functions for two reasons: (1) they are now unused, and (2) future
potential callers of these old functions would more than likely
reintroduce the bugs this series is trying to fix if they were to use
them and should be discouraged from doing so
  * rename {remove_path,remove_dir_recursively}_not_cwd() to remove
the "_not_cwd" suffix in order to have more memorable and less ugly
names

Once finished, the end result would be identical to this patch.  Well,
identical other than taking more patches to get to the same end result
and using up more reviewer time.  If there's some benefit to taking
this circuitous route, though, I'm more than willing to do so.

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Junio C Hamano]

Glen Choo <chooglen@google.com> writes:

> To me, a more compelling argument is that protecting cwd is important
> in order to ensure correctness, and user experience is an incidental
> benefit. AFAICT that is not the argument you are making, but perhaps
> there is some correctness benefit as well?

I doubt there is.

It would be annoying if the command fails to remove a directory that
becomes empty after an operation only because your interactive shell
that spawned "git" was there, especially if your system allows
interactive shell to still sit in an unlinked directory.  After such
a rmdir, `/bin/pwd` or `cd ..` may not work, but then the only thing
you need to do to recover from this situation is to run `cd
/path/to/repository` and we can continue working, without having to
worry about a leftover directory.

That may be a more pleasant end-user experience than what is being
proposed, which forces you (1) to realize that you are in a
directory that was to be removed if you weren't sitting there, and
(2) to compute how many levels of otherwise empty directories were
left because of your presense, and (3) to go up sufficient number of
levels and manually run "rm -fr" the hierarchy, to recover.

So, I dunno.  It does make the end-user experience on such a system
that allows your cwd to go away equally unpleasant to the end-user
experience on a system that does not allow your cwd to go away, so
we might be gaining a bit in the consistency department, but as has
been already discussed, "git" level protection can only notice the
process immediately above us and we wouldn't be able to pay
attention to the fact some other unrelated process is using a
directory as its cwd, so even such a consistency argument does not
go all that far.

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> On Tue, Nov 23, 2021 at 1:57 PM Junio C Hamano <gitster@pobox.com> wrote:
>>
>> Glen Choo <chooglen@google.com> writes:
>>
>> > This doesn't sound like a typical definition of 'emptiness' to me, but I
>> > can accept it if others also find it compelling. IOW if your definition
>> > of 'emptiness' is compelling enough, then I'll be convinced that there
>> > is no mixing of concerns and there would be no objection.
>>
>> FWIW, I do not find it compelling.  I can grant that it might be
>> convenient, but I do not think it is a good idea to explain the
>> reason why the directory is protected is because it is "not empty".
>
> Is the objection to my hand-wavy explanation?  If so, point taken.

The objection is against the definition of 'emptiness' Glen
perceived in your explanation ;-)

A directory is empty when there is no filesystem entity hangs below
it.  A process can have any directory as its cwd, even an empty one,
but the presense of such a process does not make an empty directory
suddenly non-empty.  That is the objection.

> However, I'm curious if you're also objecting to my commit message
> and/or the patch as well.
>
> If your objection also includes my commit message, but not the patch,
> would the following suit your taste better? :
>
> """
> remove_path() was added in 4a92d1bfb784 (Add remove_path: a function to
> remove as much as possible of a path, 2008-09-27) to, as it says, remove
> as much of a path as possible.  Why remove as much as possible?  Well,
> at the time we probably would have said something like:
>
>   * removing leading directories makes things feel tidy
>   * removing leading directories doesn't hurt anything so long as they
>     had no files in them.

I think you meant trailing, but I do not think you need to say it
twice---they say pretty much the same thing.  We are removing as
many directories that contain no files (i.e. non-directories)
underneath to make things tidy, as such a directory serves no useful
purpose.

I am not saying that an empty directory should not be protected even
when the current process sits there.  I just do not think it is a
good idea to call the protection "we protect a non-empty directory".
It is something else.

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Ævar Arnfjörð Bjarmason]

On Tue, Nov 23 2021, Elijah Newren wrote:

> On Tue, Nov 23, 2021 at 5:19 PM Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:

>> Doesn't this series also change the behavior of e.g.:
>>
>>     cd contrib/subtree
>>     git rm -r ../subtree
>
> Yes, of course!
>
> Before:
>
>     $ cd contrib/subtree
>     $ git rm -r -q ../subtree/
>     $ ls -ld ../subtree/
>     ls: cannot access '../subtree/': No such file or directory
>     $ git status --porcelain
>     fatal: Unable to read current working directory: No such file or directory
>     $ git checkout HEAD ../subtree/
>     fatal: Unable to read current working directory: No such file or directory
>
> After:
>
>     $ cd contrib/subtree
>     $ git rm -r -q ../subtree/
>     $ ls -ld ../subtree/
>     drwxrwxr-x. 1 newren newren 0 Nov 23 19:18 ../subtree/
>     $ git status --porcelain
>     D  contrib/subtree/.gitignore
>     D  contrib/subtree/COPYING
>     D  contrib/subtree/INSTALL
>     D  contrib/subtree/Makefile
>     D  contrib/subtree/README
>     D  contrib/subtree/git-subtree.sh
>     D  contrib/subtree/git-subtree.txt
>     D  contrib/subtree/t/Makefile
>     D  contrib/subtree/t/t7900-subtree.sh
>     D  contrib/subtree/todo
>     $ git checkout HEAD ../subtree/
>     Updated 10 paths from c557be478e
>
> Very nice fix, don't you think?

I'd be more sympathetic to this for the "checkout" etc. commands, but
once I add a "-f" to that "rm" I'm *really* expecting it to rm the
directory, but it won't anymore because it's in the underlying library
function.

But if the goal is to get "git status" and the like working isn't a much
more pointed fix to have setup.c handle the case of getting ENOENT from
getcwd() more gracefully. I.e. currently (and even with your patches):

    $ (mkdir blah && cd blah && rmdir ../blah && git status)
    fatal: Unable to read current working directory: No such file or directory

Whereas if we do e.g.:
	
	diff --git a/strbuf.c b/strbuf.c
	index b22e9816559..3f9a957ff9d 100644
	--- a/strbuf.c
	+++ b/strbuf.c
	@@ -600,6 +600,16 @@ int strbuf_getcwd(struct strbuf *sb)
	                        return 0;
	                }
	
	+               if (errno == ENOENT){
	+                       const char *pwd = getenv("PWD");
	+
	+                       if (pwd) {
	+                               warning(_("unable to getcwd(), but can read PWD, limping along with that..."));
	+                               strbuf_addstr(sb, pwd);
	+                               return 0;
	+                       }
	+               }
	+
	                /*
	                 * If getcwd(3) is implemented as a syscall that falls
	                 * back to a regular lookup using readdir(3) etc. then

We'll get:
	
	$ (mkdir blah && cd blah && rmdir ../blah && GIT_DISCOVERY_ACROSS_FILESYSTEM=1 ~/g/git/git status)
	warning: unable to getcwd(), but can read PWD, limping along with that...
	On branch master
	Your branch is up to date with 'origin/master'.
	
	Changes not staged for commit:
	  (use "git add <file>..." to update what will be committed)
	  (use "git restore <file>..." to discard changes in working directory)
	        modified:   ../strbuf.c
	
	no changes added to commit (use "git add" and/or "git commit -a")

I think that getenv("PWD") trick is widely supported, and once we get
past that we seem OK. The relative path to strbuf.c is even correct.

Currently you'd need to set GIT_DISCOVERY_ACROSS_FILESYSTEM=1 because we
run into another case in setup.c where we're not carrying that ENOENT
forward, but we could just patch strbuf_getcwd() or that subsequent code
to handle this edge case.

>> If so then re the "spidey sense" comment I had earlier: There's no rm
>> codepaths or tests changed by this series,
>
> That's not correct; I explicitly added a new rm test in the first
> patch in my series.  Further, that same test was modified to mark it
> as passing by this particular patch you are commenting on.

Sorry about that, I didn't look carefully enough.

>> so the implementation of
>> doing it at this lower level might be casting too wide a net.
>
> I'm getting the vibe that you are assuming I'm changing these two
> functions without realizing what places might be calling them;
> basically, that I'm just flippantly changing them.  Ignoring the
> ramifications of such an assumption, if this vibe is correct[...]

Sorry no, I didn't mean to imply that. I snipped the rest, but hopefully
this answers the questions you had well enough (and in the time I have
for this reply):

I'm not concerned that you didn't research this change well enough, I
just find it a bit iffy to introduce semantics in git around FS
operations that don't conform with that of POSIX & the underlying OS. My
*nix system happily accepts an "rm -rf" or an "rmdir" of the directory
I'm in, I'd expect git to do the same.

But whatever research we do on in-tree users we're left with changing
behavior for users in the wild, e.g. a script that cd's to each subdir
in a repo, inspects something, and if it wants to remove that does an
"git rm -r" of the directory it's in, commits, and expects the repo to
be clean afterwards.

I did follow/read at least one of the the original discussions[1] a bit,
and some earlier discussion I'm vaguely recalling around bisect in a
subdir.

If the underlying goal is to address the UX problem in git of e.g. "git
status" and the like hard-dying I wonder if something in the direction
of the setup.c/strbuf.c change above might be more of a gentle change.

That approach of a more gentler setup.c also has the benefit of having
git work when it ends up in this situation without the git commands
having landed it there, as in the above "rmdir" example.

Anyway, I really didn't have time to look at this very carefully. I just
remember looking into this with bisect/status etc. in the past, and
thinking that these problems were solvable in those cases, i.e. they
were just being overly anal about ENOENT, and not falling back on "PWD"
etc.

1. https://lore.kernel.org/git/YS3Tv7UfNkF+adry@coredump.intra.peff.net/

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Ævar Arnfjörð Bjarmason]

On Wed, Nov 24 2021, Ævar Arnfjörð Bjarmason wrote:

> On Tue, Nov 23 2021, Elijah Newren wrote:
>
>> On Tue, Nov 23, 2021 at 5:19 PM Ævar Arnfjörð Bjarmason
>> <avarab@gmail.com> wrote:
>
>>> Doesn't this series also change the behavior of e.g.:
>>>
>>>     cd contrib/subtree
>>>     git rm -r ../subtree
>>
>> Yes, of course!
>>
>> Before:
>>
>>     $ cd contrib/subtree
>>     $ git rm -r -q ../subtree/
>>     $ ls -ld ../subtree/
>>     ls: cannot access '../subtree/': No such file or directory
>>     $ git status --porcelain
>>     fatal: Unable to read current working directory: No such file or directory
>>     $ git checkout HEAD ../subtree/
>>     fatal: Unable to read current working directory: No such file or directory
>>
>> After:
>>
>>     $ cd contrib/subtree
>>     $ git rm -r -q ../subtree/
>>     $ ls -ld ../subtree/
>>     drwxrwxr-x. 1 newren newren 0 Nov 23 19:18 ../subtree/
>>     $ git status --porcelain
>>     D  contrib/subtree/.gitignore
>>     D  contrib/subtree/COPYING
>>     D  contrib/subtree/INSTALL
>>     D  contrib/subtree/Makefile
>>     D  contrib/subtree/README
>>     D  contrib/subtree/git-subtree.sh
>>     D  contrib/subtree/git-subtree.txt
>>     D  contrib/subtree/t/Makefile
>>     D  contrib/subtree/t/t7900-subtree.sh
>>     D  contrib/subtree/todo
>>     $ git checkout HEAD ../subtree/
>>     Updated 10 paths from c557be478e
>>
>> Very nice fix, don't you think?
>
> I'd be more sympathetic to this for the "checkout" etc. commands, but
> once I add a "-f" to that "rm" I'm *really* expecting it to rm the
> directory, but it won't anymore because it's in the underlying library
> function.
>
> But if the goal is to get "git status" and the like working isn't a much
> more pointed fix to have setup.c handle the case of getting ENOENT from
> getcwd() more gracefully. I.e. currently (and even with your patches):
>
>     $ (mkdir blah && cd blah && rmdir ../blah && git status)
>     fatal: Unable to read current working directory: No such file or directory
>
> Whereas if we do e.g.:
> 	
> 	diff --git a/strbuf.c b/strbuf.c
> 	index b22e9816559..3f9a957ff9d 100644
> 	--- a/strbuf.c
> 	+++ b/strbuf.c
> 	@@ -600,6 +600,16 @@ int strbuf_getcwd(struct strbuf *sb)
> 	                        return 0;
> 	                }
> 	
> 	+               if (errno == ENOENT){
> 	+                       const char *pwd = getenv("PWD");
> 	+
> 	+                       if (pwd) {
> 	+                               warning(_("unable to getcwd(), but can read PWD, limping along with that..."));
> 	+                               strbuf_addstr(sb, pwd);
> 	+                               return 0;
> 	+                       }
> 	+               }
> 	+
> 	                /*
> 	                 * If getcwd(3) is implemented as a syscall that falls
> 	                 * back to a regular lookup using readdir(3) etc. then
>
> We'll get:
> 	
> 	$ (mkdir blah && cd blah && rmdir ../blah && GIT_DISCOVERY_ACROSS_FILESYSTEM=1 ~/g/git/git status)
> 	warning: unable to getcwd(), but can read PWD, limping along with that...
> 	On branch master
> 	Your branch is up to date with 'origin/master'.
> 	
> 	Changes not staged for commit:
> 	  (use "git add <file>..." to update what will be committed)
> 	  (use "git restore <file>..." to discard changes in working directory)
> 	        modified:   ../strbuf.c
> 	
> 	no changes added to commit (use "git add" and/or "git commit -a")
>
> I think that getenv("PWD") trick is widely supported, and once we get
> past that we seem OK. The relative path to strbuf.c is even correct.
>
> Currently you'd need to set GIT_DISCOVERY_ACROSS_FILESYSTEM=1 because we
> run into another case in setup.c where we're not carrying that ENOENT
> forward, but we could just patch strbuf_getcwd() or that subsequent code
> to handle this edge case.
>
>>> If so then re the "spidey sense" comment I had earlier: There's no rm
>>> codepaths or tests changed by this series,
>>
>> That's not correct; I explicitly added a new rm test in the first
>> patch in my series.  Further, that same test was modified to mark it
>> as passing by this particular patch you are commenting on.
>
> Sorry about that, I didn't look carefully enough.
>
>>> so the implementation of
>>> doing it at this lower level might be casting too wide a net.
>>
>> I'm getting the vibe that you are assuming I'm changing these two
>> functions without realizing what places might be calling them;
>> basically, that I'm just flippantly changing them.  Ignoring the
>> ramifications of such an assumption, if this vibe is correct[...]
>
> Sorry no, I didn't mean to imply that. I snipped the rest, but hopefully
> this answers the questions you had well enough (and in the time I have
> for this reply):
>
> I'm not concerned that you didn't research this change well enough, I
> just find it a bit iffy to introduce semantics in git around FS
> operations that don't conform with that of POSIX & the underlying OS. My
> *nix system happily accepts an "rm -rf" or an "rmdir" of the directory
> I'm in, I'd expect git to do the same.
>
> But whatever research we do on in-tree users we're left with changing
> behavior for users in the wild, e.g. a script that cd's to each subdir
> in a repo, inspects something, and if it wants to remove that does an
> "git rm -r" of the directory it's in, commits, and expects the repo to
> be clean afterwards.
>
> I did follow/read at least one of the the original discussions[1] a bit,
> and some earlier discussion I'm vaguely recalling around bisect in a
> subdir.
>
> If the underlying goal is to address the UX problem in git of e.g. "git
> status" and the like hard-dying I wonder if something in the direction
> of the setup.c/strbuf.c change above might be more of a gentle change.
>
> That approach of a more gentler setup.c also has the benefit of having
> git work when it ends up in this situation without the git commands
> having landed it there, as in the above "rmdir" example.
>
> Anyway, I really didn't have time to look at this very carefully. I just
> remember looking into this with bisect/status etc. in the past, and
> thinking that these problems were solvable in those cases, i.e. they
> were just being overly anal about ENOENT, and not falling back on "PWD"
> etc.
>
> 1. https://lore.kernel.org/git/YS3Tv7UfNkF+adry@coredump.intra.peff.net/

I fleshened this out a bit in this WIP change:
https://github.com/avar/git/tree/avar/setup-handle-gone-directory +
commit:
https://github.com/avar/git/commit/97968518909eef88edba44973b7885d154b7a273

As noted there there's some caveats, but so far nothing I spotted that
can't be overcome. It's particularly painful to test it because of an
implementation detail of our test suite, the bin-wrappers are
shellscripts, and the very first thing they do is reset $PWD (none of
which happens if you run the real "git" binary).

That's b.t.w. the "shell-init" error you noted in
https://lore.kernel.org/git/CABPp-BEp3OL7F2J_LzqtC-x-8pBUPO8ZR1fTx_6XbqZeOH1kRw@mail.gmail.com/,
it's from the bin-wrapper.

I really wish we didn't have the bin-wrappers...

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Philip Oakley]

On 24/11/2021 11:14, Ævar Arnfjörð Bjarmason wrote:
> I'm not concerned that you didn't research this change well enough, I
> just find it a bit iffy to introduce semantics in git around FS
> operations that don't conform with that of POSIX & the underlying OS. My
> *nix system happily accepts an "rm -rf" or an "rmdir" of the directory
> I'm in, I'd expect git to do the same.
Isn't this the same, conceptually, as trying to remove the root
directory, but from a Git perspective?

i.e. Something along the lines of
https://superuser.com/questions/542978/is-it-possible-to-remove-the-root-directory
(their answer is 'no' without a special option, default since 2006)

If I read the arguments correctly, Elijah is saying that Git shouldn't
delete it's own root (cwd) directory, and that it is already implicit
within the current Git code.

Philip

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

>> Doesn't this series also change the behavior of e.g.:
>>
>>     cd contrib/subtree
>>     git rm -r ../subtree
>
> Yes, of course!
>
> Before:
>
>     $ cd contrib/subtree
>     $ git rm -r -q ../subtree/
>     $ ls -ld ../subtree/
>     ls: cannot access '../subtree/': No such file or directory
>     $ git status --porcelain
>     fatal: Unable to read current working directory: No such file or directory
>     $ git checkout HEAD ../subtree/
>     fatal: Unable to read current working directory: No such file or directory
>
> After:
>
>     $ cd contrib/subtree
>     $ git rm -r -q ../subtree/
>     $ ls -ld ../subtree/
>     drwxrwxr-x. 1 newren newren 0 Nov 23 19:18 ../subtree/
>     $ git status --porcelain
>     D  contrib/subtree/.gitignore
>     D  contrib/subtree/COPYING
>     D  contrib/subtree/INSTALL
>     D  contrib/subtree/Makefile
>     D  contrib/subtree/README
>     D  contrib/subtree/git-subtree.sh
>     D  contrib/subtree/git-subtree.txt
>     D  contrib/subtree/t/Makefile
>     D  contrib/subtree/t/t7900-subtree.sh
>     D  contrib/subtree/todo
>     $ git checkout HEAD ../subtree/
>     Updated 10 paths from c557be478e
>
> Very nice fix, don't you think?

Not necessarily.  Devil's advocate hat on.

Because you used "ls -ld" in your illustration, we cannot see what
happened to contrib/subtree/t/ directory, but presumably that one is
gone.

Now, after this, if we "git commit", making contrib/subtree
directory truly unnecessary, nothing reminds us that we need to
remove it manually with "rmdir" or "git clean -d", no?

With the "before" behaviour, the user still can go back to the top
level of the repository by going back to the directory the user was
in before typing "cd contrib/subtree", by saying "cd -" and "git
status" and everything else would just work, including not having to
worry about remembering to clean an empty directory contrib/subtree
that is no longer needed.

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Junio C Hamano]

Philip Oakley <philipoakley@iee.email> writes:

> On 24/11/2021 11:14, Ævar Arnfjörð Bjarmason wrote:
>> I'm not concerned that you didn't research this change well enough, I
>> just find it a bit iffy to introduce semantics in git around FS
>> operations that don't conform with that of POSIX & the underlying OS. My
>> *nix system happily accepts an "rm -rf" or an "rmdir" of the directory
>> I'm in, I'd expect git to do the same.
> Isn't this the same, conceptually, as trying to remove the root
> directory, but from a Git perspective?
>
> i.e. Something along the lines of
> https://superuser.com/questions/542978/is-it-possible-to-remove-the-root-directory
> (their answer is 'no' without a special option, default since 2006)
>
> If I read the arguments correctly, Elijah is saying that Git shouldn't
> delete it's own root (cwd) directory, and that it is already implicit
> within the current Git code.

I do not think it is about protecting "root"; the series wants

    cd t/ && git rm -r ../t

to leave an empty directory at 't/', because "git rm" was started in
that directory.

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Elijah Newren]

On Wed, Nov 24, 2021 at 3:33 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Tue, Nov 23 2021, Elijah Newren wrote:
>
> > On Tue, Nov 23, 2021 at 5:19 PM Ævar Arnfjörð Bjarmason
> > <avarab@gmail.com> wrote:
>
> >> Doesn't this series also change the behavior of e.g.:
> >>
> >>     cd contrib/subtree
> >>     git rm -r ../subtree
> >
> > Yes, of course!
> >
> > Before:
> >
> >     $ cd contrib/subtree
> >     $ git rm -r -q ../subtree/
> >     $ ls -ld ../subtree/
> >     ls: cannot access '../subtree/': No such file or directory
> >     $ git status --porcelain
> >     fatal: Unable to read current working directory: No such file or directory
> >     $ git checkout HEAD ../subtree/
> >     fatal: Unable to read current working directory: No such file or directory
> >
> > After:
> >
> >     $ cd contrib/subtree
> >     $ git rm -r -q ../subtree/
> >     $ ls -ld ../subtree/
> >     drwxrwxr-x. 1 newren newren 0 Nov 23 19:18 ../subtree/
> >     $ git status --porcelain
> >     D  contrib/subtree/.gitignore
> >     D  contrib/subtree/COPYING
> >     D  contrib/subtree/INSTALL
> >     D  contrib/subtree/Makefile
> >     D  contrib/subtree/README
> >     D  contrib/subtree/git-subtree.sh
> >     D  contrib/subtree/git-subtree.txt
> >     D  contrib/subtree/t/Makefile
> >     D  contrib/subtree/t/t7900-subtree.sh
> >     D  contrib/subtree/todo
> >     $ git checkout HEAD ../subtree/
> >     Updated 10 paths from c557be478e
> >
> > Very nice fix, don't you think?
>
> I'd be more sympathetic to this for the "checkout" etc. commands, but
> once I add a "-f" to that "rm" I'm *really* expecting it to rm the
> directory, but it won't anymore because it's in the underlying library
> function.

Please note that the command `git rm -rf dir` does not mean remove
`dir`.  It means forcibly and recursively remove all tracked paths
matching the pathspec `dir`.  There is certainly a difference, because
while it sometimes might end up deleting `dir`, it also might only
delete a subset of dir, and in some cases -- due to our added
conveniences -- it may also delete things outside of dir/.  It's been
that way for a decade and a half.  As such, the complaint that "`git
rm -rf dir` sometimes doesn't rm the directory" is not something new
being introduced by this series; that was already a possibility.  Yes,
I'm adding another case where that happens, but it's not some entirely
new qualitative difference.  More on this below.

> But if the goal is to get "git status" and the like working isn't a much
> more pointed fix to have setup.c handle the case of getting ENOENT from
> getcwd() more gracefully. I.e. currently (and even with your patches):
>
>     $ (mkdir blah && cd blah && rmdir ../blah && git status)
>     fatal: Unable to read current working directory: No such file or directory
>
> Whereas if we do e.g.:
>
>         diff --git a/strbuf.c b/strbuf.c
>         index b22e9816559..3f9a957ff9d 100644
>         --- a/strbuf.c
>         +++ b/strbuf.c
>         @@ -600,6 +600,16 @@ int strbuf_getcwd(struct strbuf *sb)
>                                 return 0;
>                         }
>
>         +               if (errno == ENOENT){
>         +                       const char *pwd = getenv("PWD");
>         +
>         +                       if (pwd) {
>         +                               warning(_("unable to getcwd(), but can read PWD, limping along with that..."));
>         +                               strbuf_addstr(sb, pwd);
>         +                               return 0;
>         +                       }
>         +               }
>         +
>                         /*
>                          * If getcwd(3) is implemented as a syscall that falls
>                          * back to a regular lookup using readdir(3) etc. then
>
> We'll get:
>
>         $ (mkdir blah && cd blah && rmdir ../blah && GIT_DISCOVERY_ACROSS_FILESYSTEM=1 ~/g/git/git status)
>         warning: unable to getcwd(), but can read PWD, limping along with that...
>         On branch master
>         Your branch is up to date with 'origin/master'.
>
>         Changes not staged for commit:
>           (use "git add <file>..." to update what will be committed)
>           (use "git restore <file>..." to discard changes in working directory)
>                 modified:   ../strbuf.c
>
>         no changes added to commit (use "git add" and/or "git commit -a")
>
> I think that getenv("PWD") trick is widely supported, and once we get
> past that we seem OK. The relative path to strbuf.c is even correct.
>
> Currently you'd need to set GIT_DISCOVERY_ACROSS_FILESYSTEM=1 because we
> run into another case in setup.c where we're not carrying that ENOENT
> forward, but we could just patch strbuf_getcwd() or that subsequent code
> to handle this edge case.

Ooh, nifty.  This sounds like a useful complement to this series; I'd
be in favor of it.

...
> I'm not concerned that you didn't research this change well enough, I
> just find it a bit iffy to introduce semantics in git around FS
> operations that don't conform with that of POSIX & the underlying OS. My
> *nix system happily accepts an "rm -rf" or an "rmdir" of the directory
> I'm in, I'd expect git to do the same.

This seems like an odd objection.  Semantics in git around FS
operations that don't conform with that of POSIX and the underlying OS
have been with us for a decade and a half now:

For example, if we ran the following:
  mkdir -p a/b/c
  touch a/b/c/{tracked,untracked}
  git commit -m "whatever" a/b/c/tracked
  cd a/b

Then we'd see the following:

`rm -rf ../b`: leaves a/ as an empty directory
`git rm -rf ../b`: does not nuke a/b/; in fact, it leaves
a/b/c/untracked around afterward

Also, if left a/b/c/untracked out of our setup, then we'd instead see
the following:

`rm -rf ../b`: leaves a/ as an empty directory
`git rm -rf ../b`: not only deletes a/b, but also deletes a/ -- the
parent of what was specified

Also, prior to d9b814cc97f1 (Add builtin "git rm" command,
2006-05-19), git rm would never delete the directories, so I guess you
can take our non-conforming behavior back even further than a decade
and a half.

> But whatever research we do on in-tree users we're left with changing
> behavior for users in the wild, e.g. a script that cd's to each subdir
> in a repo, inspects something, and if it wants to remove that does an
> "git rm -r" of the directory it's in, commits, and expects the repo to
> be clean afterwards.

Yes, this series is about changing behavior.  It's not an
optimization, or a code cleanup, or new command or anything.  It's
very specifically about git's behavior of deleting leading empty
directories after removing other files being limited to when those
empty directories are not the original current working directory of
the process.

However, although I still didn't know of any concrete or even
theoretical examples of problems this series would cause, I decided
after your email to do a bit more work; perhaps some of it will help:

  * I split this final patch in two, one for each affected function,
with much more detailed history and writeup.
  * remove_dir_recursively() already takes a flags parameter; I added
one for specifying whether the caller wants the traditional behavior
or this new behavior.
  * I revisited each and every caller of remove_path() and
remove_dir_recursively() and re-evaluated each one in case I had
missed something.  Of the 32 callers, most are unaffected by the
change.  Of those that are affected, most should take it, but I did
end up finding one that sometimes, in a very specific corner case,
should get the old behavior (namely when a submodule is present and
cwd is within the submodule and you pass all of --git-dir and
--work-tree and -f to rm when removing the submodule).  I added some
testcases and ensured it worked.

Since the flag is used, it's not at risk of being deleted, and thus
library callers can pick their preferred behavior.  Does that address,
or at least ameliorate, your concerns?

I'll submit the new series soon.

> I did follow/read at least one of the the original discussions[1] a bit,
> and some earlier discussion I'm vaguely recalling around bisect in a
> subdir.
>
> If the underlying goal is to address the UX problem in git of e.g. "git
> status" and the like hard-dying I wonder if something in the direction
> of the setup.c/strbuf.c change above might be more of a gentle change.
>
> That approach of a more gentler setup.c also has the benefit of having
> git work when it ends up in this situation without the git commands
> having landed it there, as in the above "rmdir" example.
>
> Anyway, I really didn't have time to look at this very carefully. I just
> remember looking into this with bisect/status etc. in the past, and
> thinking that these problems were solvable in those cases, i.e. they
> were just being overly anal about ENOENT, and not falling back on "PWD"
> etc.

The fact that git aborts in such directories is one obvious
manifestation of the problem, but my commit messages and cover letter
both called out that it can happen with non-git commands too.  I
certainly think your suggestion is very intriguing and worth pursuing,
but I think it complements rather than supplants this series.

> 1. https://lore.kernel.org/git/YS3Tv7UfNkF+adry@coredump.intra.peff.net/

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Elijah Newren]

On Wed, Nov 24, 2021 at 6:18 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> I fleshened this out a bit in this WIP change:
> https://github.com/avar/git/tree/avar/setup-handle-gone-directory +
> commit:
> https://github.com/avar/git/commit/97968518909eef88edba44973b7885d154b7a273

Sweet!

> As noted there there's some caveats, but so far nothing I spotted that
> can't be overcome. It's particularly painful to test it because of an
> implementation detail of our test suite, the bin-wrappers are
> shellscripts, and the very first thing they do is reset $PWD (none of
> which happens if you run the real "git" binary).
>
> That's b.t.w. the "shell-init" error you noted in
> https://lore.kernel.org/git/CABPp-BEp3OL7F2J_LzqtC-x-8pBUPO8ZR1fTx_6XbqZeOH1kRw@mail.gmail.com/,
> it's from the bin-wrapper.

To be a bit more precise, the error/warning is from bash rather than
the bin-wrappers specifically; for example, you can also trigger the
same shell-init message with a simple `bash -c 'echo hello'` when your
cwd has been deleted.

> I really wish we didn't have the bin-wrappers...

I'm curious: is this issue the only reason, or am I reading between
the lines correctly that you have additional reasons for that?

[PATCH v2 0/9] Avoid removing the current working directory, even if it becomes empty

[Elijah Newren via GitGitGadget]

Traditionally, if folks run git commands such as checkout or rebase from a
subdirectory, that git command could remove their current working directory
and result in subsequent git and non-git commands either getting confused or
printing messages that confuse the user (e.g. "fatal: Unable to read current
working directory: No such file or directory"). Many commands either
silently avoid removing directories that are not empty (i.e. those that have
untracked or modified files in them)[1], or show an error and abort,
depending on which is more appropriate for the command in question. With
this series, we augment the reasons to avoid removing directories to include
not just has-untracked-or-modified-files, but also to avoid removing the
original_cwd as well.

Peff and Junio provided some good pros/cons, if it helps:

 * Pros: Peff (original suggester of the idea)[2], and Junio[3]
 * Cons: Peff [2, again -- see the "P.S."], and Junio[4]

[1] well, with a few exceptions; see
https://lore.kernel.org/git/pull.1036.v3.git.1632760428.gitgitgadget@gmail.com/
[2] https://lore.kernel.org/git/YS8eEtwQvF7TaLCb@coredump.intra.peff.net/
[3] https://lore.kernel.org/git/xmqqo86elyht.fsf@gitster.g/ [4]
https://lore.kernel.org/git/xmqqo8691gr8.fsf@gitster.g/

Changes since v1:

 * clarified multiple commit messages
 * renamed the_cwd to startup_info->original_cwd to make it clearer that
   it's our parent process'es cwd that really matters, which we inherited at
   program startup. Also pulls it out of the global namespace.
 * Normalize the path for startup_info->original_cwd, and ensure that it's
   actually the original cwd even if -C is passed to git.
 * small code cleanups suggested by René and Ævar
 * split the final patch, which got the most comments into two, one for each
   function being modified; significantly extending the first of the two
   commit messages with a lot of history
 * no longer has a content conflict with so/stash-staged
 * add another value for the flags parameter that remove_dir_recursively()
   takes so that it can opt into either the old or the new behavior. Use
   that for the one special corner case I could find where it matters, and
   add a few tests around it to highlight the utility of the flag.

Elijah Newren (9):
  t2501: add various tests for removing the current working directory
  setup: introduce startup_info->original_cwd
  unpack-trees: refuse to remove startup_info->original_cwd
  unpack-trees: add special cwd handling
  symlinks: do not include startup_info->original_cwd in dir removal
  clean: do not attempt to remove startup_info->original_cwd
  stash: do not attempt to remove startup_info->original_cwd
  dir: avoid incidentally removing the original_cwd in remove_path()
  dir: new flag to remove_dir_recurse() to spare the original_cwd

 builtin/clean.c      |  47 +++++--
 builtin/rm.c         |   3 +-
 builtin/stash.c      |   6 +-
 cache.h              |   1 +
 dir.c                |  15 ++-
 dir.h                |   9 +-
 git.c                |   2 +
 setup.c              |  49 +++++++
 symlinks.c           |   8 +-
 t/t2501-cwd-empty.sh | 297 +++++++++++++++++++++++++++++++++++++++++++
 unpack-trees.c       |  30 ++++-
 unpack-trees.h       |   1 +
 12 files changed, 446 insertions(+), 22 deletions(-)
 create mode 100755 t/t2501-cwd-empty.sh


base-commit: 88d915a634b449147855041d44875322de2b286d
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1140%2Fnewren%2Fcwd_removal-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1140/newren/cwd_removal-v2
Pull-Request: https://github.com/git/git/pull/1140

Range-diff vs v1:

  1:  0b71996a3b4 !  1:  38a120f5c03 t2501: add various tests for removing the current working directory
     @@ Commit message
          Numerous commands will remove empty working directories, especially if
          they are in the way of placing needed files.  That is normally fine, but
          removing the current working directory can cause confusion for the user
     -    when they run subsequent commands.  Add some tests checking for such
     -    problems.
     +    when they run subsequent commands.  For example, after one git process
     +    has removed the current working directory, git status/log/diff will all
     +    abort with the message:
     +
     +        fatal: Unable to read current working directory: No such file or directory
     +
     +    Since there are several code paths that can result in the current
     +    working directory being removed, add several tests of various different
     +    codepaths that check for the behavior we would instead like to see.
     +    This include a number of new error messages that we will be adding in
     +    subsequent commits as we implement the desired checks.
      
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
  2:  7b0c665fb75 <  -:  ----------- repository, setup: introduce the_cwd
  -:  ----------- >  2:  f6129a8ac9c setup: introduce startup_info->original_cwd
  3:  94bf468be09 !  3:  e74975e83cc unpack-trees: refuse to remove the current working directory
     @@ Metadata
      Author: Elijah Newren <newren@gmail.com>
      
       ## Commit message ##
     -    unpack-trees: refuse to remove the current working directory
     +    unpack-trees: refuse to remove startup_info->original_cwd
      
          In the past, when a directory needs to be removed to make room for a
          file, we have always errored out when that directory contains any
          untracked (but not ignored) files.  Add an extra condition on that: also
     -    error out if the directory is the current working directory.
     +    error out if the directory is the current working directory we inherited
     +    from our parent process.
      
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
     @@ unpack-trees.c: static int verify_clean_subdirectory(const struct cache_entry *c
       	if (i)
       		return add_rejected_path(o, ERROR_NOT_UPTODATE_DIR, ce->name);
      +
     -+	/* Do not lose the current working directory. */
     -+	if (the_cwd && !strcmp(the_cwd, ce->name))
     ++	/* Do not lose startup_info->original_cwd */
     ++	if (startup_info->original_cwd &&
     ++	    !strcmp(startup_info->original_cwd, ce->name))
      +		return add_rejected_path(o, ERROR_CWD_IN_THE_WAY, ce->name);
      +
       	return cnt;
  4:  39830fffd45 !  4:  e06806e3a32 unpack-trees: add special cwd handling
     @@ Commit message
          code from the previous commit in verify_clean_subdirectory().  However,
          when we are preserving untracked files, we would rather any error
          messages about untracked files being in the way take precedence over
     -    error messages about a subdirectory that happens to be the current
     -    working directory being in the way.  But in the
     -    UNPACK_RESET_OVERWRITE_UNTRACKED case, there is no untracked checking to
     -    be done, so we simply add a special case near the top of
     -    verify_absent_1.
     +    error messages about a subdirectory that happens to be the_original_cwd
     +    being in the way.  But in the UNPACK_RESET_OVERWRITE_UNTRACKED case,
     +    there is no untracked checking to be done, so we simply add a special
     +    case near the top of verify_absent_1.
      
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
     @@ unpack-trees.c: static int verify_absent_1(const struct cache_entry *ce,
       		return 0;
       
      +	if (o->reset == UNPACK_RESET_OVERWRITE_UNTRACKED) {
     -+		/* Avoid nuking cwd... */
     -+		if (the_cwd && !strcmp(the_cwd, ce->name))
     ++		/* Avoid nuking startup_info->original_cwd... */
     ++		if (startup_info->original_cwd &&
     ++		    !strcmp(startup_info->original_cwd, ce->name))
      +			return add_rejected_path(o, ERROR_CWD_IN_THE_WAY,
      +						 ce->name);
      +		/* ...but nuke anything else. */
  5:  8a69d2878c9 !  5:  46728f74ea1 symlinks: do not include current working directory in dir removal
     @@ Metadata
      Author: Elijah Newren <newren@gmail.com>
      
       ## Commit message ##
     -    symlinks: do not include current working directory in dir removal
     +    symlinks: do not include startup_info->original_cwd in dir removal
      
          symlinks has a pair of schedule_dir_for_removal() and
          remove_scheduled_dirs() functions that ensure that directories made
          empty by removing other files also themselves get removed.  However, we
     -    want to exclude the current working directory and leave it around so
     -    that subsequent git commands (and non-git commands) that the user runs
     -    afterwards don't cause the user to get confused.
     +    want to exclude startup_info->original_cwd and leave it around.  This
     +    avoids the user getting confused by subsequent git commands (and non-git
     +    commands) that would otherwise report confusing messages about being
     +    unable to read the current working directory.
      
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## symlinks.c ##
     -@@ symlinks.c: static int threaded_has_dirs_only_path(struct cache_def *cache, const char *name
     - 
     - static struct strbuf removal = STRBUF_INIT;
     - 
     -+static int cant_remove(char *dirname)
     -+{
     -+	if (the_cwd && !strcmp(dirname, the_cwd))
     -+		return 1;
     -+	return rmdir(dirname);
     -+}
     -+
     - static void do_remove_scheduled_dirs(int new_len)
     +@@ symlinks.c: static void do_remove_scheduled_dirs(int new_len)
       {
       	while (removal.len > new_len) {
       		removal.buf[removal.len] = '\0';
      -		if (rmdir(removal.buf))
     -+		if (cant_remove(removal.buf))
     ++		if ((startup_info->original_cwd &&
     ++		     !strcmp(removal.buf, startup_info->original_cwd)) ||
     ++		    rmdir(removal.buf))
       			break;
       		do {
       			removal.len--;
     @@ symlinks.c: void schedule_dir_for_removal(const char *name, int len)
       {
       	int match_len, last_slash, i, previous_slash;
       
     -+	if (the_cwd && !strcmp(name, the_cwd))
     ++	if (startup_info->original_cwd &&
     ++	    !strcmp(name, startup_info->original_cwd))
      +		return;	/* Do not remove the current working directory */
      +
       	match_len = last_slash = i =
  6:  69bcaf0aab2 !  6:  01ce9444dae clean: do not attempt to remove current working directory
     @@ Metadata
      Author: Elijah Newren <newren@gmail.com>
      
       ## Commit message ##
     -    clean: do not attempt to remove current working directory
     +    clean: do not attempt to remove startup_info->original_cwd
      
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
     @@ builtin/clean.c: static const char *msg_skip_git_dir = N_("Skipping repository %
       
       enum color_clean {
       	CLEAN_COLOR_RESET = 0,
     +@@ builtin/clean.c: static int remove_dirs(struct strbuf *path, const char *prefix, int force_flag,
     + {
     + 	DIR *dir;
     + 	struct strbuf quoted = STRBUF_INIT;
     ++	struct strbuf realpath = STRBUF_INIT;
     ++	struct strbuf real_ocwd = STRBUF_INIT;
     + 	struct dirent *e;
     + 	int res = 0, ret = 0, gone = 1, original_len = path->len, len;
     + 	struct string_list dels = STRING_LIST_INIT_DUP;
      @@ builtin/clean.c: static int remove_dirs(struct strbuf *path, const char *prefix, int force_flag,
       	strbuf_setlen(path, original_len);
       
     @@ builtin/clean.c: static int remove_dirs(struct strbuf *path, const char *prefix,
      -			quote_path(path->buf, prefix, &quoted, 0);
      -			errno = saved_errno;
      -			warning_errno(_(msg_warn_remove_failed), quoted.buf);
     -+		int prefixlen = prefix ? strlen(prefix) : 0;
     -+		if (prefix &&
     -+		    path->len == prefixlen + 2 &&
     -+		    !strncmp(path->buf, prefix, prefixlen) &&
     -+		    !strcmp(path->buf + prefixlen, "./")) {
     ++		/*
     ++		 * Normalize path components in path->buf, e.g. change '\' to
     ++		 * '/' on Windows.
     ++		 */
     ++		strbuf_realpath(&realpath, path->buf, 1);
     ++
     ++		/*
     ++		 * path and realpath are absolute; for comparison, we want
     ++		 * startup_info->original_cwd to be an absolute path too.  We
     ++		 * can use strbuf_realpath for this.  Also, if original_cwd
     ++		 * started out as the empty string, then it corresponded to
     ++		 * the top of the worktree, which is protected by other means
     ++		 * so we just leave it blank.
     ++		 */
     ++		 if (*startup_info->original_cwd)
     ++			 strbuf_realpath(&real_ocwd,
     ++					 startup_info->original_cwd, 1);
     ++
     ++		if (!strbuf_cmp(&realpath, &real_ocwd)) {
      +			printf("%s", dry_run ? _(msg_would_skip_cwd) : _(msg_skip_cwd));
       			*dir_gone = 0;
      -			ret = 1;
     @@ builtin/clean.c: static int remove_dirs(struct strbuf *path, const char *prefix,
       		}
       	}
       
     +@@ builtin/clean.c: static int remove_dirs(struct strbuf *path, const char *prefix, int force_flag,
     + 			printf(dry_run ?  _(msg_would_remove) : _(msg_remove), dels.items[i].string);
     + 	}
     + out:
     ++	strbuf_release(&realpath);
     ++	strbuf_release(&real_ocwd);
     + 	strbuf_release(&quoted);
     + 	string_list_clear(&dels, 0);
     + 	return ret;
      
       ## t/t2501-cwd-empty.sh ##
      @@ t/t2501-cwd-empty.sh: test_expect_failure 'apply does not remove cwd incidentally' '
  7:  93005b17c28 !  7:  edec0894ca2 stash: do not attempt to remove current working directory
     @@ Metadata
      Author: Elijah Newren <newren@gmail.com>
      
       ## Commit message ##
     -    stash: do not attempt to remove current working directory
     +    stash: do not attempt to remove startup_info->original_cwd
      
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## builtin/stash.c ##
     -@@ builtin/stash.c: static int create_stash(int argc, const char **argv, const char *prefix)
     - 	return ret;
     - }
     - 
     --static int do_push_stash(const struct pathspec *ps, const char *stash_msg, int quiet,
     -+static int do_push_stash(const struct pathspec *ps, const char *stash_msg,
     -+			 const char *prefix, int quiet,
     - 			 int keep_index, int patch_mode, int include_untracked)
     - {
     - 	int ret = 0;
      @@ builtin/stash.c: static int do_push_stash(const struct pathspec *ps, const char *stash_msg, int q
       			struct child_process cp = CHILD_PROCESS_INIT;
       
       			cp.git_cmd = 1;
     -+			if (prefix)
     -+				strvec_pushl(&cp.args, "-C", prefix, NULL);
     ++			if (startup_info->original_cwd &&
     ++			    *startup_info->original_cwd &&
     ++			    !is_absolute_path(startup_info->original_cwd))
     ++				cp.dir = startup_info->original_cwd;
       			strvec_pushl(&cp.args, "clean", "--force",
      -				     "--quiet", "-d", NULL);
      +				     "--quiet", "-d", ":/", NULL);
       			if (include_untracked == INCLUDE_ALL_FILES)
       				strvec_push(&cp.args, "-x");
       			if (run_command(&cp)) {
     -@@ builtin/stash.c: static int push_stash(int argc, const char **argv, const char *prefix,
     - 		die(_("--pathspec-file-nul requires --pathspec-from-file"));
     - 	}
     - 
     --	return do_push_stash(&ps, stash_msg, quiet, keep_index, patch_mode,
     --			     include_untracked);
     -+	return do_push_stash(&ps, stash_msg, prefix, quiet, keep_index,
     -+			     patch_mode, include_untracked);
     - }
     - 
     - static int save_stash(int argc, const char **argv, const char *prefix)
     -@@ builtin/stash.c: static int save_stash(int argc, const char **argv, const char *prefix)
     - 		stash_msg = strbuf_join_argv(&stash_msg_buf, argc, argv, ' ');
     - 
     - 	memset(&ps, 0, sizeof(ps));
     --	ret = do_push_stash(&ps, stash_msg, quiet, keep_index,
     -+	ret = do_push_stash(&ps, stash_msg, prefix, quiet, keep_index,
     - 			    patch_mode, include_untracked);
     - 
     - 	strbuf_release(&stash_msg_buf);
      
       ## t/t2501-cwd-empty.sh ##
      @@ t/t2501-cwd-empty.sh: test_expect_success 'clean does not remove cwd incidentally' '
  8:  a5528cbb14d !  8:  1815f18592b dir: avoid removing the current working directory
     @@ Metadata
      Author: Elijah Newren <newren@gmail.com>
      
       ## Commit message ##
     -    dir: avoid removing the current working directory
     +    dir: avoid incidentally removing the original_cwd in remove_path()
      
     -    dir has a convenient remove_path() helper that will both remove a file
     -    in a directory and remove its containing directory if it becomes empty
     -    as a result of the removal, recursing all the way up.  However, we do
     -    not want the current working directory to be removed, even if it becomes
     -    empty.
     +    Modern git often tries to avoid leaving empty directories around when
     +    removing files.  Originally, it did not bother.  This behavior started
     +    with commit 80e21a9ed809 (merge-recursive::removeFile: remove empty
     +    directories, 2005-11-19), stating the reason simply as:
      
     -    dir also has a remove_dir_recursively() function which appears to mostly
     -    be used to remove metadata directories or temporary directories or
     -    submodules or worktrees.  I am not sure if it needs to be protected
     -    against removing the current working directory, but did so for good
     -    measure.
     +        When the last file in a directory is removed as the result of a
     +        merge, try to rmdir the now-empty directory.
     +
     +    This was reimplemented in C and renamed to remove_path() in commit
     +    e1b3a2cad7 ("Build-in merge-recursive", 2008-02-07), but was still
     +    internal to merge-recursive.
     +
     +    This trend towards removing leading empty directories continued with
     +    commit d9b814cc97f1 (Add builtin "git rm" command, 2006-05-19), which
     +    stated the reasoning as:
     +
     +        The other question is what to do with leading directories. The old
     +        "git rm" script didn't do anything, which is somewhat inconsistent.
     +        This one will actually clean up directories that have become empty
     +        as a result of removing the last file, but maybe we want to have a
     +        flag to decide the behaviour?
     +
     +    remove_path() in dir.c was added in 4a92d1bfb784 (Add remove_path: a
     +    function to remove as much as possible of a path, 2008-09-27), because
     +    it was noted that we had two separate implementations of the same idea
     +    AND both were buggy.  It described the purpose of the function as
     +
     +        a function to remove as much as possible of a path
     +
     +    Why remove as much as possible?  Well, at the time we probably would
     +    have said something like:
     +
     +      * removing leading directories makes things feel tidy
     +      * removing leading directories doesn't hurt anything so long as they
     +        had no files in them.
     +
     +    But I don't believe those reasons hold when the empty directory happens
     +    to be the current working directory we inherited from our parent
     +    process.  Leaving the parent process in a deleted directory can cause
     +    user confusion when subsequent processes fail: any git command, for
     +    example, will immediately fail with
     +
     +        fatal: Unable to read current working directory: No such file or directory
     +
     +    Other commands may similarly get confused.  Modify remove_path() so that
     +    the empty leading directories it also deletes does not include the
     +    current working directory we inherited from our parent process.  I have
     +    looked through every caller of remove_path() in the current codebase to
     +    make sure that all should take this change.
      
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## dir.c ##
     -@@ dir.c: static int remove_dir_recurse(struct strbuf *path, int flag, int *kept_up)
     - 	closedir(dir);
     - 
     - 	strbuf_setlen(path, original_len);
     --	if (!ret && !keep_toplevel && !kept_down)
     --		ret = (!rmdir(path->buf) || errno == ENOENT) ? 0 : -1;
     --	else if (kept_up)
     -+	if (!ret && !keep_toplevel && !kept_down) {
     -+		if (the_cwd && !strcmp(the_cwd, path->buf))
     -+			ret = -1; /* Do not remove current working directory */
     -+		else
     -+			ret = (!rmdir(path->buf) || errno == ENOENT) ? 0 : -1;
     -+	} else if (kept_up)
     - 		/*
     - 		 * report the uplevel that it is not an error that we
     - 		 * did not rmdir() our directory.
      @@ dir.c: int remove_path(const char *name)
       		slash = dirs + (slash - name);
       		do {
       			*slash = '\0';
     -+			if (the_cwd && !strcmp(the_cwd, dirs))
     ++			if (startup_info->original_cwd &&
     ++			    !strcmp(startup_info->original_cwd, dirs))
      +				break;
       		} while (rmdir(dirs) == 0 && (slash = strrchr(dirs, '/')));
       		free(dirs);
       	}
      
     + ## dir.h ##
     +@@ dir.h: int get_sparse_checkout_patterns(struct pattern_list *pl);
     +  */
     + int remove_dir_recursively(struct strbuf *path, int flag);
     + 
     +-/* tries to remove the path with empty directories along it, ignores ENOENT */
     ++/*
     ++ * Tries to remove the path, along with leading empty directories so long as
     ++ * those empty directories are not startup_info->original_cwd.  Ignores
     ++ * ENOENT.
     ++ */
     + int remove_path(const char *path);
     + 
     + int fspathcmp(const char *a, const char *b);
     +
       ## t/t2501-cwd-empty.sh ##
      @@ t/t2501-cwd-empty.sh: test_expect_success 'revert fails if cwd needs to be removed' '
       	test_path_is_dir dirORfile
  -:  ----------- >  9:  adaad7aeaac dir: new flag to remove_dir_recurse() to spare the original_cwd

-- 
gitgitgadget

[PATCH v2 1/9] t2501: add various tests for removing the current working directory

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Numerous commands will remove empty working directories, especially if
they are in the way of placing needed files.  That is normally fine, but
removing the current working directory can cause confusion for the user
when they run subsequent commands.  For example, after one git process
has removed the current working directory, git status/log/diff will all
abort with the message:

    fatal: Unable to read current working directory: No such file or directory

Since there are several code paths that can result in the current
working directory being removed, add several tests of various different
codepaths that check for the behavior we would instead like to see.
This include a number of new error messages that we will be adding in
subsequent commits as we implement the desired checks.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh | 255 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 255 insertions(+)
 create mode 100755 t/t2501-cwd-empty.sh

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
new file mode 100755
index 00000000000..5dfb456a691
--- /dev/null
+++ b/t/t2501-cwd-empty.sh
@@ -0,0 +1,255 @@
+#!/bin/sh
+
+test_description='Test handling of the current working directory becoming empty'
+
+. ./test-lib.sh
+
+test_expect_success setup '
+	test_commit init &&
+	mkdir subdir &&
+	test_commit subdir/file &&
+
+	git branch fd_conflict &&
+
+	mkdir -p foo/bar &&
+	test_commit foo/bar/baz &&
+
+	git revert HEAD &&
+	git tag reverted &&
+
+	git checkout fd_conflict &&
+	git rm subdir/file.t &&
+	echo not-a-directory >dirORfile &&
+	git add dirORfile &&
+	git commit -m dirORfile
+'
+
+test_expect_failure 'checkout does not clean cwd incidentally' '
+	git checkout foo/bar/baz &&
+	test_path_is_dir foo/bar &&
+
+	(
+		cd foo &&
+		git checkout init &&
+		cd ..
+	) &&
+	test_path_is_missing foo/bar/baz &&
+	test_path_is_missing foo/bar &&
+	test_path_is_dir foo
+'
+
+test_expect_failure 'checkout fails if cwd needs to be removed' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir dirORfile &&
+	(
+		cd dirORfile &&
+
+		test_must_fail git checkout fd_conflict 2>../error &&
+		grep "Refusing to remove the current working directory" ../error
+	) &&
+
+	test_path_is_dir dirORfile
+'
+
+test_expect_failure 'reset --hard does not clean cwd incidentally' '
+	git checkout foo/bar/baz &&
+	test_path_is_dir foo/bar &&
+
+	(
+		cd foo &&
+		git reset --hard init &&
+		cd ..
+	) &&
+	test_path_is_missing foo/bar/baz &&
+	test_path_is_missing foo/bar &&
+	test_path_is_dir foo
+'
+
+test_expect_failure 'reset --hard fails if cwd needs to be removed' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir dirORfile &&
+	(
+		cd dirORfile &&
+
+		test_must_fail git reset --hard fd_conflict 2>../error &&
+		grep "Refusing to remove.*the current working directory" ../error
+	) &&
+
+	test_path_is_dir dirORfile
+'
+
+test_expect_failure 'merge does not remove cwd incidentally' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	(
+		cd subdir &&
+		git merge fd_conflict
+	) &&
+
+	test_path_is_missing subdir/file.t &&
+	test_path_is_dir subdir
+'
+
+test_expect_failure 'merge fails if cwd needs to be removed' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir dirORfile &&
+	(
+		cd dirORfile &&
+		test_must_fail git merge fd_conflict 2>../error &&
+		grep "Refusing to remove the current working directory" ../error
+	) &&
+
+	test_path_is_dir dirORfile
+'
+
+test_expect_failure 'cherry-pick does not remove cwd incidentally' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	(
+		cd subdir &&
+		git cherry-pick fd_conflict
+	) &&
+
+	test_path_is_missing subdir/file.t &&
+	test_path_is_dir subdir
+'
+
+test_expect_failure 'cherry-pick fails if cwd needs to be removed' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir dirORfile &&
+	(
+		cd dirORfile &&
+		test_must_fail git cherry-pick fd_conflict 2>../error &&
+		grep "Refusing to remove the current working directory" ../error
+	) &&
+
+	test_path_is_dir dirORfile
+'
+
+test_expect_failure 'rebase does not remove cwd incidentally' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	(
+		cd subdir &&
+		git rebase foo/bar/baz fd_conflict
+	) &&
+
+	test_path_is_missing subdir/file.t &&
+	test_path_is_dir subdir
+'
+
+test_expect_failure 'rebase fails if cwd needs to be removed' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir dirORfile &&
+	(
+		cd dirORfile &&
+		test_must_fail git rebase foo/bar/baz fd_conflict 2>../error &&
+		grep "Refusing to remove the current working directory" ../error
+	) &&
+
+	test_path_is_dir dirORfile
+'
+
+test_expect_failure 'revert does not remove cwd incidentally' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	(
+		cd subdir &&
+		git revert subdir/file
+	) &&
+
+	test_path_is_missing subdir/file.t &&
+	test_path_is_dir subdir
+'
+
+test_expect_failure 'revert fails if cwd needs to be removed' '
+	git checkout fd_conflict &&
+	git revert HEAD &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir dirORfile &&
+	(
+		cd dirORfile &&
+		test_must_fail git revert HEAD 2>../error &&
+		grep "Refusing to remove the current working directory" ../error
+	) &&
+
+	test_path_is_dir dirORfile
+'
+
+test_expect_failure 'rm does not remove cwd incidentally' '
+	test_when_finished "git reset --hard" &&
+	git checkout foo/bar/baz &&
+
+	(
+		cd foo &&
+		git rm bar/baz.t
+	) &&
+
+	test_path_is_missing foo/bar/baz &&
+	test_path_is_missing foo/bar &&
+	test_path_is_dir foo
+'
+
+test_expect_failure 'apply does not remove cwd incidentally' '
+	test_when_finished "git reset --hard" &&
+	git checkout foo/bar/baz &&
+
+	(
+		cd subdir &&
+		git diff subdir/file init | git apply
+	) &&
+
+	test_path_is_missing subdir/file.t &&
+	test_path_is_dir subdir
+'
+
+test_expect_failure 'clean does not remove cwd incidentally' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir empty &&
+	mkdir untracked &&
+	>untracked/random &&
+	(
+		cd untracked &&
+		git clean -fd -e warnings :/ >../warnings &&
+		grep "Refusing to remove current working directory" ../warnings
+	) &&
+
+	test_path_is_missing empty &&
+	test_path_is_missing untracked/random &&
+	test_path_is_dir untracked
+'
+
+test_expect_failure 'stash does not remove cwd incidentally' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir untracked &&
+	>untracked/random &&
+	(
+		cd untracked &&
+		git stash --include-untracked &&
+		git status
+	) &&
+
+	test_path_is_missing untracked/random &&
+	test_path_is_dir untracked
+'
+
+test_done
-- 
gitgitgadget

[PATCH v2 2/9] setup: introduce startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Removing the current working directory causes all subsequent git
commands run from that directory to get confused and fail with a message
about being unable to read the current working directory:

    $ git status
    fatal: Unable to read current working directory: No such file or directory

Non-git commands likely have similar warnings or even errors, e.g.

    $ bash -c 'echo hello'
    shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    hello

This confuses end users, particularly since the command they get the
error from is not the one that caused the problem; the problem came from
the side-effect of some previous command.

We would like to avoid removing the current working directory of our
parent process; towards this end, introduce a new variable,
startup_info->original_cwd, that tracks the current working directory
that we inherited from our parent process.  For convenience of later
comparisons, we prefer that this new variable store a path relative to
the toplevel working directory (thus much like 'prefix'), except without
the trailing slash.

Subsequent commits will make use of this new variable.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 cache.h |  1 +
 git.c   |  2 ++
 setup.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 52 insertions(+)

diff --git a/cache.h b/cache.h
index eba12487b99..d7903c65b57 100644
--- a/cache.h
+++ b/cache.h
@@ -1834,6 +1834,7 @@ void overlay_tree_on_index(struct index_state *istate,
 struct startup_info {
 	int have_repository;
 	const char *prefix;
+	const char *original_cwd;
 };
 extern struct startup_info *startup_info;
 
diff --git a/git.c b/git.c
index 5ff21be21f3..2c98ab48936 100644
--- a/git.c
+++ b/git.c
@@ -866,6 +866,8 @@ int cmd_main(int argc, const char **argv)
 
 	trace_command_performance(argv);
 
+	startup_info->original_cwd = xgetcwd();
+
 	/*
 	 * "git-xxxx" is the same as "git xxxx", but we obviously:
 	 *
diff --git a/setup.c b/setup.c
index 347d7181ae9..f30657723ea 100644
--- a/setup.c
+++ b/setup.c
@@ -432,6 +432,54 @@ void setup_work_tree(void)
 	initialized = 1;
 }
 
+static void setup_original_cwd(void)
+{
+	struct strbuf tmp = STRBUF_INIT;
+	const char *worktree = NULL;
+	int offset = -1;
+
+	/*
+	 * startup_info->original_cwd wass set early on in cmd_main(), unless
+	 * we're an auxiliary tool like git-remote-http or test-tool.
+	 */
+	if (!startup_info->original_cwd)
+		return;
+
+	/*
+	 * startup_info->original_cwd points to the current working
+	 * directory we inherited from our parent process, which is a
+	 * directory we want to avoid incidentally removing.
+	 *
+	 * For convience, we would like to have the path relative to the
+	 * worktree instead of an absolute path.
+	 *
+	 * Yes, startup_info->original_cwd is usually the same as 'prefix',
+	 * but differs in two ways:
+	 *   - prefix has a trailing '/'
+	 *   - if the user passes '-C' to git, that modifies the prefix but
+	 *     not startup_info->original_cwd.
+	 */
+
+	/* Normalize the directory */
+	strbuf_realpath(&tmp, startup_info->original_cwd, 1);
+	free((char*)startup_info->original_cwd);
+	startup_info->original_cwd = strbuf_detach(&tmp, NULL);
+
+	/* Find out if this is in the worktree */
+	worktree = get_git_work_tree();
+	if (worktree)
+		offset = dir_inside_of(startup_info->original_cwd, worktree);
+	if (offset >= 0) {
+		/*
+		 * original_cwd was inside worktree; precompose it just as
+		 * we do prefix so that built up paths will match
+		 */
+		startup_info->original_cwd = \
+			precompose_string_if_needed(startup_info->original_cwd
+						    + offset);
+	}
+}
+
 static int read_worktree_config(const char *var, const char *value, void *vdata)
 {
 	struct repository_format *data = vdata;
@@ -1330,6 +1378,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
 		setenv(GIT_PREFIX_ENVIRONMENT, "", 1);
 	}
 
+	setup_original_cwd();
 
 	strbuf_release(&dir);
 	strbuf_release(&gitdir);
-- 
gitgitgadget

[PATCH v2 3/9] unpack-trees: refuse to remove startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

In the past, when a directory needs to be removed to make room for a
file, we have always errored out when that directory contains any
untracked (but not ignored) files.  Add an extra condition on that: also
error out if the directory is the current working directory we inherited
from our parent process.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh | 10 +++++-----
 unpack-trees.c       | 17 +++++++++++++----
 unpack-trees.h       |  1 +
 3 files changed, 19 insertions(+), 9 deletions(-)

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 5dfb456a691..212676d71c3 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -38,7 +38,7 @@ test_expect_failure 'checkout does not clean cwd incidentally' '
 	test_path_is_dir foo
 '
 
-test_expect_failure 'checkout fails if cwd needs to be removed' '
+test_expect_success 'checkout fails if cwd needs to be removed' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -95,7 +95,7 @@ test_expect_failure 'merge does not remove cwd incidentally' '
 	test_path_is_dir subdir
 '
 
-test_expect_failure 'merge fails if cwd needs to be removed' '
+test_expect_success 'merge fails if cwd needs to be removed' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -122,7 +122,7 @@ test_expect_failure 'cherry-pick does not remove cwd incidentally' '
 	test_path_is_dir subdir
 '
 
-test_expect_failure 'cherry-pick fails if cwd needs to be removed' '
+test_expect_success 'cherry-pick fails if cwd needs to be removed' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -149,7 +149,7 @@ test_expect_failure 'rebase does not remove cwd incidentally' '
 	test_path_is_dir subdir
 '
 
-test_expect_failure 'rebase fails if cwd needs to be removed' '
+test_expect_success 'rebase fails if cwd needs to be removed' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -176,7 +176,7 @@ test_expect_failure 'revert does not remove cwd incidentally' '
 	test_path_is_dir subdir
 '
 
-test_expect_failure 'revert fails if cwd needs to be removed' '
+test_expect_success 'revert fails if cwd needs to be removed' '
 	git checkout fd_conflict &&
 	git revert HEAD &&
 	test_when_finished "git clean -fdx" &&
diff --git a/unpack-trees.c b/unpack-trees.c
index 89ca95ce90b..6bc16f3a714 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -36,6 +36,9 @@ static const char *unpack_plumbing_errors[NB_UNPACK_TREES_WARNING_TYPES] = {
 	/* ERROR_NOT_UPTODATE_DIR */
 	"Updating '%s' would lose untracked files in it",
 
+	/* ERROR_CWD_IN_THE_WAY */
+	"Refusing to remove '%s' since it is the current working directory.",
+
 	/* ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN */
 	"Untracked working tree file '%s' would be overwritten by merge.",
 
@@ -131,6 +134,9 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 	msgs[ERROR_NOT_UPTODATE_DIR] =
 		_("Updating the following directories would lose untracked files in them:\n%s");
 
+	msgs[ERROR_CWD_IN_THE_WAY] =
+		_("Refusing to remove the current working directory:\n%s");
+
 	if (!strcmp(cmd, "checkout"))
 		msg = advice_enabled(ADVICE_COMMIT_BEFORE_MERGE)
 		      ? _("The following untracked working tree files would be removed by checkout:\n%%s"
@@ -2146,10 +2152,7 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,
 		cnt++;
 	}
 
-	/*
-	 * Then we need to make sure that we do not lose a locally
-	 * present file that is not ignored.
-	 */
+	/* Do not lose a locally present file that is not ignored. */
 	pathbuf = xstrfmt("%.*s/", namelen, ce->name);
 
 	memset(&d, 0, sizeof(d));
@@ -2160,6 +2163,12 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,
 	free(pathbuf);
 	if (i)
 		return add_rejected_path(o, ERROR_NOT_UPTODATE_DIR, ce->name);
+
+	/* Do not lose startup_info->original_cwd */
+	if (startup_info->original_cwd &&
+	    !strcmp(startup_info->original_cwd, ce->name))
+		return add_rejected_path(o, ERROR_CWD_IN_THE_WAY, ce->name);
+
 	return cnt;
 }
 
diff --git a/unpack-trees.h b/unpack-trees.h
index 71ffb7eeb0c..efb9edfbb27 100644
--- a/unpack-trees.h
+++ b/unpack-trees.h
@@ -19,6 +19,7 @@ enum unpack_trees_error_types {
 	ERROR_WOULD_OVERWRITE = 0,
 	ERROR_NOT_UPTODATE_FILE,
 	ERROR_NOT_UPTODATE_DIR,
+	ERROR_CWD_IN_THE_WAY,
 	ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN,
 	ERROR_WOULD_LOSE_UNTRACKED_REMOVED,
 	ERROR_BIND_OVERLAP,
-- 
gitgitgadget

[PATCH v2 4/9] unpack-trees: add special cwd handling

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

When running commands such as `git reset --hard` from a subdirectory, if
that subdirectory is in the way of adding needed files, bail with an
error message.

Note that this change looks kind of like it duplicates the new lines of
code from the previous commit in verify_clean_subdirectory().  However,
when we are preserving untracked files, we would rather any error
messages about untracked files being in the way take precedence over
error messages about a subdirectory that happens to be the_original_cwd
being in the way.  But in the UNPACK_RESET_OVERWRITE_UNTRACKED case,
there is no untracked checking to be done, so we simply add a special
case near the top of verify_absent_1.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh |  2 +-
 unpack-trees.c       | 13 +++++++++++--
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 212676d71c3..526d8ec2ee3 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -67,7 +67,7 @@ test_expect_failure 'reset --hard does not clean cwd incidentally' '
 	test_path_is_dir foo
 '
 
-test_expect_failure 'reset --hard fails if cwd needs to be removed' '
+test_expect_success 'reset --hard fails if cwd needs to be removed' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
diff --git a/unpack-trees.c b/unpack-trees.c
index 6bc16f3a714..5852807d2fb 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -2261,10 +2261,19 @@ static int verify_absent_1(const struct cache_entry *ce,
 	int len;
 	struct stat st;
 
-	if (o->index_only || !o->update ||
-	    o->reset == UNPACK_RESET_OVERWRITE_UNTRACKED)
+	if (o->index_only || !o->update)
 		return 0;
 
+	if (o->reset == UNPACK_RESET_OVERWRITE_UNTRACKED) {
+		/* Avoid nuking startup_info->original_cwd... */
+		if (startup_info->original_cwd &&
+		    !strcmp(startup_info->original_cwd, ce->name))
+			return add_rejected_path(o, ERROR_CWD_IN_THE_WAY,
+						 ce->name);
+		/* ...but nuke anything else. */
+		return 0;
+	}
+
 	len = check_leading_path(ce->name, ce_namelen(ce), 0);
 	if (!len)
 		return 0;
-- 
gitgitgadget

[PATCH v2 5/9] symlinks: do not include startup_info->original_cwd in dir removal

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

symlinks has a pair of schedule_dir_for_removal() and
remove_scheduled_dirs() functions that ensure that directories made
empty by removing other files also themselves get removed.  However, we
want to exclude startup_info->original_cwd and leave it around.  This
avoids the user getting confused by subsequent git commands (and non-git
commands) that would otherwise report confusing messages about being
unable to read the current working directory.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 symlinks.c           |  8 +++++++-
 t/t2501-cwd-empty.sh | 12 ++++++------
 2 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/symlinks.c b/symlinks.c
index 5232d02020c..c667baa949b 100644
--- a/symlinks.c
+++ b/symlinks.c
@@ -279,7 +279,9 @@ static void do_remove_scheduled_dirs(int new_len)
 {
 	while (removal.len > new_len) {
 		removal.buf[removal.len] = '\0';
-		if (rmdir(removal.buf))
+		if ((startup_info->original_cwd &&
+		     !strcmp(removal.buf, startup_info->original_cwd)) ||
+		    rmdir(removal.buf))
 			break;
 		do {
 			removal.len--;
@@ -293,6 +295,10 @@ void schedule_dir_for_removal(const char *name, int len)
 {
 	int match_len, last_slash, i, previous_slash;
 
+	if (startup_info->original_cwd &&
+	    !strcmp(name, startup_info->original_cwd))
+		return;	/* Do not remove the current working directory */
+
 	match_len = last_slash = i =
 		longest_path_match(name, len, removal.buf, removal.len,
 				   &previous_slash);
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 526d8ec2ee3..b92e1a9bb16 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -24,7 +24,7 @@ test_expect_success setup '
 	git commit -m dirORfile
 '
 
-test_expect_failure 'checkout does not clean cwd incidentally' '
+test_expect_success 'checkout does not clean cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_path_is_dir foo/bar &&
 
@@ -53,7 +53,7 @@ test_expect_success 'checkout fails if cwd needs to be removed' '
 	test_path_is_dir dirORfile
 '
 
-test_expect_failure 'reset --hard does not clean cwd incidentally' '
+test_expect_success 'reset --hard does not clean cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_path_is_dir foo/bar &&
 
@@ -82,7 +82,7 @@ test_expect_success 'reset --hard fails if cwd needs to be removed' '
 	test_path_is_dir dirORfile
 '
 
-test_expect_failure 'merge does not remove cwd incidentally' '
+test_expect_success 'merge does not remove cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -109,7 +109,7 @@ test_expect_success 'merge fails if cwd needs to be removed' '
 	test_path_is_dir dirORfile
 '
 
-test_expect_failure 'cherry-pick does not remove cwd incidentally' '
+test_expect_success 'cherry-pick does not remove cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -136,7 +136,7 @@ test_expect_success 'cherry-pick fails if cwd needs to be removed' '
 	test_path_is_dir dirORfile
 '
 
-test_expect_failure 'rebase does not remove cwd incidentally' '
+test_expect_success 'rebase does not remove cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -163,7 +163,7 @@ test_expect_success 'rebase fails if cwd needs to be removed' '
 	test_path_is_dir dirORfile
 '
 
-test_expect_failure 'revert does not remove cwd incidentally' '
+test_expect_success 'revert does not remove cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
-- 
gitgitgadget

[PATCH v2 6/9] clean: do not attempt to remove startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/clean.c      | 47 +++++++++++++++++++++++++++++++++++---------
 t/t2501-cwd-empty.sh |  2 +-
 2 files changed, 39 insertions(+), 10 deletions(-)

diff --git a/builtin/clean.c b/builtin/clean.c
index 98a2860409b..68c8d593c77 100644
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@ -36,6 +36,8 @@ static const char *msg_skip_git_dir = N_("Skipping repository %s\n");
 static const char *msg_would_skip_git_dir = N_("Would skip repository %s\n");
 static const char *msg_warn_remove_failed = N_("failed to remove %s");
 static const char *msg_warn_lstat_failed = N_("could not lstat %s\n");
+static const char *msg_skip_cwd = N_("Refusing to remove current working directory\n");
+static const char *msg_would_skip_cwd = N_("Would refuse to remove current working directory\n");
 
 enum color_clean {
 	CLEAN_COLOR_RESET = 0,
@@ -153,6 +155,8 @@ static int remove_dirs(struct strbuf *path, const char *prefix, int force_flag,
 {
 	DIR *dir;
 	struct strbuf quoted = STRBUF_INIT;
+	struct strbuf realpath = STRBUF_INIT;
+	struct strbuf real_ocwd = STRBUF_INIT;
 	struct dirent *e;
 	int res = 0, ret = 0, gone = 1, original_len = path->len, len;
 	struct string_list dels = STRING_LIST_INIT_DUP;
@@ -231,16 +235,39 @@ static int remove_dirs(struct strbuf *path, const char *prefix, int force_flag,
 	strbuf_setlen(path, original_len);
 
 	if (*dir_gone) {
-		res = dry_run ? 0 : rmdir(path->buf);
-		if (!res)
-			*dir_gone = 1;
-		else {
-			int saved_errno = errno;
-			quote_path(path->buf, prefix, &quoted, 0);
-			errno = saved_errno;
-			warning_errno(_(msg_warn_remove_failed), quoted.buf);
+		/*
+		 * Normalize path components in path->buf, e.g. change '\' to
+		 * '/' on Windows.
+		 */
+		strbuf_realpath(&realpath, path->buf, 1);
+
+		/*
+		 * path and realpath are absolute; for comparison, we want
+		 * startup_info->original_cwd to be an absolute path too.  We
+		 * can use strbuf_realpath for this.  Also, if original_cwd
+		 * started out as the empty string, then it corresponded to
+		 * the top of the worktree, which is protected by other means
+		 * so we just leave it blank.
+		 */
+		 if (*startup_info->original_cwd)
+			 strbuf_realpath(&real_ocwd,
+					 startup_info->original_cwd, 1);
+
+		if (!strbuf_cmp(&realpath, &real_ocwd)) {
+			printf("%s", dry_run ? _(msg_would_skip_cwd) : _(msg_skip_cwd));
 			*dir_gone = 0;
-			ret = 1;
+		} else {
+			res = dry_run ? 0 : rmdir(path->buf);
+			if (!res)
+				*dir_gone = 1;
+			else {
+				int saved_errno = errno;
+				quote_path(path->buf, prefix, &quoted, 0);
+				errno = saved_errno;
+				warning_errno(_(msg_warn_remove_failed), quoted.buf);
+				*dir_gone = 0;
+				ret = 1;
+			}
 		}
 	}
 
@@ -250,6 +277,8 @@ static int remove_dirs(struct strbuf *path, const char *prefix, int force_flag,
 			printf(dry_run ?  _(msg_would_remove) : _(msg_remove), dels.items[i].string);
 	}
 out:
+	strbuf_release(&realpath);
+	strbuf_release(&real_ocwd);
 	strbuf_release(&quoted);
 	string_list_clear(&dels, 0);
 	return ret;
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index b92e1a9bb16..20e1b6adede 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -218,7 +218,7 @@ test_expect_failure 'apply does not remove cwd incidentally' '
 	test_path_is_dir subdir
 '
 
-test_expect_failure 'clean does not remove cwd incidentally' '
+test_expect_success 'clean does not remove cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
-- 
gitgitgadget

[PATCH v2 7/9] stash: do not attempt to remove startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/stash.c      | 6 +++++-
 t/t2501-cwd-empty.sh | 2 +-
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/builtin/stash.c b/builtin/stash.c
index a0ccc8654df..50b4875980c 100644
--- a/builtin/stash.c
+++ b/builtin/stash.c
@@ -1485,8 +1485,12 @@ static int do_push_stash(const struct pathspec *ps, const char *stash_msg, int q
 			struct child_process cp = CHILD_PROCESS_INIT;
 
 			cp.git_cmd = 1;
+			if (startup_info->original_cwd &&
+			    *startup_info->original_cwd &&
+			    !is_absolute_path(startup_info->original_cwd))
+				cp.dir = startup_info->original_cwd;
 			strvec_pushl(&cp.args, "clean", "--force",
-				     "--quiet", "-d", NULL);
+				     "--quiet", "-d", ":/", NULL);
 			if (include_untracked == INCLUDE_ALL_FILES)
 				strvec_push(&cp.args, "-x");
 			if (run_command(&cp)) {
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 20e1b6adede..ff4e7cd89fa 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -236,7 +236,7 @@ test_expect_success 'clean does not remove cwd incidentally' '
 	test_path_is_dir untracked
 '
 
-test_expect_failure 'stash does not remove cwd incidentally' '
+test_expect_success 'stash does not remove cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
-- 
gitgitgadget

[PATCH v2 8/9] dir: avoid incidentally removing the original_cwd in remove_path()

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Modern git often tries to avoid leaving empty directories around when
removing files.  Originally, it did not bother.  This behavior started
with commit 80e21a9ed809 (merge-recursive::removeFile: remove empty
directories, 2005-11-19), stating the reason simply as:

    When the last file in a directory is removed as the result of a
    merge, try to rmdir the now-empty directory.

This was reimplemented in C and renamed to remove_path() in commit
e1b3a2cad7 ("Build-in merge-recursive", 2008-02-07), but was still
internal to merge-recursive.

This trend towards removing leading empty directories continued with
commit d9b814cc97f1 (Add builtin "git rm" command, 2006-05-19), which
stated the reasoning as:

    The other question is what to do with leading directories. The old
    "git rm" script didn't do anything, which is somewhat inconsistent.
    This one will actually clean up directories that have become empty
    as a result of removing the last file, but maybe we want to have a
    flag to decide the behaviour?

remove_path() in dir.c was added in 4a92d1bfb784 (Add remove_path: a
function to remove as much as possible of a path, 2008-09-27), because
it was noted that we had two separate implementations of the same idea
AND both were buggy.  It described the purpose of the function as

    a function to remove as much as possible of a path

Why remove as much as possible?  Well, at the time we probably would
have said something like:

  * removing leading directories makes things feel tidy
  * removing leading directories doesn't hurt anything so long as they
    had no files in them.

But I don't believe those reasons hold when the empty directory happens
to be the current working directory we inherited from our parent
process.  Leaving the parent process in a deleted directory can cause
user confusion when subsequent processes fail: any git command, for
example, will immediately fail with

    fatal: Unable to read current working directory: No such file or directory

Other commands may similarly get confused.  Modify remove_path() so that
the empty leading directories it also deletes does not include the
current working directory we inherited from our parent process.  I have
looked through every caller of remove_path() in the current codebase to
make sure that all should take this change.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 dir.c                | 3 +++
 dir.h                | 6 +++++-
 t/t2501-cwd-empty.sh | 4 ++--
 3 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/dir.c b/dir.c
index 94489298f4c..97d6b71c872 100644
--- a/dir.c
+++ b/dir.c
@@ -3327,6 +3327,9 @@ int remove_path(const char *name)
 		slash = dirs + (slash - name);
 		do {
 			*slash = '\0';
+			if (startup_info->original_cwd &&
+			    !strcmp(startup_info->original_cwd, dirs))
+				break;
 		} while (rmdir(dirs) == 0 && (slash = strrchr(dirs, '/')));
 		free(dirs);
 	}
diff --git a/dir.h b/dir.h
index 83f46c0fb4c..d6a5d03bec2 100644
--- a/dir.h
+++ b/dir.h
@@ -504,7 +504,11 @@ int get_sparse_checkout_patterns(struct pattern_list *pl);
  */
 int remove_dir_recursively(struct strbuf *path, int flag);
 
-/* tries to remove the path with empty directories along it, ignores ENOENT */
+/*
+ * Tries to remove the path, along with leading empty directories so long as
+ * those empty directories are not startup_info->original_cwd.  Ignores
+ * ENOENT.
+ */
 int remove_path(const char *path);
 
 int fspathcmp(const char *a, const char *b);
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index ff4e7cd89fa..4362e7b15e5 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -191,7 +191,7 @@ test_expect_success 'revert fails if cwd needs to be removed' '
 	test_path_is_dir dirORfile
 '
 
-test_expect_failure 'rm does not remove cwd incidentally' '
+test_expect_success 'rm does not remove cwd incidentally' '
 	test_when_finished "git reset --hard" &&
 	git checkout foo/bar/baz &&
 
@@ -205,7 +205,7 @@ test_expect_failure 'rm does not remove cwd incidentally' '
 	test_path_is_dir foo
 '
 
-test_expect_failure 'apply does not remove cwd incidentally' '
+test_expect_success 'apply does not remove cwd incidentally' '
 	test_when_finished "git reset --hard" &&
 	git checkout foo/bar/baz &&
 
-- 
gitgitgadget

[PATCH v2 9/9] dir: new flag to remove_dir_recurse() to spare the original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

remove_dir_recurse(), and its non-static wrapper called
remove_dir_recursively(), both take flags for modifying its behavior.
As with the previous commits, we would generally like to protect
the original_cwd, but we want to forced user commands (e.g. 'git rm -rf
...') or other special cases to remove it.  Add a flag for this purpose.
After reading through every caller of remove_dir_recursively() in the
current codebase, there was only one that should be adjusted and that
one only in a very unusual circumstance.  Add a pair of new testcases to
highlight that very specific case involving submodules && --git-dir &&
--work-tree.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/rm.c         |  3 ++-
 dir.c                | 12 +++++++++---
 dir.h                |  3 +++
 t/t2501-cwd-empty.sh | 42 ++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 56 insertions(+), 4 deletions(-)

diff --git a/builtin/rm.c b/builtin/rm.c
index 3d0967cdc11..b4132e5d8ee 100644
--- a/builtin/rm.c
+++ b/builtin/rm.c
@@ -399,12 +399,13 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
 	if (!index_only) {
 		int removed = 0, gitmodules_modified = 0;
 		struct strbuf buf = STRBUF_INIT;
+		int flag = force ? REMOVE_DIR_PURGE_ORIGINAL_CWD : 0;
 		for (i = 0; i < list.nr; i++) {
 			const char *path = list.entry[i].name;
 			if (list.entry[i].is_submodule) {
 				strbuf_reset(&buf);
 				strbuf_addstr(&buf, path);
-				if (remove_dir_recursively(&buf, 0))
+				if (remove_dir_recursively(&buf, flag))
 					die(_("could not remove '%s'"), path);
 
 				removed = 1;
diff --git a/dir.c b/dir.c
index 97d6b71c872..52064345a6b 100644
--- a/dir.c
+++ b/dir.c
@@ -3204,6 +3204,7 @@ static int remove_dir_recurse(struct strbuf *path, int flag, int *kept_up)
 	int ret = 0, original_len = path->len, len, kept_down = 0;
 	int only_empty = (flag & REMOVE_DIR_EMPTY_ONLY);
 	int keep_toplevel = (flag & REMOVE_DIR_KEEP_TOPLEVEL);
+	int purge_original_cwd = (flag & REMOVE_DIR_PURGE_ORIGINAL_CWD);
 	struct object_id submodule_head;
 
 	if ((flag & REMOVE_DIR_KEEP_NESTED_GIT) &&
@@ -3259,9 +3260,14 @@ static int remove_dir_recurse(struct strbuf *path, int flag, int *kept_up)
 	closedir(dir);
 
 	strbuf_setlen(path, original_len);
-	if (!ret && !keep_toplevel && !kept_down)
-		ret = (!rmdir(path->buf) || errno == ENOENT) ? 0 : -1;
-	else if (kept_up)
+	if (!ret && !keep_toplevel && !kept_down) {
+		if (!purge_original_cwd &&
+		    startup_info->original_cwd &&
+		    !strcmp(startup_info->original_cwd, path->buf))
+			ret = -1; /* Do not remove current working directory */
+		else
+			ret = (!rmdir(path->buf) || errno == ENOENT) ? 0 : -1;
+	} else if (kept_up)
 		/*
 		 * report the uplevel that it is not an error that we
 		 * did not rmdir() our directory.
diff --git a/dir.h b/dir.h
index d6a5d03bec2..8e02dfb505d 100644
--- a/dir.h
+++ b/dir.h
@@ -495,6 +495,9 @@ int get_sparse_checkout_patterns(struct pattern_list *pl);
 /* Remove the contents of path, but leave path itself. */
 #define REMOVE_DIR_KEEP_TOPLEVEL 04
 
+/* Remove the_original_cwd too */
+#define REMOVE_DIR_PURGE_ORIGINAL_CWD 0x08
+
 /*
  * Remove path and its contents, recursively. flags is a combination
  * of the above REMOVE_DIR_* constants. Return 0 on success.
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 4362e7b15e5..8da3fb1f5d3 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -252,4 +252,46 @@ test_expect_success 'stash does not remove cwd incidentally' '
 	test_path_is_dir untracked
 '
 
+test_expect_success 'rm -r leaves submodule if cwd inside' '
+	test_when_finished "git reset --hard HEAD~1" &&
+	test_when_finished "rm -rf .git/modules/my_submodule" &&
+
+	git checkout foo/bar/baz &&
+
+	git init my_submodule &&
+	touch my_submodule/file &&
+	git -C my_submodule add file &&
+	git -C my_submodule commit -m "initial commit" &&
+	git submodule add ./my_submodule &&
+	git commit -m "Add the submodule" &&
+
+	(
+		cd my_submodule &&
+		test_must_fail git --git-dir=../.git --work-tree=.. rm -r ../my_submodule/
+	) &&
+
+	test_path_is_dir my_submodule
+'
+
+test_expect_success 'rm -rf removes submodule even if cwd inside' '
+	test_when_finished "git reset --hard HEAD~1" &&
+	test_when_finished "rm -rf .git/modules/my_submodule" &&
+
+	git checkout foo/bar/baz &&
+
+	git init my_submodule &&
+	touch my_submodule/file &&
+	git -C my_submodule add file &&
+	git -C my_submodule commit -m "initial commit" &&
+	git submodule add ./my_submodule &&
+	git commit -m "Add the submodule" &&
+
+	(
+		cd my_submodule &&
+		git --git-dir=../.git --work-tree=.. rm -rf ../my_submodule/
+	) &&
+
+	test_path_is_missing my_submodule
+'
+
 test_done
-- 
gitgitgadget

Re: [PATCH 1/8] t2501: add various tests for removing the current working directory

[Ævar Arnfjörð Bjarmason]

On Mon, Nov 22 2021, Elijah Newren wrote:

> On Mon, Nov 22, 2021 at 6:27 PM Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>>
>> On Mon, Nov 22 2021, Elijah Newren wrote:
>>
>> > On Sun, Nov 21, 2021 at 9:59 AM Ævar Arnfjörð Bjarmason
>> > <avarab@gmail.com> wrote:
>> >>
>> >> On Sun, Nov 21 2021, Elijah Newren via GitGitGadget wrote:
>> >>
>> >> > From: Elijah Newren <newren@gmail.com>
>> >>
>> >> > +test_expect_failure 'checkout fails if cwd needs to be removed' '
>> >> > +     git checkout foo/bar/baz &&
>> >> > +     test_when_finished "git clean -fdx" &&
>> >> > +
>> >> > +     mkdir dirORfile &&
>> >> > +     (
>> >> > +             cd dirORfile &&
>> >> > +
>> >> > +             test_must_fail git checkout fd_conflict 2>../error &&
>> >> > +             grep "Refusing to remove the current working directory" ../error
>> >> > +     ) &&
>> >> > +
>> >> > +     test_path_is_dir dirORfile
>> >>
>> >>
>> >> I'd find this & the rest of this series much easier to understand if we
>> >> started out by positively asserting the current behavior here, and
>> >> didn't test_cmp/grep for erro r messages that don't exist anymore.
>> >
>> > Yeah, this is my fault for my bad commit message.  I stated I was
>> > adding tests checking for the problems of interest, making it sound
>> > like I was testing existing behavior, but I should have stated I was
>> > adding tests with the behavior we'd prefer to have (i.e. basically a
>> > test-driven-development) setup.
>> >
>> > Also, there really wouldn't need to be so many tests for describing
>> > the existing behavior.  It's basically just `git
>> > $OPERATION_THAT_REMOVES_CWD_AS_SIDE_EFFECT` followed by nearly any
>> > other git command will cause the second and later commands to fail
>> > with:
>> >
>> > ```
>> > shell-init: error retrieving current directory: getcwd: cannot access
>> > parent directories: No such file or directory
>> > fatal: Unable to read current working directory: No such file or directory
>> > ```
>> >
>> > However, we do need a lot of tests for corrected behavior, because
>> > there are so many different codepaths we can follow which will lead to
>> > deletion of the current working directory.
>>
>> Currently if I do e.g.:
>>
>>     git checkout master
>>     git clean -dxf
>>     cd perl
>>     git checkout v0.99
>>     cd ../
>>     git clean -dxfn
>>
>> Nothing breaks and I don't end up with an empty perl/ to remove. With
>> these patches we'd either die on the "checkout" (I think) keep the
>> "perl" and have an empty perl/ to report in the "git clean -dxfn" at the
>> end (I'm not sure which, I forgot and haven't re-read this series just
>> now).
>
> You'd have an empty perl/ left after the checkout, which would be
> cleaned up by your final git clean command.

FWIW that's with a -n, just showing that we've got things left.

I.e. just a demo of the new state, in any case not really applicable
since the end-state behavior is that we die...

>> I think changing it anyway might be justifiable, but changing the
>> behavior of things like that tickles my spidey sense a bit. I.e. I can
>> see people having written scripts like that which would break (it's
>> often easier to cd around after globbing than staying at the top-level,
>> then jump back).
>
> I disagree this would break any user scripts.  If people expect a 'git
> checkout' or 'git rebase' to always work, their script is _already_
> broken.

Broken as in they'll upgrade git and a previously working script or
cronjob stops working.

Maybe it's too obscure to worry about, but saying nothing broke that
wasn't working before seems to be moving goalposts to an odd place.

Anyway, I see from
https://lore.kernel.org/git/CABPp-BETpWU9Rkd6pcxh6+gav2QtYnu_5V8ji_1_3kMnVswp1Q@mail.gmail.com/
that you're not really saying that & that we've partially got a split
thread here. Just replying here...

> The presence of any untracked files within the directory
> already results in a hard error -- we refuse to remove non-empty
> directories (unless all files are tracked and unmodified).  This rule
> deserves a clarification: treat the current working directory as
> non-empty since the parent process is likely still parked there..

I'm not going to make this my hill to die on, and honestly wouldn't care
that much if this were changed, aside from wondering how much is really
the x-y problem of setup.c.

But just in the interest of clarity, what I'm talking about is that git
is a *nix tool, and usually just follows *nix FS semantics.

I think we should have a strong bias towards continuing until the OS
tell us to stop for any FS operations. Rather than aborting early in
anticipation of an eventual error, particularly when as I've shown that
eventual error is fixable.

> Further, our own commands are broken/misbehaving due to us not
> erroring out; see e.g.
> https://lore.kernel.org/git/xmqqv93n7q1v.fsf@gitster.g/ and its
> grandparent.  User scripts likely have lurking problems too.

*Nod*. Not to go on about it, but I think those issues are pointing out
things that would be fixed by the the more narrow WIP patch I've got of
"let's just make setup.c not suck in that case" :)

>> So I wonder (especially with Glen's comment in
>> <20211123003958.3978-1-chooglen@google.com>) if this is being done at
>> the right API level.
>
> Glen's comment was interesting, but provided no specifics about how
> the changes I made could cause any possible harm in his case.
> Further, the fact that others are adding extra places doing cleanup
> sound like additional codepaths that should be protected for the exact
> same reasons.  I think we absolutely want my changes affecting his new
> codepaths.

Again, "not my hill" etc., but I think the harm is pretty much that
every time we start adding any sort of unexpected special-sauce on top
of doing stuff until we get a syscall error we're subverting the default
expectations of users.

I think about this as being in the same category as changing "checkout",
"rm" etc. to error out if we detect the target file is open (Windows),
or refusing to remove an executable that we detect as being in use
(AIX).

If you're not on those platforms having a random tool you're scripting
imposing strictures outside of what the OS would do is just another
thing to explain to users. Let's just try and run into the OS error, or
fix our handling of the subsequent error (if any).

>> E.g. maybe it would be better for some commands to
>> ease into this with an advise() or warning() and not a die() or error(),
>> or have the die() be in the likes of "git switch" but not "reset
>> --hard".
>
> The commands that don't need to remove the current working directory
> but just were as a convenience, no longer do and continue on just
> fine.  Commands that need to remove the current working directory in
> order to place a file there will error out, just as they would have
> when trying to remove a directory with untracked files.  I see no need
> to ease into anything here.

Whatever direction we decide on here, I really don't think it makes
sense to think about tracked and modified local content the same as
untracked and unmodified content.

For "tracked and modified" we clearly "own" it, and erroring out is what
everyone would expect us to do. We don't want git to shred user data.

For "untracked and unmodified" (the part after "and" being redundant
:-)) we've got the "precious" edge-case, i.e. we could shred something
the user cares about (which you've also been working on & I very much
applaud).

But I think by any sane definition we'd still classify such content as
less important (since in those cases of shredding it's matching a
.gitignore).

And surely *way lower* in that foodchain is "untracked and
irrelevant". I.e. it's conceivable that someone is making meaningful use
of an empty directory, but it's very unlikely, and in any case your
argument for this change isn't that the directory is important per-se,
but that 

You note in
https://lore.kernel.org/git/CABPp-BETpWU9Rkd6pcxh6+gav2QtYnu_5V8ji_1_3kMnVswp1Q@mail.gmail.com/
that for "rm -r" we already left the directory behind for a "dir" that
had partially tracked/untracked content, that's correct. I'm referring
to the case where "x" is entirely tracked.

>> Or maybe not, just food for thought...
>
> You may also be interested in reading more of the other thread I
> linked to from my cover letter; all these cases were discussed in good
> detail over there.  For example, look at
> https://lore.kernel.org/git/CABPp-BFmU+RaAjq4_0-PSfRgH1Jc63nN0fMuDWk2+iDbdz7CCA@mail.gmail.com/.
> Peff's previous suggestion was to just make the commands error out if
> they'd normally remove the current working directory and require the
> user to run from a different directory instead.  My version lightened
> that requirement so it only errors out if the current working
> directory needed to be removed in order to place something else there
> (and if nothing else was needed to be placed there, then just leaving
> the directory around).

I've read through those, and might have missed something, but it seems
to all really be discussing the shortcomings of setup.c's handling of
failing getcwd() by proxy.

That and references to other 3rd party commands potentially being
equally confused.

In summary I think we should aim more towards:

    $ grep -m 1 Git ../README.md 
    Git - fast, scalable, distributed revision control system

Than:

    $ git grep --no-index -m 1 . ../README.md 
    fatal: Unable to read current working directory: No such file or directory

Which fails due to the setup.c issue I've noted, but also because we
stupidly really translate that to the equivalent of a (in an x/ that
doesn't exist anymore):

    $ grep -m 1 Git ../x/../README.md 
    grep: ../x/../README.md: No such file or directory

Re: [PATCH v2 1/9] t2501: add various tests for removing the current working directory

[Ævar Arnfjörð Bjarmason]

On Thu, Nov 25 2021, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> Numerous commands will remove empty working directories, especially if
> they are in the way of placing needed files.  That is normally fine, but
> removing the current working directory can cause confusion for the user
> when they run subsequent commands.  For example, after one git process
> has removed the current working directory, git status/log/diff will all
> abort with the message:
>
>     fatal: Unable to read current working directory: No such file or directory
>
> Since there are several code paths that can result in the current
> working directory being removed, add several tests of various different
> codepaths that check for the behavior we would instead like to see.
> This include a number of new error messages that we will be adding in
> subsequent commits as we implement the desired checks.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  t/t2501-cwd-empty.sh | 255 +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 255 insertions(+)
>  create mode 100755 t/t2501-cwd-empty.sh
>
> [...]

As noted on v1 (and particularly if we're going to have something like
my proposed alternate "let's not make setup.c die then?" I really think
this should positively assert our existing behavior, and we can then
update this later for the behavior change.

I got that ~working locally, I think you can squash the below in, and
then cherry-pick this version on top in the actual change that adds the
"Refusing to remove" behavior change.

I think this really helps to explain the change, and to make sure we
test existing behavior.

I added a new test for "rm" with the "-f" flag, and by the "~" in
"working" I mean that this worked as I expected aside from the last
couple of tests.

I think that's a bug in your existing test that's hidden by the use of
test_expect_failure here. I.e. in the clean test we'll exit with:

    [...]
    + git clean -fd -e warnings :/
    warning: failed to remove ./: Invalid argument

A "bug" in the sense that this AFAICT would never have removed that
directory anyway since "clean" puts out before, but with your change
we'll catch that earlier and emit a new "error". Also do we need ":/"
there, isn't "." more obvious? In any case, the updated test below shows
that we already punt out in that case, but perhaps it's incomplete. Will
"clean" remove these directories in other cases already?

The "status" then had to be removed from the "stash" test, was it
leftover debugging cruft?

Finally these are quite repetitive. It would be very welcome to factor
these into e.g.:

    test_expect_untracked_dir hard foo/bar/baz -- <command>

Where we just do the common case of "if $1 = hard" we do the setup with
"reset --hard", otherwise the "git clean -dxf" etc. Then if it's the
"foo/bar/baz" case add the 3x "test_path_is_missing" for that etc.

Another issue: You have a "git cmd | other git cmd" with "| git apply"
there, should be moved into two split by a && to not potentially hide an
error on the LHS of the pipe.

I think there's also probably numerous missing tests here when it comes
to how other commands behaved before/after we removed the CWD. E.g. the
case of:

    # in x/
    git rm -r ../x
    git reset ../some-file.txt

Which I noted in another follow-up to v1, i.e. that fails currently due
to our path construction in setup.c. I.e. in terms of "selling" the
change and showing the greater behavior impact, i.e. we from:

    # works
    git rm -r ../x
    # fails
    git reset ../some-file.txt

To:

    # fails (or keeps x/?)
    git rm -r ../x
    # works
    git reset ../some-file.txt

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 5dfb456a691..f1fb8b4a872 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -24,7 +24,7 @@ test_expect_success setup '
 	git commit -m dirORfile
 '
 
-test_expect_failure 'checkout does not clean cwd incidentally' '
+test_expect_success 'checkout cleans cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_path_is_dir foo/bar &&
 
@@ -35,10 +35,10 @@ test_expect_failure 'checkout does not clean cwd incidentally' '
 	) &&
 	test_path_is_missing foo/bar/baz &&
 	test_path_is_missing foo/bar &&
-	test_path_is_dir foo
+	test_path_is_missing foo
 '
 
-test_expect_failure 'checkout fails if cwd needs to be removed' '
+test_expect_success 'checkout if cwd needs to be removed' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -46,14 +46,14 @@ test_expect_failure 'checkout fails if cwd needs to be removed' '
 	(
 		cd dirORfile &&
 
-		test_must_fail git checkout fd_conflict 2>../error &&
-		grep "Refusing to remove the current working directory" ../error
+		git checkout fd_conflict 2>../error &&
+		grep "Switched to branch" ../error
 	) &&
 
-	test_path_is_dir dirORfile
+	test_path_is_file dirORfile
 '
 
-test_expect_failure 'reset --hard does not clean cwd incidentally' '
+test_expect_success 'reset --hard cleans cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_path_is_dir foo/bar &&
 
@@ -64,10 +64,10 @@ test_expect_failure 'reset --hard does not clean cwd incidentally' '
 	) &&
 	test_path_is_missing foo/bar/baz &&
 	test_path_is_missing foo/bar &&
-	test_path_is_dir foo
+	test_path_is_missing foo
 '
 
-test_expect_failure 'reset --hard fails if cwd needs to be removed' '
+test_expect_success 'reset --hard succeeds if cwd needs to be removed' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -75,14 +75,14 @@ test_expect_failure 'reset --hard fails if cwd needs to be removed' '
 	(
 		cd dirORfile &&
 
-		test_must_fail git reset --hard fd_conflict 2>../error &&
-		grep "Refusing to remove.*the current working directory" ../error
+		git reset --hard fd_conflict 2>../error &&
+		test_must_be_empty ../error
 	) &&
 
-	test_path_is_dir dirORfile
+	test_path_is_file dirORfile
 '
 
-test_expect_failure 'merge does not remove cwd incidentally' '
+test_expect_success 'merge removes cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -92,24 +92,24 @@ test_expect_failure 'merge does not remove cwd incidentally' '
 	) &&
 
 	test_path_is_missing subdir/file.t &&
-	test_path_is_dir subdir
+	test_path_is_missing subdir
 '
 
-test_expect_failure 'merge fails if cwd needs to be removed' '
+test_expect_success 'merge succeeds if cwd needs to be removed' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
 	mkdir dirORfile &&
 	(
 		cd dirORfile &&
-		test_must_fail git merge fd_conflict 2>../error &&
-		grep "Refusing to remove the current working directory" ../error
+		git merge fd_conflict 2>../error &&
+		test_must_be_empty ../error
 	) &&
 
-	test_path_is_dir dirORfile
+	test_path_is_file dirORfile
 '
 
-test_expect_failure 'cherry-pick does not remove cwd incidentally' '
+test_expect_success 'cherry-pick removes cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -119,24 +119,23 @@ test_expect_failure 'cherry-pick does not remove cwd incidentally' '
 	) &&
 
 	test_path_is_missing subdir/file.t &&
-	test_path_is_dir subdir
+	test_path_is_missing subdir
 '
 
-test_expect_failure 'cherry-pick fails if cwd needs to be removed' '
+test_expect_success 'cherry-pick suceeds if cwd needs to be removed' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
 	mkdir dirORfile &&
 	(
 		cd dirORfile &&
-		test_must_fail git cherry-pick fd_conflict 2>../error &&
-		grep "Refusing to remove the current working directory" ../error
+		git cherry-pick fd_conflict 2>../error
 	) &&
 
-	test_path_is_dir dirORfile
+	test_path_is_file dirORfile
 '
 
-test_expect_failure 'rebase does not remove cwd incidentally' '
+test_expect_success 'rebase does removes cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -146,24 +145,23 @@ test_expect_failure 'rebase does not remove cwd incidentally' '
 	) &&
 
 	test_path_is_missing subdir/file.t &&
-	test_path_is_dir subdir
+	test_path_is_missing subdir
 '
 
-test_expect_failure 'rebase fails if cwd needs to be removed' '
+test_expect_success 'rebase succeeds if cwd needs to be removed' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
 	mkdir dirORfile &&
 	(
 		cd dirORfile &&
-		test_must_fail git rebase foo/bar/baz fd_conflict 2>../error &&
-		grep "Refusing to remove the current working directory" ../error
+		git rebase foo/bar/baz fd_conflict 2>../error
 	) &&
 
-	test_path_is_dir dirORfile
+	test_path_is_file dirORfile
 '
 
-test_expect_failure 'revert does not remove cwd incidentally' '
+test_expect_success 'revert removes cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -173,10 +171,10 @@ test_expect_failure 'revert does not remove cwd incidentally' '
 	) &&
 
 	test_path_is_missing subdir/file.t &&
-	test_path_is_dir subdir
+	test_path_is_missing subdir
 '
 
-test_expect_failure 'revert fails if cwd needs to be removed' '
+test_expect_success 'revert succeeds if cwd needs to be removed' '
 	git checkout fd_conflict &&
 	git revert HEAD &&
 	test_when_finished "git clean -fdx" &&
@@ -184,14 +182,13 @@ test_expect_failure 'revert fails if cwd needs to be removed' '
 	mkdir dirORfile &&
 	(
 		cd dirORfile &&
-		test_must_fail git revert HEAD 2>../error &&
-		grep "Refusing to remove the current working directory" ../error
+		git revert HEAD 2>../error
 	) &&
 
-	test_path_is_dir dirORfile
+	test_path_is_file dirORfile
 '
 
-test_expect_failure 'rm does not remove cwd incidentally' '
+test_expect_success 'rm removes cwd incidentally' '
 	test_when_finished "git reset --hard" &&
 	git checkout foo/bar/baz &&
 
@@ -202,10 +199,24 @@ test_expect_failure 'rm does not remove cwd incidentally' '
 
 	test_path_is_missing foo/bar/baz &&
 	test_path_is_missing foo/bar &&
-	test_path_is_dir foo
+	test_path_is_missing foo
 '
 
-test_expect_failure 'apply does not remove cwd incidentally' '
+test_expect_success 'rm -f removes cwd incidentally' '
+	test_when_finished "git reset --hard" &&
+	git checkout foo/bar/baz &&
+
+	(
+		cd foo &&
+		git rm -f bar/baz.t
+	) &&
+
+	test_path_is_missing foo/bar/baz &&
+	test_path_is_missing foo/bar &&
+	test_path_is_missing foo
+'
+
+test_expect_success 'apply removes cwd incidentally' '
 	test_when_finished "git reset --hard" &&
 	git checkout foo/bar/baz &&
 
@@ -215,10 +226,10 @@ test_expect_failure 'apply does not remove cwd incidentally' '
 	) &&
 
 	test_path_is_missing subdir/file.t &&
-	test_path_is_dir subdir
+	test_path_is_missing subdir
 '
 
-test_expect_failure 'clean does not remove cwd incidentally' '
+test_expect_success 'clean does not remove cwd incidentally (cannot match pathspec)' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -227,16 +238,15 @@ test_expect_failure 'clean does not remove cwd incidentally' '
 	>untracked/random &&
 	(
 		cd untracked &&
-		git clean -fd -e warnings :/ >../warnings &&
-		grep "Refusing to remove current working directory" ../warnings
+		test_must_fail git clean -fd .
 	) &&
 
-	test_path_is_missing empty &&
+	test_path_is_dir empty &&
 	test_path_is_missing untracked/random &&
 	test_path_is_dir untracked
 '
 
-test_expect_failure 'stash does not remove cwd incidentally' '
+test_expect_success 'stash removes cwd incidentally' '
 	git checkout foo/bar/baz &&
 	test_when_finished "git clean -fdx" &&
 
@@ -244,12 +254,11 @@ test_expect_failure 'stash does not remove cwd incidentally' '
 	>untracked/random &&
 	(
 		cd untracked &&
-		git stash --include-untracked &&
-		git status
+		git stash --include-untracked
 	) &&
 
 	test_path_is_missing untracked/random &&
-	test_path_is_dir untracked
+	test_path_is_missing untracked
 '
 
 test_done

Re: [PATCH v2 2/9] setup: introduce startup_info->original_cwd

[Ævar Arnfjörð Bjarmason]

On Thu, Nov 25 2021, Elijah Newren via GitGitGadget wrote:

> Removing the current working directory causes all subsequent git
> commands run from that directory to get confused and fail with a message
> about being unable to read the current working directory:
>
>     $ git status
>     fatal: Unable to read current working directory: No such file or directory
>
> Non-git commands likely have similar warnings or even errors, e.g.
>
>     $ bash -c 'echo hello'
>     shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
>     hello

Okey, but...

> diff --git a/git.c b/git.c
> index 5ff21be21f3..2c98ab48936 100644
> --- a/git.c
> +++ b/git.c
> @@ -866,6 +866,8 @@ int cmd_main(int argc, const char **argv)
>  
>  	trace_command_performance(argv);
>  
> +	startup_info->original_cwd = xgetcwd();
> +
>  	/*
>  	 * "git-xxxx" is the same as "git xxxx", but we obviously:
>  	 *
> diff --git a/setup.c b/setup.c
> index 347d7181ae9..f30657723ea 100644
> --- a/setup.c
> +++ b/setup.c
> @@ -432,6 +432,54 @@ void setup_work_tree(void)
>  	initialized = 1;
>  }
>  
> +static void setup_original_cwd(void)
> +{
> +	struct strbuf tmp = STRBUF_INIT;
> +	const char *worktree = NULL;
> +	int offset = -1;
> +
> +	/*
> +	 * startup_info->original_cwd wass set early on in cmd_main(), unless
> +	 * we're an auxiliary tool like git-remote-http or test-tool.
> +	 */
> +	if (!startup_info->original_cwd)
> +		return;

This really doesn't belong in those places, by calling xgetcwd() so
early you'll be making commands that don't care about RUN_SETUP at all
die. E.g. with this change:

    mkdir x &&
    cd x &&
    rm -rf ../x &&
    git version

Will die.

So as a follow-up to my comment on this v2's 01/09 I think what's also
missing here is something that does that, but instead of "git version"
does it for all of the "while read builtin" list in t0012-help.sh.

There's other cans of worms to be had here, the discussion downthread of
the not-integrated[1] points to some of that.

I.e. how should the various commands like "ls-remote" that can work
with/without a repo behave here? That one happens to die before/after,
but as noted in that thread that's also a bug.

So anything that adds more really early dying in this area should also
think about the greater goals of what we're doing with RUN_SETUP &
related flags. I.e. does the direction make sense?

If this is moved to some soft recording of the getcwd() (and maybe the
$PWD, as in my WIP change?) shouldn't this go into common-main.c? With
git.c's cmd_main() we're excluding the test helpers and things like
git-daemon. Is that intentional?

I'd also think we'd want to do this much earlier if e.g.  thing like the
trace2 setup wanted to call the remove_directory() call.

Per what Glen & mentioned I'm still not sure if I'm on board with that
idea at all, just running with the ball you put in play :) I.e. if we're
modifying all callers, let's make sure we catch all callers.

Perhaps a better approach is to intercept chdir() instead? And anything
that calls chdir() sets some GIT_* variable so we'll pass "here's our
original cwd" down to sub-processes, like we do with "git -c" for
config.

That would presumably save you the effort of in-advance whitelisting
everything like "git version", we can just move to doing this lazily. If
you're a command that does a RUN_SETUP or otherwise needs chdir() we'll
record the original getcwd(), otherwise...

1. https://lore.kernel.org/git/patch-1.1-fc26c46d39-20210722T140648Z-avarab@gmail.com/

Re: [PATCH v2 3/9] unpack-trees: refuse to remove startup_info->original_cwd

[Ævar Arnfjörð Bjarmason]

On Thu, Nov 25 2021, Elijah Newren via GitGitGadget wrote:

> +	/* ERROR_CWD_IN_THE_WAY */
> +	"Refusing to remove '%s' since it is the current working directory.",
> +
>  	/* ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN */
>  	"Untracked working tree file '%s' would be overwritten by merge.",
>  
> @@ -131,6 +134,9 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
>  	msgs[ERROR_NOT_UPTODATE_DIR] =
>  		_("Updating the following directories would lose untracked files in them:\n%s");
>  
> +	msgs[ERROR_CWD_IN_THE_WAY] =
> +		_("Refusing to remove the current working directory:\n%s");
> +

We end up capitalizing the first letter here, which isn't our usual
style, but I see we do it for all of unpack_plumbing_errors already, and
some related things like the error setup.c would emit.

Still, perhaps it's better to not follow that convention for new
messages?

Re: [PATCH v2 7/9] stash: do not attempt to remove startup_info->original_cwd

[Ævar Arnfjörð Bjarmason]

On Thu, Nov 25 2021, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  builtin/stash.c      | 6 +++++-
>  t/t2501-cwd-empty.sh | 2 +-
>  2 files changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/builtin/stash.c b/builtin/stash.c
> index a0ccc8654df..50b4875980c 100644
> --- a/builtin/stash.c
> +++ b/builtin/stash.c
> @@ -1485,8 +1485,12 @@ static int do_push_stash(const struct pathspec *ps, const char *stash_msg, int q
>  			struct child_process cp = CHILD_PROCESS_INIT;
>  
>  			cp.git_cmd = 1;
> +			if (startup_info->original_cwd &&
> +			    *startup_info->original_cwd &&
> +			    !is_absolute_path(startup_info->original_cwd))
> +				cp.dir = startup_info->original_cwd;
>  			strvec_pushl(&cp.args, "clean", "--force",
> -				     "--quiet", "-d", NULL);
> +				     "--quiet", "-d", ":/", NULL);

I saw you used :/ earlier in 01, and I commented that maybe we can use
".".

Right now I can't remember the difference. I think for tired reviewer
eyes it would really help to amend the commit message to call out what
desired behavior change we're getting to by adding this pathspec.

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Ævar Arnfjörð Bjarmason]

On Wed, Nov 24 2021, Elijah Newren wrote:

> On Wed, Nov 24, 2021 at 6:18 AM Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>>
>> I fleshened this out a bit in this WIP change:
>> https://github.com/avar/git/tree/avar/setup-handle-gone-directory +
>> commit:
>> https://github.com/avar/git/commit/97968518909eef88edba44973b7885d154b7a273
>
> Sweet!
>
>> As noted there there's some caveats, but so far nothing I spotted that
>> can't be overcome. It's particularly painful to test it because of an
>> implementation detail of our test suite, the bin-wrappers are
>> shellscripts, and the very first thing they do is reset $PWD (none of
>> which happens if you run the real "git" binary).
>>
>> That's b.t.w. the "shell-init" error you noted in
>> https://lore.kernel.org/git/CABPp-BEp3OL7F2J_LzqtC-x-8pBUPO8ZR1fTx_6XbqZeOH1kRw@mail.gmail.com/,
>> it's from the bin-wrapper.
>
> To be a bit more precise, the error/warning is from bash rather than
> the bin-wrappers specifically; for example, you can also trigger the
> same shell-init message with a simple `bash -c 'echo hello'` when your
> cwd has been deleted.

*nod*

>> I really wish we didn't have the bin-wrappers...
>
> I'm curious: is this issue the only reason, or am I reading between
> the lines correctly that you have additional reasons for that?

I think it sucks in general :)

This is a much bigger change than what's needed for this topic or the
proposed setup.c change I've got, but what we should be doing in general
with bin-wrapper is to not have them at all under RUNTIME_PREFIX=Y
(which we can make the default), and simply have a built/ directory or
something, the structure of that directory will then 1=1 map to all the
relative paths you'd get if you did a "make install".

Currently in a built checkout you can do:

    ./git version

But not:

    # need to add --exec-path=$PWD
    ./git clone <http url>

Or:

    ./git send-email ...

Which all comes down to sillyness in how we try to discover these paths,
the *.mo files etc., there's also Perl-specific caveats in there.

We'd still need something like a shellscript wrapper for the case of
GIT_DEBUGGER, but we could just move that to being a compile-time flag,
i.e. we'd swap out the "git" binary for a shellscript that chain-loaded
to the "real" binary.

It probably matters a bit for e.g. test suite performance, currently we
spawn an extra shell for every single "git" invocation, but I mostly
care about it for correctness, e.g. in this case of $PWD munging, and
being able to have running a built binary from a checkout Just Work
without the wrapper.

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Philip Oakley]

On 24/11/2021 19:46, Junio C Hamano wrote:
> Philip Oakley <philipoakley@iee.email> writes:
>
>> On 24/11/2021 11:14, Ævar Arnfjörð Bjarmason wrote:
>>> I'm not concerned that you didn't research this change well enough, I
>>> just find it a bit iffy to introduce semantics in git around FS
>>> operations that don't conform with that of POSIX & the underlying OS. My
>>> *nix system happily accepts an "rm -rf" or an "rmdir" of the directory
>>> I'm in, I'd expect git to do the same.
>> Isn't this the same, conceptually, as trying to remove the root
>> directory, but from a Git perspective?
>>
>> i.e. Something along the lines of
>> https://superuser.com/questions/542978/is-it-possible-to-remove-the-root-directory
>> (their answer is 'no' without a special option, default since 2006)
>>
>> If I read the arguments correctly, Elijah is saying that Git shouldn't
>> delete it's own root (cwd) directory, and that it is already implicit
>> within the current Git code.
> I do not think it is about protecting "root"; the series wants
>
>     cd t/ && git rm -r ../t
>
> to leave an empty directory at 't/', because "git rm" was started in
> that directory.
My point was about where the conceptual 'root' (for Git and it's rm
command) was deemed to be.

For instance, can/should we be able to elevate ourselves into a super
project for the deletion? I did notice that a regular `cd / && cd
../../` will happily recycle itself at `/`, rather than bugging out.

Whichever way is decided (cwd, GIT_WORK_TREE, or higher), ensuring that
the documentation is plain and clear , and not just the code, is
important for future readers, to help avoid future confusions.

Philip

Re: [PATCH 8/8] dir: avoid removing the current working directory

[Ævar Arnfjörð Bjarmason]

On Thu, Nov 25 2021, Philip Oakley wrote:

> On 24/11/2021 19:46, Junio C Hamano wrote:
>> Philip Oakley <philipoakley@iee.email> writes:
>>
>>> On 24/11/2021 11:14, Ævar Arnfjörð Bjarmason wrote:
>>>> I'm not concerned that you didn't research this change well enough, I
>>>> just find it a bit iffy to introduce semantics in git around FS
>>>> operations that don't conform with that of POSIX & the underlying OS. My
>>>> *nix system happily accepts an "rm -rf" or an "rmdir" of the directory
>>>> I'm in, I'd expect git to do the same.
>>> Isn't this the same, conceptually, as trying to remove the root
>>> directory, but from a Git perspective?
>>>
>>> i.e. Something along the lines of
>>> https://superuser.com/questions/542978/is-it-possible-to-remove-the-root-directory
>>> (their answer is 'no' without a special option, default since 2006)
>>>
>>> If I read the arguments correctly, Elijah is saying that Git shouldn't
>>> delete it's own root (cwd) directory, and that it is already implicit
>>> within the current Git code.
>> I do not think it is about protecting "root"; the series wants
>>
>>     cd t/ && git rm -r ../t
>>
>> to leave an empty directory at 't/', because "git rm" was started in
>> that directory.
> My point was about where the conceptual 'root' (for Git and it's rm
> command) was deemed to be.

Makes sense, but nothing being discussed here has to do with crossing
repository boundaries so far.

> For instance, can/should we be able to elevate ourselves into a super
> project for the deletion? I did notice that a regular `cd / && cd
> ../../` will happily recycle itself at `/`, rather than bugging out.

This behavior explicitly standardized in POSIX:
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html:

    The special filename dot shall refer to the directory
    specified byits predecessor. The special filename dot-dot
    shall refer to the parent directory of its predecessor
    directory. As a special case, in the root directory, dot-dot
    may refer to the root directory itself.c

The same goes for "foo", "./foo", "./././././foo" etc.

> Whichever way is decided (cwd, GIT_WORK_TREE, or higher), ensuring that
> the documentation is plain and clear , and not just the code, is
> important for future readers, to help avoid future confusions.

Yes, offhand I don't know where we canonically document our behavior of
not crossing repo boundaries, or rather not going "up".

Re: [PATCH v2 2/9] setup: introduce startup_info->original_cwd

[Junio C Hamano]

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> diff --git a/git.c b/git.c
> index 5ff21be21f3..2c98ab48936 100644
> --- a/git.c
> +++ b/git.c
> @@ -866,6 +866,8 @@ int cmd_main(int argc, const char **argv)
>  
>  	trace_command_performance(argv);
>  
> +	startup_info->original_cwd = xgetcwd();

We assume that, unless we are an auxiliary tool like git-remote-http,
we should always have cwd?

> diff --git a/setup.c b/setup.c
> index 347d7181ae9..f30657723ea 100644
> --- a/setup.c
> +++ b/setup.c
> @@ -432,6 +432,54 @@ void setup_work_tree(void)
>  	initialized = 1;
>  }
>  
> +static void setup_original_cwd(void)
> +{
> +	struct strbuf tmp = STRBUF_INIT;
> +	const char *worktree = NULL;
> +	int offset = -1;
> +
> +	/*
> +	 * startup_info->original_cwd wass set early on in cmd_main(), unless

s/wass/was/;

> +	 * we're an auxiliary tool like git-remote-http or test-tool.
> +	 */
> +	if (!startup_info->original_cwd)
> +		return;
> +
> +	/*
> +	 * startup_info->original_cwd points to the current working
> +	 * directory we inherited from our parent process, which is a
> +	 * directory we want to avoid incidentally removing.
> +	 *
> +	 * For convience, we would like to have the path relative to the
> +	 * worktree instead of an absolute path.
> +	 *
> +	 * Yes, startup_info->original_cwd is usually the same as 'prefix',
> +	 * but differs in two ways:
> +	 *   - prefix has a trailing '/'
> +	 *   - if the user passes '-C' to git, that modifies the prefix but
> +	 *     not startup_info->original_cwd.
> +	 */
> +
> +	/* Normalize the directory */
> +	strbuf_realpath(&tmp, startup_info->original_cwd, 1);
> +	free((char*)startup_info->original_cwd);
> +	startup_info->original_cwd = strbuf_detach(&tmp, NULL);
> +
> +	/* Find out if this is in the worktree */
> +	worktree = get_git_work_tree();
> +	if (worktree)
> +		offset = dir_inside_of(startup_info->original_cwd, worktree);
> +	if (offset >= 0) {
> +		/*
> +		 * original_cwd was inside worktree; precompose it just as
> +		 * we do prefix so that built up paths will match
> +		 */
> +		startup_info->original_cwd = \
> +			precompose_string_if_needed(startup_info->original_cwd
> +						    + offset);
> +	}

I wonder if we want to clear the .original_cwd member, so that the
"cwd protection" do not have to worry about anything at all when we
are in a bare repository.

> +}
> +
>  static int read_worktree_config(const char *var, const char *value, void *vdata)
>  {
>  	struct repository_format *data = vdata;
> @@ -1330,6 +1378,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
>  		setenv(GIT_PREFIX_ENVIRONMENT, "", 1);
>  	}
>  
> +	setup_original_cwd();
>  
>  	strbuf_release(&dir);
>  	strbuf_release(&gitdir);

Re: [PATCH v2 2/9] setup: introduce startup_info->original_cwd

[Elijah Newren]

On Thu, Nov 25, 2021 at 2:55 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Thu, Nov 25 2021, Elijah Newren via GitGitGadget wrote:
>
> > Removing the current working directory causes all subsequent git
> > commands run from that directory to get confused and fail with a message
> > about being unable to read the current working directory:
> >
> >     $ git status
> >     fatal: Unable to read current working directory: No such file or directory
> >
> > Non-git commands likely have similar warnings or even errors, e.g.
> >
> >     $ bash -c 'echo hello'
> >     shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
> >     hello
>
> Okey, but...
>
> > diff --git a/git.c b/git.c
> > index 5ff21be21f3..2c98ab48936 100644
> > --- a/git.c
> > +++ b/git.c
> > @@ -866,6 +866,8 @@ int cmd_main(int argc, const char **argv)
> >
> >       trace_command_performance(argv);
> >
> > +     startup_info->original_cwd = xgetcwd();
> > +
> >       /*
> >        * "git-xxxx" is the same as "git xxxx", but we obviously:
> >        *
> > diff --git a/setup.c b/setup.c
> > index 347d7181ae9..f30657723ea 100644
> > --- a/setup.c
> > +++ b/setup.c
> > @@ -432,6 +432,54 @@ void setup_work_tree(void)
> >       initialized = 1;
> >  }
> >
> > +static void setup_original_cwd(void)
> > +{
> > +     struct strbuf tmp = STRBUF_INIT;
> > +     const char *worktree = NULL;
> > +     int offset = -1;
> > +
> > +     /*
> > +      * startup_info->original_cwd wass set early on in cmd_main(), unless
> > +      * we're an auxiliary tool like git-remote-http or test-tool.
> > +      */
> > +     if (!startup_info->original_cwd)
> > +             return;
>
> This really doesn't belong in those places, by calling xgetcwd() so
> early you'll be making commands that don't care about RUN_SETUP at all
> die. E.g. with this change:
>
>     mkdir x &&
>     cd x &&
>     rm -rf ../x &&
>     git version
>
> Will die.

Whoops!  Yeah, I should have used strbuf_getcwd() there.  Thanks for
the careful reading.

> So as a follow-up to my comment on this v2's 01/09 I think what's also
> missing here is something that does that, but instead of "git version"
> does it for all of the "while read builtin" list in t0012-help.sh.

If I had to do some kind of special casing, or if I did potentially
call a path that could die() in the final version, then yes I'd need
to test all of these.  But by using strbuf_getcwd(), I remove the
possibility of die()'ing, so I think testing one of these is good
enough.

> There's other cans of worms to be had here, the discussion downthread of
> the not-integrated[1] points to some of that.
>
> I.e. how should the various commands like "ls-remote" that can work
> with/without a repo behave here? That one happens to die before/after,
> but as noted in that thread that's also a bug.
>
> So anything that adds more really early dying in this area should also
> think about the greater goals of what we're doing with RUN_SETUP &
> related flags. I.e. does the direction make sense?
>
> If this is moved to some soft recording of the getcwd() (and maybe the
> $PWD, as in my WIP change?) shouldn't this go into common-main.c? With
> git.c's cmd_main() we're excluding the test helpers and things like
> git-daemon. Is that intentional?

I didn't think e.g. git-remote-http, git-daemon, etc. mattered.
test-tool is supposed to only be used in the testsuite to my
knowledge.  But fair point, I'll move it to common-main.c.  And, as
per Junio's suggestion in this thread, I'll make it only affect
worktrees.

> I'd also think we'd want to do this much earlier if e.g.  thing like the
> trace2 setup wanted to call the remove_directory() call.

Nah, I'll just put the original_cwd in a temporary, and then if setup
is never called, it won't affect anything else.  And setup will only
make it affect the worktree (and thus won't affect bare repos, nor
affect paths outside the worktree if the user is running from outside
the worktree).

> Per what Glen & mentioned I'm still not sure if I'm on board with that
> idea at all, just running with the ball you put in play :) I.e. if we're
> modifying all callers, let's make sure we catch all callers.
>
> Perhaps a better approach is to intercept chdir() instead? And anything
> that calls chdir() sets some GIT_* variable so we'll pass "here's our
> original cwd" down to sub-processes, like we do with "git -c" for
> config.
>
> That would presumably save you the effort of in-advance whitelisting
> everything like "git version", we can just move to doing this lazily. If
> you're a command that does a RUN_SETUP or otherwise needs chdir() we'll
> record the original getcwd(), otherwise...

There is a chdir_notify(), but of course it isn't called by the
codepath that handles the -C argument.  I believe the above solution
of a temporary handles things just fine, though.

Thanks for the thoughtful and careful review!

Re: [PATCH v2 2/9] setup: introduce startup_info->original_cwd

[Elijah Newren]

On Thu, Nov 25, 2021 at 10:52 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > diff --git a/git.c b/git.c
> > index 5ff21be21f3..2c98ab48936 100644
> > --- a/git.c
> > +++ b/git.c
> > @@ -866,6 +866,8 @@ int cmd_main(int argc, const char **argv)
> >
> >       trace_command_performance(argv);
> >
> > +     startup_info->original_cwd = xgetcwd();
>
> We assume that, unless we are an auxiliary tool like git-remote-http,
> we should always have cwd?

Yeah, Ævar caught this mistake too; I should have used
strbuf_getcwd().  Will fix.

> > diff --git a/setup.c b/setup.c
> > index 347d7181ae9..f30657723ea 100644
> > --- a/setup.c
> > +++ b/setup.c
> > @@ -432,6 +432,54 @@ void setup_work_tree(void)
> >       initialized = 1;
> >  }
> >
> > +static void setup_original_cwd(void)
> > +{
> > +     struct strbuf tmp = STRBUF_INIT;
> > +     const char *worktree = NULL;
> > +     int offset = -1;
> > +
> > +     /*
> > +      * startup_info->original_cwd wass set early on in cmd_main(), unless
>
> s/wass/was/;

Thanks.

> > +      * we're an auxiliary tool like git-remote-http or test-tool.
> > +      */
> > +     if (!startup_info->original_cwd)
> > +             return;
> > +
> > +     /*
> > +      * startup_info->original_cwd points to the current working
> > +      * directory we inherited from our parent process, which is a
> > +      * directory we want to avoid incidentally removing.
> > +      *
> > +      * For convience, we would like to have the path relative to the
> > +      * worktree instead of an absolute path.
> > +      *
> > +      * Yes, startup_info->original_cwd is usually the same as 'prefix',
> > +      * but differs in two ways:
> > +      *   - prefix has a trailing '/'
> > +      *   - if the user passes '-C' to git, that modifies the prefix but
> > +      *     not startup_info->original_cwd.
> > +      */
> > +
> > +     /* Normalize the directory */
> > +     strbuf_realpath(&tmp, startup_info->original_cwd, 1);
> > +     free((char*)startup_info->original_cwd);
> > +     startup_info->original_cwd = strbuf_detach(&tmp, NULL);
> > +
> > +     /* Find out if this is in the worktree */
> > +     worktree = get_git_work_tree();
> > +     if (worktree)
> > +             offset = dir_inside_of(startup_info->original_cwd, worktree);
> > +     if (offset >= 0) {
> > +             /*
> > +              * original_cwd was inside worktree; precompose it just as
> > +              * we do prefix so that built up paths will match
> > +              */
> > +             startup_info->original_cwd = \
> > +                     precompose_string_if_needed(startup_info->original_cwd
> > +                                                 + offset);
> > +     }
>
> I wonder if we want to clear the .original_cwd member, so that the
> "cwd protection" do not have to worry about anything at all when we
> are in a bare repository.

Yeah, I wondered about this a bit and whether it'd even matter either
way if the original_cwd was something outside the working tree.  I can
obviously come up with scenarios, but can't see protecting or not
protecting such directories as mattering; if the user is in a
directory where git internals are stored and deleted, well, they're
already off-roading.

But, since you bring it up, I might as well scope this to just
protecting a directory within the working tree.  Perhaps that'd make
Ævar more comfortable with the change too.  I'll do that.

Re: [PATCH v2 7/9] stash: do not attempt to remove startup_info->original_cwd

[Elijah Newren]

On Thu, Nov 25, 2021 at 2:58 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Thu, Nov 25 2021, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > Signed-off-by: Elijah Newren <newren@gmail.com>
> > ---
> >  builtin/stash.c      | 6 +++++-
> >  t/t2501-cwd-empty.sh | 2 +-
> >  2 files changed, 6 insertions(+), 2 deletions(-)
> >
> > diff --git a/builtin/stash.c b/builtin/stash.c
> > index a0ccc8654df..50b4875980c 100644
> > --- a/builtin/stash.c
> > +++ b/builtin/stash.c
> > @@ -1485,8 +1485,12 @@ static int do_push_stash(const struct pathspec *ps, const char *stash_msg, int q
> >                       struct child_process cp = CHILD_PROCESS_INIT;
> >
> >                       cp.git_cmd = 1;
> > +                     if (startup_info->original_cwd &&
> > +                         *startup_info->original_cwd &&
> > +                         !is_absolute_path(startup_info->original_cwd))
> > +                             cp.dir = startup_info->original_cwd;
> >                       strvec_pushl(&cp.args, "clean", "--force",
> > -                                  "--quiet", "-d", NULL);
> > +                                  "--quiet", "-d", ":/", NULL);
>
> I saw you used :/ earlier in 01, and I commented that maybe we can use
> ".".

I tweaked that other one to avoid :/ a different way.

> Right now I can't remember the difference. I think for tired reviewer
> eyes it would really help to amend the commit message to call out what
> desired behavior change we're getting to by adding this pathspec.

Yeah, I want the ':/' to be explicit that it should clean from the
toplevel of the repository despite starting in a subdirectory; I'll
amend the commit message.

Re: [PATCH v2 3/9] unpack-trees: refuse to remove startup_info->original_cwd

[Elijah Newren]

On Thu, Nov 25, 2021 at 2:57 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Thu, Nov 25 2021, Elijah Newren via GitGitGadget wrote:
>
> > +     /* ERROR_CWD_IN_THE_WAY */
> > +     "Refusing to remove '%s' since it is the current working directory.",
> > +
> >       /* ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN */
> >       "Untracked working tree file '%s' would be overwritten by merge.",
> >
> > @@ -131,6 +134,9 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
> >       msgs[ERROR_NOT_UPTODATE_DIR] =
> >               _("Updating the following directories would lose untracked files in them:\n%s");
> >
> > +     msgs[ERROR_CWD_IN_THE_WAY] =
> > +             _("Refusing to remove the current working directory:\n%s");
> > +
>
> We end up capitalizing the first letter here, which isn't our usual
> style, but I see we do it for all of unpack_plumbing_errors already, and
> some related things like the error setup.c would emit.
>
> Still, perhaps it's better to not follow that convention for new
> messages?

If someone else wants to submit a patch afterwards to clean up all
these error messages, that'd be fine, but I think being internally
inconsistent is more jarring than not following the global rule.  I'll
leave it as is, and let someone else do the wider cleanup.

The overhead of bin-wrappers/ (was: [PATCH 8/8] dir: avoid removing the current working directory)

[Ævar Arnfjörð Bjarmason]

On Thu, Nov 25 2021, Ævar Arnfjörð Bjarmason wrote:

> On Wed, Nov 24 2021, Elijah Newren wrote:
>
>> On Wed, Nov 24, 2021 at 6:18 AM Ævar Arnfjörð Bjarmason
>> <avarab@gmail.com> wrote:
> [...]
>>> I really wish we didn't have the bin-wrappers...
>>
>> I'm curious: is this issue the only reason, or am I reading between
>> the lines correctly that you have additional reasons for that?
>
> I think it sucks in general :)
>
> This is a much bigger change than what's needed for this topic or the
> proposed setup.c change I've got, but what we should be doing in general
> with bin-wrapper is to not have them at all under RUNTIME_PREFIX=Y
> (which we can make the default), and simply have a built/ directory or
> something, the structure of that directory will then 1=1 map to all the
> relative paths you'd get if you did a "make install".
>
> Currently in a built checkout you can do:
>
>     ./git version
>
> But not:
>
>     # need to add --exec-path=$PWD
>     ./git clone <http url>
>
> Or:
>
>     ./git send-email ...
>
> Which all comes down to sillyness in how we try to discover these paths,
> the *.mo files etc., there's also Perl-specific caveats in there.
>
> We'd still need something like a shellscript wrapper for the case of
> GIT_DEBUGGER, but we could just move that to being a compile-time flag,
> i.e. we'd swap out the "git" binary for a shellscript that chain-loaded
> to the "real" binary.
>
> It probably matters a bit for e.g. test suite performance, currently we
> spawn an extra shell for every single "git" invocation, but I mostly
> care about it for correctness, e.g. in this case of $PWD munging, and
> being able to have running a built binary from a checkout Just Work
> without the wrapper.

It also adds quite a bit of overhead to the tests, more than I vaguely
recalled. It differs by test, but here's one that invokes "git" a
lot". Trying with GIT_TEST_INSTALLED is a handy way to avoid the
bin-wrappers:
    
    $ hyperfine -L e ,GIT_TEST_INSTALLED=/home/avar/g/git/installed/bin '{e} ./t3070-wildmatch.sh --root=/dev/shm/git'
    Benchmark 1:  ./t3070-wildmatch.sh --root=/dev/shm/git
      Time (mean ± σ):      4.385 s ±  0.028 s    [User: 3.730 s, System: 0.824 s]
      Range (min … max):    4.340 s …  4.430 s    10 runs
     
    Benchmark 2: GIT_TEST_INSTALLED=/home/avar/g/git/installed/bin ./t3070-wildmatch.sh --root=/dev/shm/git
      Time (mean ± σ):      3.737 s ±  0.036 s    [User: 3.240 s, System: 0.656 s]
      Range (min … max):    3.686 s …  3.796 s    10 runs
     
    Summary
      'GIT_TEST_INSTALLED=/home/avar/g/git/installed/bin ./t3070-wildmatch.sh --root=/dev/shm/git' ran
        1.17 ± 0.01 times faster than ' ./t3070-wildmatch.sh --root=/dev/shm/git'

So a ~15-20% speedup, a more common case of a ~10% speedup:
    
    $ hyperfine -L e ,GIT_TEST_INSTALLED=/home/avar/g/git/installed/bin '{e} ./t0001-init.sh --root=/dev/shm/git'
    Benchmark 1:  ./t0001-init.sh --root=/dev/shm/git
      Time (mean ± σ):     522.8 ms ±  22.8 ms    [User: 438.9 ms, System: 130.2 ms]
      Range (min … max):   502.8 ms … 581.3 ms    10 runs
     
    Benchmark 2: GIT_TEST_INSTALLED=/home/avar/g/git/installed/bin ./t0001-init.sh --root=/dev/shm/git
      Time (mean ± σ):     473.4 ms ±  12.6 ms    [User: 406.7 ms, System: 117.5 ms]
      Range (min … max):   461.0 ms … 502.1 ms    10 runs
     
    Summary
      'GIT_TEST_INSTALLED=/home/avar/g/git/installed/bin ./t0001-init.sh --root=/dev/shm/git' ran
        1.10 ± 0.06 times faster than ' ./t0001-init.sh --root=/dev/shm/git'

And that's on a box where process spawning is really fast, and against a
ramdisk.

[PATCH v3 00/11] Avoid removing the current working directory, even if it becomes empty

[Elijah Newren via GitGitGadget]

Traditionally, if folks run git commands such as checkout or rebase from a
subdirectory, that git command could remove their current working directory
and result in subsequent git and non-git commands either getting confused or
printing messages that confuse the user (e.g. "fatal: Unable to read current
working directory: No such file or directory"). Many commands either
silently avoid removing directories that are not empty (i.e. those that have
untracked or modified files in them)[1], or show an error and abort,
depending on which is more appropriate for the command in question. With
this series, we augment the reasons to avoid removing directories to include
not just has-untracked-or-modified-files, but also to avoid removing the
original_cwd as well.

Peff and Junio provided some good pros/cons, if it helps:

 * Pros: Peff (original suggester of the idea)[2], and Junio[3]
 * Cons: Peff [2, again -- see the "P.S."], and Junio[4]

[1] well, with a few exceptions; see
https://lore.kernel.org/git/pull.1036.v3.git.1632760428.gitgitgadget@gmail.com/
[2] https://lore.kernel.org/git/YS8eEtwQvF7TaLCb@coredump.intra.peff.net/
[3] https://lore.kernel.org/git/xmqqo86elyht.fsf@gitster.g/ [4]
https://lore.kernel.org/git/xmqqo8691gr8.fsf@gitster.g/

Changes since v2:

 * the series is now only about the working tree. So if the original cwd is
   outside the worktree (or we're in a bare repo), then the new code is a
   no-op.
 * fixed ugly early die() possibility (uses strbuf_getcwd() instead of
   xgetcwd())
 * modified the initial tests to show both expected and desired behavior.
   subsequent patches fix the tests. One new patch added at the end which
   simplifies the tests to only check for desired behavior.
 * NULLify startup_info->original_cwd when it matches the toplevel worktree;
   that is already protected and we don't need secondary protection for it.
   This simplified some other codepaths so we don't have to check for
   startup_info->original_cwd == "".
 * clarified some commit messages

Changes since v1:

 * clarified multiple commit messages
 * renamed the_cwd to startup_info->original_cwd to make it clearer that
   it's our parent process'es cwd that really matters, which we inherited at
   program startup. Also pulls it out of the global namespace.
 * Normalize the path for startup_info->original_cwd, and ensure that it's
   actually the original cwd even if -C is passed to git.
 * small code cleanups suggested by René and Ævar
 * split the final patch, which got the most comments into two, one for each
   function being modified; significantly extending the first of the two
   commit messages with a lot of history
 * no longer has a content conflict with so/stash-staged
 * add another value for the flags parameter that remove_dir_recursively()
   takes so that it can opt into either the old or the new behavior. Use
   that for the one special corner case I could find where it matters, and
   add a few tests around it to highlight the utility of the flag.

Elijah Newren (11):
  t2501: add various tests for removing the current working directory
  setup: introduce startup_info->original_cwd
  unpack-trees: refuse to remove startup_info->original_cwd
  unpack-trees: add special cwd handling
  symlinks: do not include startup_info->original_cwd in dir removal
  clean: do not attempt to remove startup_info->original_cwd
  rebase: do not attempt to remove startup_info->original_cwd
  stash: do not attempt to remove startup_info->original_cwd
  dir: avoid incidentally removing the original_cwd in remove_path()
  dir: new flag to remove_dir_recurse() to spare the original_cwd
  t2501: simplify the tests since we can now assume desired behavior

 builtin/clean.c      |  44 +++++--
 builtin/rm.c         |   3 +-
 builtin/stash.c      |   5 +-
 cache.h              |   2 +
 common-main.c        |   4 +
 dir.c                |  15 ++-
 dir.h                |   9 +-
 sequencer.c          |   3 +
 setup.c              |  65 +++++++++++
 symlinks.c           |   8 +-
 t/t2501-cwd-empty.sh | 268 +++++++++++++++++++++++++++++++++++++++++++
 unpack-trees.c       |  30 ++++-
 unpack-trees.h       |   1 +
 13 files changed, 435 insertions(+), 22 deletions(-)
 create mode 100755 t/t2501-cwd-empty.sh


base-commit: 88d915a634b449147855041d44875322de2b286d
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1140%2Fnewren%2Fcwd_removal-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1140/newren/cwd_removal-v3
Pull-Request: https://github.com/git/git/pull/1140

Range-diff vs v2:

  1:  38a120f5c03 !  1:  4b0044656b0 t2501: add various tests for removing the current working directory
     @@ Metadata
       ## Commit message ##
          t2501: add various tests for removing the current working directory
      
     -    Numerous commands will remove empty working directories, especially if
     -    they are in the way of placing needed files.  That is normally fine, but
     -    removing the current working directory can cause confusion for the user
     -    when they run subsequent commands.  For example, after one git process
     -    has removed the current working directory, git status/log/diff will all
     -    abort with the message:
     +    Numerous commands will remove directories left empty as a "convenience"
     +    after removing files within them.  That is normally fine, but removing
     +    the current working directory can be rather inconvenient since it can
     +    cause confusion for the user when they run subsequent commands.  For
     +    example, after one git process has removed the current working
     +    directory, git status/log/diff will all abort with the message:
      
              fatal: Unable to read current working directory: No such file or directory
      
     +    We also have code paths that, when a file needs to be placed where a
     +    directory is (due to e.g. checkout, merge, reset, whatever), will check
     +    if this is okay and error out if not.  These rules include:
     +      * all tracked files under that directory are intended to be removed by
     +        the operation
     +      * none of the tracked files under that directory have uncommitted
     +        modification
     +      * there are no untracked files under that directory
     +    However, if we end up remove the current working directory, we can cause
     +    user confusion when they run subsequent commands, so we would prefer if
     +    there was a fourth rule added to this list: avoid removing the current
     +    working directory.
     +
          Since there are several code paths that can result in the current
          working directory being removed, add several tests of various different
     -    codepaths that check for the behavior we would instead like to see.
     -    This include a number of new error messages that we will be adding in
     -    subsequent commits as we implement the desired checks.
     +    codepaths.  To make it clearer what the difference between the current
     +    behavior and the behavior at the end of the series, code both of them
     +    into the tests and have the appropriate behavior be selected by a flag.
     +    Subsequent commits will toggle the flag from current to desired
     +    behavior.
     +
     +    Also add a few tests suggested during the review of earlier rounds of
     +    this patch series.
      
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
     @@ t/t2501-cwd-empty.sh (new)
      +
      +test_expect_success setup '
      +	test_commit init &&
     -+	mkdir subdir &&
     -+	test_commit subdir/file &&
      +
      +	git branch fd_conflict &&
      +
     @@ t/t2501-cwd-empty.sh (new)
      +	git tag reverted &&
      +
      +	git checkout fd_conflict &&
     -+	git rm subdir/file.t &&
     ++	mkdir dirORfile &&
     ++	test_commit dirORfile/foo &&
     ++
     ++	git rm -r dirORfile &&
      +	echo not-a-directory >dirORfile &&
      +	git add dirORfile &&
     -+	git commit -m dirORfile
     ++	git commit -m dirORfile &&
     ++
     ++	git switch -c df_conflict HEAD~1 &&
     ++	test_commit random_file &&
     ++
     ++	git switch -c undo_fd_conflict fd_conflict &&
     ++	git revert HEAD
      +'
      +
     -+test_expect_failure 'checkout does not clean cwd incidentally' '
     -+	git checkout foo/bar/baz &&
     ++test_incidental_dir_removal () {
     ++	works=$1 &&
     ++	shift &&
     ++
     ++	test_when_finished "git reset --hard" &&
     ++
     ++	git checkout foo/bar/baz^{commit} &&
      +	test_path_is_dir foo/bar &&
      +
      +	(
      +		cd foo &&
     -+		git checkout init &&
     -+		cd ..
     ++		"$@" &&
     ++
     ++		# Although we want pwd & git status to pass, test for existing
     ++		# rather than desired behavior.
     ++		if [[ $works == "success" ]]; then
     ++			pwd -P &&
     ++			git status --porcelain
     ++		else
     ++			! pwd -P &&
     ++			test_might_fail git status --porcelain
     ++		fi
      +	) &&
      +	test_path_is_missing foo/bar/baz &&
      +	test_path_is_missing foo/bar &&
     -+	test_path_is_dir foo
     -+'
      +
     -+test_expect_failure 'checkout fails if cwd needs to be removed' '
     -+	git checkout foo/bar/baz &&
     ++	# Although we want dir to be present, test for existing rather
     ++	# than desired behavior.
     ++	if [[ $works == "success" ]]; then
     ++		test_path_is_dir foo
     ++	else
     ++		test_path_is_missing foo
     ++	fi
     ++}
     ++
     ++test_required_dir_removal () {
     ++	works=$1 &&
     ++	shift &&
     ++
     ++	git checkout df_conflict^{commit} &&
      +	test_when_finished "git clean -fdx" &&
      +
     -+	mkdir dirORfile &&
      +	(
      +		cd dirORfile &&
      +
     -+		test_must_fail git checkout fd_conflict 2>../error &&
     -+		grep "Refusing to remove the current working directory" ../error
     ++		# We'd like for the command to fail (much as it would if there
     ++		# was an untracked file there), and for pwd & git status to
     ++		# succeed afterwards.  But test for existing rather than
     ++		# desired behavior.
     ++		if [[ $works == "success" ]]; then
     ++			test_must_fail "$@" 2>../error &&
     ++			grep "Refusing to remove.*current working directory" ../error &&
     ++			pwd -P &&
     ++			git status --porcelain
     ++		else
     ++			"$@" &&
     ++			! pwd -P &&
     ++			test_might_fail git status --porcelain
     ++		fi
      +	) &&
      +
     -+	test_path_is_dir dirORfile
     ++	# Although we want dirORfile to be present, test for existing rather
     ++	# than desired behavior.
     ++	if [[ $works == "success" ]]; then
     ++		test_path_is_dir dirORfile
     ++	else
     ++		test_path_is_file dirORfile
     ++	fi
     ++}
     ++
     ++test_expect_success 'checkout does not clean cwd incidentally' '
     ++	test_incidental_dir_removal failure git checkout init
      +'
      +
     -+test_expect_failure 'reset --hard does not clean cwd incidentally' '
     -+	git checkout foo/bar/baz &&
     -+	test_path_is_dir foo/bar &&
     ++test_expect_success 'checkout fails if cwd needs to be removed' '
     ++	test_required_dir_removal failure git checkout fd_conflict
     ++'
      +
     -+	(
     -+		cd foo &&
     -+		git reset --hard init &&
     -+		cd ..
     -+	) &&
     -+	test_path_is_missing foo/bar/baz &&
     -+	test_path_is_missing foo/bar &&
     -+	test_path_is_dir foo
     ++test_expect_success 'reset --hard does not clean cwd incidentally' '
     ++	test_incidental_dir_removal failure git reset --hard init
     ++'
     ++
     ++test_expect_success 'reset --hard fails if cwd needs to be removed' '
     ++	test_required_dir_removal failure git reset --hard fd_conflict
     ++'
     ++
     ++test_expect_success 'merge does not clean cwd incidentally' '
     ++	test_incidental_dir_removal failure git merge reverted
      +'
      +
     -+test_expect_failure 'reset --hard fails if cwd needs to be removed' '
     ++# This file uses some simple merges where
     ++#   Base: 'dirORfile/' exists
     ++#   Side1: random other file changed
     ++#   Side2: 'dirORfile/' removed, 'dirORfile' added
     ++# this should resolve cleanly, but merge-recursive throws merge conflicts
     ++# because it's dumb.  Add a special test for checking merge-recursive (and
     ++# merge-ort), then after this just hard require ort for all remaining tests.
     ++#
     ++test_expect_success 'merge fails if cwd needs to be removed; recursive friendly' '
      +	git checkout foo/bar/baz &&
      +	test_when_finished "git clean -fdx" &&
      +
     @@ t/t2501-cwd-empty.sh (new)
      +	(
      +		cd dirORfile &&
      +
     -+		test_must_fail git reset --hard fd_conflict 2>../error &&
     -+		grep "Refusing to remove.*the current working directory" ../error
     ++		# We would rather this failed, but we test for existing
     ++		# rather than desired behavior
     ++		git merge fd_conflict 2>../error
      +	) &&
      +
     -+	test_path_is_dir dirORfile
     ++	## Here is the behavior we would rather have:
     ++	#test_path_is_dir dirORfile &&
     ++	#grep "Refusing to remove the current working directory" error
     ++	## But instead we test for existing behavior
     ++	test_path_is_file dirORfile &&
     ++	test_must_be_empty error
      +'
      +
     -+test_expect_failure 'merge does not remove cwd incidentally' '
     -+	git checkout foo/bar/baz &&
     -+	test_when_finished "git clean -fdx" &&
     ++GIT_TEST_MERGE_ALGORITHM=ort
      +
     -+	(
     -+		cd subdir &&
     -+		git merge fd_conflict
     -+	) &&
     ++test_expect_success 'merge fails if cwd needs to be removed' '
     ++	test_required_dir_removal failure git merge fd_conflict
     ++'
      +
     -+	test_path_is_missing subdir/file.t &&
     -+	test_path_is_dir subdir
     ++test_expect_success 'cherry-pick does not clean cwd incidentally' '
     ++	test_incidental_dir_removal failure git cherry-pick reverted
      +'
      +
     -+test_expect_failure 'merge fails if cwd needs to be removed' '
     -+	git checkout foo/bar/baz &&
     -+	test_when_finished "git clean -fdx" &&
     ++test_expect_success 'cherry-pick fails if cwd needs to be removed' '
     ++	test_required_dir_removal failure git cherry-pick fd_conflict
     ++'
      +
     -+	mkdir dirORfile &&
     -+	(
     -+		cd dirORfile &&
     -+		test_must_fail git merge fd_conflict 2>../error &&
     -+		grep "Refusing to remove the current working directory" ../error
     -+	) &&
     ++test_expect_success 'rebase does not clean cwd incidentally' '
     ++	test_incidental_dir_removal failure git rebase reverted
     ++'
      +
     -+	test_path_is_dir dirORfile
     ++test_expect_success 'rebase fails if cwd needs to be removed' '
     ++	test_required_dir_removal failure git rebase fd_conflict
      +'
      +
     -+test_expect_failure 'cherry-pick does not remove cwd incidentally' '
     -+	git checkout foo/bar/baz &&
     -+	test_when_finished "git clean -fdx" &&
     ++test_expect_success 'revert does not clean cwd incidentally' '
     ++	test_incidental_dir_removal failure git revert HEAD
     ++'
      +
     -+	(
     -+		cd subdir &&
     -+		git cherry-pick fd_conflict
     -+	) &&
     ++test_expect_success 'revert fails if cwd needs to be removed' '
     ++	test_required_dir_removal failure git revert undo_fd_conflict
     ++'
      +
     -+	test_path_is_missing subdir/file.t &&
     -+	test_path_is_dir subdir
     ++test_expect_success 'rm does not clean cwd incidentally' '
     ++	test_incidental_dir_removal failure git rm bar/baz.t
      +'
      +
     -+test_expect_failure 'cherry-pick fails if cwd needs to be removed' '
     -+	git checkout foo/bar/baz &&
     -+	test_when_finished "git clean -fdx" &&
     ++test_expect_success 'apply does not remove cwd incidentally' '
     ++	git diff HEAD HEAD~1 >patch &&
     ++	test_incidental_dir_removal failure git apply ../patch
     ++'
      +
     -+	mkdir dirORfile &&
     -+	(
     -+		cd dirORfile &&
     -+		test_must_fail git cherry-pick fd_conflict 2>../error &&
     -+		grep "Refusing to remove the current working directory" ../error
     -+	) &&
     ++test_incidental_untracked_dir_removal () {
     ++	works=$1 &&
     ++	shift &&
      +
     -+	test_path_is_dir dirORfile
     -+'
     ++	test_when_finished "git reset --hard" &&
      +
     -+test_expect_failure 'rebase does not remove cwd incidentally' '
     -+	git checkout foo/bar/baz &&
     -+	test_when_finished "git clean -fdx" &&
     ++	git checkout foo/bar/baz^{commit} &&
     ++	mkdir -p untracked &&
     ++	mkdir empty
     ++	>untracked/random &&
      +
      +	(
     -+		cd subdir &&
     -+		git rebase foo/bar/baz fd_conflict
     ++		cd untracked &&
     ++		"$@" &&
     ++
     ++		# Although we want pwd & git status to pass, test for existing
     ++		# rather than desired behavior.
     ++		if [[ $works == "success" ]]; then
     ++			pwd -P &&
     ++			git status --porcelain
     ++		else
     ++			! pwd -P &&
     ++			test_might_fail git status --porcelain
     ++		fi
      +	) &&
     ++	test_path_is_missing empty &&
     ++	test_path_is_missing untracked/random &&
      +
     -+	test_path_is_missing subdir/file.t &&
     -+	test_path_is_dir subdir
     ++	# Although we want dir to be present, test for existing rather
     ++	# than desired behavior.
     ++	if [[ $works == "success" ]]; then
     ++		test_path_is_dir untracked
     ++	else
     ++		test_path_is_missing untracked
     ++	fi
     ++}
     ++
     ++test_expect_success 'clean does not remove cwd incidentally' '
     ++	test_incidental_untracked_dir_removal failure \
     ++		git -C .. clean -fd -e warnings . >warnings
      +'
      +
     -+test_expect_failure 'rebase fails if cwd needs to be removed' '
     -+	git checkout foo/bar/baz &&
     -+	test_when_finished "git clean -fdx" &&
     -+
     -+	mkdir dirORfile &&
     -+	(
     -+		cd dirORfile &&
     -+		test_must_fail git rebase foo/bar/baz fd_conflict 2>../error &&
     -+		grep "Refusing to remove the current working directory" ../error
     -+	) &&
     -+
     -+	test_path_is_dir dirORfile
     ++test_expect_success 'stash does not remove cwd incidentally' '
     ++	test_incidental_untracked_dir_removal failure \
     ++		git stash --include-untracked
      +'
      +
     -+test_expect_failure 'revert does not remove cwd incidentally' '
     -+	git checkout foo/bar/baz &&
     -+	test_when_finished "git clean -fdx" &&
     ++test_expect_success '`rm -rf dir` only removes a subset of dir' '
     ++	test_when_finished "rm -rf a/" &&
     ++
     ++	mkdir -p a/b/c &&
     ++	>a/b/c/untracked &&
     ++	>a/b/c/tracked &&
     ++	git add a/b/c/tracked &&
      +
      +	(
     -+		cd subdir &&
     -+		git revert subdir/file
     ++		cd a/b &&
     ++		git rm -rf ../b
      +	) &&
      +
     -+	test_path_is_missing subdir/file.t &&
     -+	test_path_is_dir subdir
     ++	test_path_is_dir a/b &&
     ++	test_path_is_missing a/b/c/tracked &&
     ++	test_path_is_file a/b/c/untracked
      +'
      +
     -+test_expect_failure 'revert fails if cwd needs to be removed' '
     -+	git checkout fd_conflict &&
     -+	git revert HEAD &&
     -+	test_when_finished "git clean -fdx" &&
     ++test_expect_success '`rm -rf dir` even with only tracked files will remove something else' '
     ++	test_when_finished "rm -rf a/" &&
     ++
     ++	mkdir -p a/b/c &&
     ++	>a/b/c/tracked &&
     ++	git add a/b/c/tracked &&
      +
     -+	mkdir dirORfile &&
      +	(
     -+		cd dirORfile &&
     -+		test_must_fail git revert HEAD 2>../error &&
     -+		grep "Refusing to remove the current working directory" ../error
     ++		cd a/b &&
     ++		git rm -rf ../b
      +	) &&
      +
     -+	test_path_is_dir dirORfile
     ++	test_path_is_missing a/b/c/tracked &&
     ++	## We would prefer if a/b was still present, though empty, since it
     ++	## was the current working directory
     ++	#test_path_is_dir a/b
     ++	## But the current behavior is that it not only deletes the directory
     ++	## a/b as requested, but also goes and deletes a
     ++	test_path_is_missing a
      +'
      +
     -+test_expect_failure 'rm does not remove cwd incidentally' '
     -+	test_when_finished "git reset --hard" &&
     -+	git checkout foo/bar/baz &&
     -+
     ++test_expect_success 'git version continues working from a deleted dir' '
     ++	mkdir tmp &&
      +	(
     -+		cd foo &&
     -+		git rm bar/baz.t
     -+	) &&
     -+
     -+	test_path_is_missing foo/bar/baz &&
     -+	test_path_is_missing foo/bar &&
     -+	test_path_is_dir foo
     ++		cd tmp &&
     ++		rm -rf ../tmp &&
     ++		git version
     ++	)
      +'
      +
     -+test_expect_failure 'apply does not remove cwd incidentally' '
     -+	test_when_finished "git reset --hard" &&
     -+	git checkout foo/bar/baz &&
     ++test_submodule_removal () {
     ++	path_status=$1 &&
     ++	shift &&
      +
     -+	(
     -+		cd subdir &&
     -+		git diff subdir/file init | git apply
     -+	) &&
     ++	test_status=
     ++	test $path_status = dir && test_status=test_must_fail
      +
     -+	test_path_is_missing subdir/file.t &&
     -+	test_path_is_dir subdir
     -+'
     ++	# Actually, while path_status == dir && test_status=test_must_fail
     ++	# reflect our desired behavior, current behavior is:
     ++	path_status=missing
     ++	test_status=
     ++
     ++	test_when_finished "git reset --hard HEAD~1" &&
     ++	test_when_finished "rm -rf .git/modules/my_submodule" &&
      +
     -+test_expect_failure 'clean does not remove cwd incidentally' '
      +	git checkout foo/bar/baz &&
     -+	test_when_finished "git clean -fdx" &&
      +
     -+	mkdir empty &&
     -+	mkdir untracked &&
     -+	>untracked/random &&
     ++	git init my_submodule &&
     ++	touch my_submodule/file &&
     ++	git -C my_submodule add file &&
     ++	git -C my_submodule commit -m "initial commit" &&
     ++	git submodule add ./my_submodule &&
     ++	git commit -m "Add the submodule" &&
     ++
      +	(
     -+		cd untracked &&
     -+		git clean -fd -e warnings :/ >../warnings &&
     -+		grep "Refusing to remove current working directory" ../warnings
     ++		cd my_submodule &&
     ++		$test_status "$@"
      +	) &&
      +
     -+	test_path_is_missing empty &&
     -+	test_path_is_missing untracked/random &&
     -+	test_path_is_dir untracked
     -+'
     ++	test_path_is_${path_status} my_submodule
     ++}
      +
     -+test_expect_failure 'stash does not remove cwd incidentally' '
     -+	git checkout foo/bar/baz &&
     -+	test_when_finished "git clean -fdx" &&
     ++test_expect_success 'rm -r with -C leaves submodule if cwd inside' '
     ++	test_submodule_removal dir git -C .. rm -r my_submodule/
     ++'
      +
     -+	mkdir untracked &&
     -+	>untracked/random &&
     -+	(
     -+		cd untracked &&
     -+		git stash --include-untracked &&
     -+		git status
     -+	) &&
     ++test_expect_success 'rm -r leaves submodule if cwd inside' '
     ++	test_submodule_removal dir \
     ++		git --git-dir=../.git --work-tree=.. rm -r ../my_submodule/
     ++'
      +
     -+	test_path_is_missing untracked/random &&
     -+	test_path_is_dir untracked
     ++test_expect_success 'rm -rf removes submodule even if cwd inside' '
     ++	test_submodule_removal missing \
     ++		git --git-dir=../.git --work-tree=.. rm -rf ../my_submodule/
      +'
      +
      +test_done
  2:  f6129a8ac9c !  2:  200ddece05d setup: introduce startup_info->original_cwd
     @@ cache.h: void overlay_tree_on_index(struct index_state *istate,
      +	const char *original_cwd;
       };
       extern struct startup_info *startup_info;
     ++extern const char *tmp_original_cwd;
       
     + /* merge.c */
     + struct commit_list;
      
     - ## git.c ##
     -@@ git.c: int cmd_main(int argc, const char **argv)
     + ## common-main.c ##
     +@@ common-main.c: static void restore_sigpipe_to_default(void)
     + int main(int argc, const char **argv)
     + {
     + 	int result;
     ++	struct strbuf tmp = STRBUF_INIT;
     + 
     + 	trace2_initialize_clock();
       
     - 	trace_command_performance(argv);
     +@@ common-main.c: int main(int argc, const char **argv)
     + 	trace2_cmd_start(argv);
     + 	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
       
     -+	startup_info->original_cwd = xgetcwd();
     ++	if (!strbuf_getcwd(&tmp))
     ++		tmp_original_cwd = strbuf_detach(&tmp, NULL);
      +
     - 	/*
     - 	 * "git-xxxx" is the same as "git xxxx", but we obviously:
     - 	 *
     + 	result = cmd_main(argc, argv);
     + 
     + 	trace2_cmd_exit(result);
      
       ## setup.c ##
     +@@ setup.c: static int work_tree_config_is_bogus;
     + 
     + static struct startup_info the_startup_info;
     + struct startup_info *startup_info = &the_startup_info;
     ++const char *tmp_original_cwd;
     + 
     + /*
     +  * The input parameter must contain an absolute path, and it must already be
      @@ setup.c: void setup_work_tree(void)
       	initialized = 1;
       }
     @@ setup.c: void setup_work_tree(void)
      +	const char *worktree = NULL;
      +	int offset = -1;
      +
     -+	/*
     -+	 * startup_info->original_cwd wass set early on in cmd_main(), unless
     -+	 * we're an auxiliary tool like git-remote-http or test-tool.
     -+	 */
     -+	if (!startup_info->original_cwd)
     ++	if (!tmp_original_cwd)
      +		return;
      +
      +	/*
      +	 * startup_info->original_cwd points to the current working
      +	 * directory we inherited from our parent process, which is a
     -+	 * directory we want to avoid incidentally removing.
     ++	 * directory we want to avoid removing.
      +	 *
      +	 * For convience, we would like to have the path relative to the
      +	 * worktree instead of an absolute path.
     @@ setup.c: void setup_work_tree(void)
      +	 */
      +
      +	/* Normalize the directory */
     -+	strbuf_realpath(&tmp, startup_info->original_cwd, 1);
     -+	free((char*)startup_info->original_cwd);
     ++	strbuf_realpath(&tmp, tmp_original_cwd, 1);
     ++	free((char*)tmp_original_cwd);
     ++	tmp_original_cwd = NULL;
      +	startup_info->original_cwd = strbuf_detach(&tmp, NULL);
      +
     -+	/* Find out if this is in the worktree */
     ++	/*
     ++	 * Get our worktree; we only protect the current working directory
     ++	 * if it's in the worktree.
     ++	 */
      +	worktree = get_git_work_tree();
     -+	if (worktree)
     -+		offset = dir_inside_of(startup_info->original_cwd, worktree);
     ++	if (!worktree)
     ++		goto no_prevention_needed;
     ++
     ++	offset = dir_inside_of(startup_info->original_cwd, worktree);
      +	if (offset >= 0) {
      +		/*
     ++		 * If startup_info->original_cwd == worktree, that is already
     ++		 * protected and we don't need original_cwd as a secondary
     ++		 * protection measure.
     ++		 */
     ++		if (!*(startup_info->original_cwd + offset))
     ++			goto no_prevention_needed;
     ++
     ++		/*
      +		 * original_cwd was inside worktree; precompose it just as
      +		 * we do prefix so that built up paths will match
      +		 */
     @@ setup.c: void setup_work_tree(void)
      +			precompose_string_if_needed(startup_info->original_cwd
      +						    + offset);
      +	}
     ++	return;
     ++
     ++no_prevention_needed:
     ++	free((char*)startup_info->original_cwd);
     ++	startup_info->original_cwd = NULL;
      +}
      +
       static int read_worktree_config(const char *var, const char *value, void *vdata)
  3:  e74975e83cc !  3:  68ae90546fe unpack-trees: refuse to remove startup_info->original_cwd
     @@ Commit message
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## t/t2501-cwd-empty.sh ##
     -@@ t/t2501-cwd-empty.sh: test_expect_failure 'checkout does not clean cwd incidentally' '
     - 	test_path_is_dir foo
     +@@ t/t2501-cwd-empty.sh: test_expect_success 'checkout does not clean cwd incidentally' '
       '
       
     --test_expect_failure 'checkout fails if cwd needs to be removed' '
     -+test_expect_success 'checkout fails if cwd needs to be removed' '
     - 	git checkout foo/bar/baz &&
     - 	test_when_finished "git clean -fdx" &&
     + test_expect_success 'checkout fails if cwd needs to be removed' '
     +-	test_required_dir_removal failure git checkout fd_conflict
     ++	test_required_dir_removal success git checkout fd_conflict
     + '
     + 
     + test_expect_success 'reset --hard does not clean cwd incidentally' '
     +@@ t/t2501-cwd-empty.sh: test_expect_success 'merge fails if cwd needs to be removed; recursive friendly'
     + 	(
     + 		cd dirORfile &&
     + 
     +-		# We would rather this failed, but we test for existing
     +-		# rather than desired behavior
     +-		git merge fd_conflict 2>../error
     ++		test_must_fail git merge fd_conflict 2>../error
     + 	) &&
       
     -@@ t/t2501-cwd-empty.sh: test_expect_failure 'merge does not remove cwd incidentally' '
     - 	test_path_is_dir subdir
     +-	## Here is the behavior we would rather have:
     +-	#test_path_is_dir dirORfile &&
     +-	#grep "Refusing to remove the current working directory" error
     +-	## But instead we test for existing behavior
     +-	test_path_is_file dirORfile &&
     +-	test_must_be_empty error
     ++	test_path_is_dir dirORfile &&
     ++	grep "Refusing to remove the current working directory" error
       '
       
     --test_expect_failure 'merge fails if cwd needs to be removed' '
     -+test_expect_success 'merge fails if cwd needs to be removed' '
     - 	git checkout foo/bar/baz &&
     - 	test_when_finished "git clean -fdx" &&
     + GIT_TEST_MERGE_ALGORITHM=ort
       
     -@@ t/t2501-cwd-empty.sh: test_expect_failure 'cherry-pick does not remove cwd incidentally' '
     - 	test_path_is_dir subdir
     + test_expect_success 'merge fails if cwd needs to be removed' '
     +-	test_required_dir_removal failure git merge fd_conflict
     ++	test_required_dir_removal success git merge fd_conflict
       '
       
     --test_expect_failure 'cherry-pick fails if cwd needs to be removed' '
     -+test_expect_success 'cherry-pick fails if cwd needs to be removed' '
     - 	git checkout foo/bar/baz &&
     - 	test_when_finished "git clean -fdx" &&
     + test_expect_success 'cherry-pick does not clean cwd incidentally' '
     +@@ t/t2501-cwd-empty.sh: test_expect_success 'cherry-pick does not clean cwd incidentally' '
     + '
       
     -@@ t/t2501-cwd-empty.sh: test_expect_failure 'rebase does not remove cwd incidentally' '
     - 	test_path_is_dir subdir
     + test_expect_success 'cherry-pick fails if cwd needs to be removed' '
     +-	test_required_dir_removal failure git cherry-pick fd_conflict
     ++	test_required_dir_removal success git cherry-pick fd_conflict
       '
       
     --test_expect_failure 'rebase fails if cwd needs to be removed' '
     -+test_expect_success 'rebase fails if cwd needs to be removed' '
     - 	git checkout foo/bar/baz &&
     - 	test_when_finished "git clean -fdx" &&
     + test_expect_success 'rebase does not clean cwd incidentally' '
     +@@ t/t2501-cwd-empty.sh: test_expect_success 'revert does not clean cwd incidentally' '
     + '
       
     -@@ t/t2501-cwd-empty.sh: test_expect_failure 'revert does not remove cwd incidentally' '
     - 	test_path_is_dir subdir
     + test_expect_success 'revert fails if cwd needs to be removed' '
     +-	test_required_dir_removal failure git revert undo_fd_conflict
     ++	test_required_dir_removal success git revert undo_fd_conflict
       '
       
     --test_expect_failure 'revert fails if cwd needs to be removed' '
     -+test_expect_success 'revert fails if cwd needs to be removed' '
     - 	git checkout fd_conflict &&
     - 	git revert HEAD &&
     - 	test_when_finished "git clean -fdx" &&
     + test_expect_success 'rm does not clean cwd incidentally' '
      
       ## unpack-trees.c ##
      @@ unpack-trees.c: static const char *unpack_plumbing_errors[NB_UNPACK_TREES_WARNING_TYPES] = {
  4:  e06806e3a32 !  4:  1bb8905900c unpack-trees: add special cwd handling
     @@ Commit message
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## t/t2501-cwd-empty.sh ##
     -@@ t/t2501-cwd-empty.sh: test_expect_failure 'reset --hard does not clean cwd incidentally' '
     - 	test_path_is_dir foo
     +@@ t/t2501-cwd-empty.sh: test_expect_success 'reset --hard does not clean cwd incidentally' '
       '
       
     --test_expect_failure 'reset --hard fails if cwd needs to be removed' '
     -+test_expect_success 'reset --hard fails if cwd needs to be removed' '
     - 	git checkout foo/bar/baz &&
     - 	test_when_finished "git clean -fdx" &&
     + test_expect_success 'reset --hard fails if cwd needs to be removed' '
     +-	test_required_dir_removal failure git reset --hard fd_conflict
     ++	test_required_dir_removal success git reset --hard fd_conflict
     + '
       
     + test_expect_success 'merge does not clean cwd incidentally' '
      
       ## unpack-trees.c ##
      @@ unpack-trees.c: static int verify_absent_1(const struct cache_entry *ce,
  5:  46728f74ea1 !  5:  8a33d74e7cf symlinks: do not include startup_info->original_cwd in dir removal
     @@ symlinks.c: void schedule_dir_for_removal(const char *name, int len)
       				   &previous_slash);
      
       ## t/t2501-cwd-empty.sh ##
     -@@ t/t2501-cwd-empty.sh: test_expect_success setup '
     - 	git commit -m dirORfile
     - '
     +@@ t/t2501-cwd-empty.sh: test_required_dir_removal () {
     + }
       
     --test_expect_failure 'checkout does not clean cwd incidentally' '
     -+test_expect_success 'checkout does not clean cwd incidentally' '
     - 	git checkout foo/bar/baz &&
     - 	test_path_is_dir foo/bar &&
     + test_expect_success 'checkout does not clean cwd incidentally' '
     +-	test_incidental_dir_removal failure git checkout init
     ++	test_incidental_dir_removal success git checkout init
     + '
       
     + test_expect_success 'checkout fails if cwd needs to be removed' '
      @@ t/t2501-cwd-empty.sh: test_expect_success 'checkout fails if cwd needs to be removed' '
     - 	test_path_is_dir dirORfile
       '
       
     --test_expect_failure 'reset --hard does not clean cwd incidentally' '
     -+test_expect_success 'reset --hard does not clean cwd incidentally' '
     - 	git checkout foo/bar/baz &&
     - 	test_path_is_dir foo/bar &&
     + test_expect_success 'reset --hard does not clean cwd incidentally' '
     +-	test_incidental_dir_removal failure git reset --hard init
     ++	test_incidental_dir_removal success git reset --hard init
     + '
       
     + test_expect_success 'reset --hard fails if cwd needs to be removed' '
      @@ t/t2501-cwd-empty.sh: test_expect_success 'reset --hard fails if cwd needs to be removed' '
     - 	test_path_is_dir dirORfile
       '
       
     --test_expect_failure 'merge does not remove cwd incidentally' '
     -+test_expect_success 'merge does not remove cwd incidentally' '
     - 	git checkout foo/bar/baz &&
     - 	test_when_finished "git clean -fdx" &&
     + test_expect_success 'merge does not clean cwd incidentally' '
     +-	test_incidental_dir_removal failure git merge reverted
     ++	test_incidental_dir_removal success git merge reverted
     + '
       
     + # This file uses some simple merges where
      @@ t/t2501-cwd-empty.sh: test_expect_success 'merge fails if cwd needs to be removed' '
     - 	test_path_is_dir dirORfile
       '
       
     --test_expect_failure 'cherry-pick does not remove cwd incidentally' '
     -+test_expect_success 'cherry-pick does not remove cwd incidentally' '
     - 	git checkout foo/bar/baz &&
     - 	test_when_finished "git clean -fdx" &&
     - 
     -@@ t/t2501-cwd-empty.sh: test_expect_success 'cherry-pick fails if cwd needs to be removed' '
     - 	test_path_is_dir dirORfile
     + test_expect_success 'cherry-pick does not clean cwd incidentally' '
     +-	test_incidental_dir_removal failure git cherry-pick reverted
     ++	test_incidental_dir_removal success git cherry-pick reverted
       '
       
     --test_expect_failure 'rebase does not remove cwd incidentally' '
     -+test_expect_success 'rebase does not remove cwd incidentally' '
     - 	git checkout foo/bar/baz &&
     - 	test_when_finished "git clean -fdx" &&
     - 
     + test_expect_success 'cherry-pick fails if cwd needs to be removed' '
      @@ t/t2501-cwd-empty.sh: test_expect_success 'rebase fails if cwd needs to be removed' '
     - 	test_path_is_dir dirORfile
       '
       
     --test_expect_failure 'revert does not remove cwd incidentally' '
     -+test_expect_success 'revert does not remove cwd incidentally' '
     - 	git checkout foo/bar/baz &&
     - 	test_when_finished "git clean -fdx" &&
     + test_expect_success 'revert does not clean cwd incidentally' '
     +-	test_incidental_dir_removal failure git revert HEAD
     ++	test_incidental_dir_removal success git revert HEAD
     + '
       
     + test_expect_success 'revert fails if cwd needs to be removed' '
  6:  01ce9444dae !  6:  11e4ec881bb clean: do not attempt to remove startup_info->original_cwd
     @@ builtin/clean.c: static int remove_dirs(struct strbuf *path, const char *prefix,
      +		strbuf_realpath(&realpath, path->buf, 1);
      +
      +		/*
     -+		 * path and realpath are absolute; for comparison, we want
     -+		 * startup_info->original_cwd to be an absolute path too.  We
     -+		 * can use strbuf_realpath for this.  Also, if original_cwd
     -+		 * started out as the empty string, then it corresponded to
     -+		 * the top of the worktree, which is protected by other means
     -+		 * so we just leave it blank.
     ++		 * path and realpath are absolute; for comparison, we would
     ++		 * like to transform startup_info->original_cwd to an absolute
     ++		 * path too.
      +		 */
     -+		 if (*startup_info->original_cwd)
     ++		 if (startup_info->original_cwd)
      +			 strbuf_realpath(&real_ocwd,
      +					 startup_info->original_cwd, 1);
      +
     @@ builtin/clean.c: static int remove_dirs(struct strbuf *path, const char *prefix,
       	return ret;
      
       ## t/t2501-cwd-empty.sh ##
     -@@ t/t2501-cwd-empty.sh: test_expect_failure 'apply does not remove cwd incidentally' '
     - 	test_path_is_dir subdir
     - '
     +@@ t/t2501-cwd-empty.sh: test_incidental_untracked_dir_removal () {
     + }
       
     --test_expect_failure 'clean does not remove cwd incidentally' '
     -+test_expect_success 'clean does not remove cwd incidentally' '
     - 	git checkout foo/bar/baz &&
     - 	test_when_finished "git clean -fdx" &&
     + test_expect_success 'clean does not remove cwd incidentally' '
     +-	test_incidental_untracked_dir_removal failure \
     +-		git -C .. clean -fd -e warnings . >warnings
     ++	test_incidental_untracked_dir_removal success \
     ++		git -C .. clean -fd -e warnings . >warnings &&
     ++	grep "Refusing to remove current working directory" warnings
     + '
       
     + test_expect_success 'stash does not remove cwd incidentally' '
  -:  ----------- >  7:  39b1f3a225e rebase: do not attempt to remove startup_info->original_cwd
  7:  edec0894ca2 !  8:  0110462a19c stash: do not attempt to remove startup_info->original_cwd
     @@ Metadata
       ## Commit message ##
          stash: do not attempt to remove startup_info->original_cwd
      
     +    Since stash spawns a `clean` subprocess, make sure we run that from the
     +    startup_info->original_cwd directory, so that the `clean` processs knows
     +    to protect that directory.  Also, since the `clean` command might no
     +    longer run from the toplevel, pass the ':/' magic pathspec to ensure we
     +    still clean from the toplevel.
     +
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## builtin/stash.c ##
     @@ builtin/stash.c: static int do_push_stash(const struct pathspec *ps, const char
       
       			cp.git_cmd = 1;
      +			if (startup_info->original_cwd &&
     -+			    *startup_info->original_cwd &&
      +			    !is_absolute_path(startup_info->original_cwd))
      +				cp.dir = startup_info->original_cwd;
       			strvec_pushl(&cp.args, "clean", "--force",
     @@ builtin/stash.c: static int do_push_stash(const struct pathspec *ps, const char
      
       ## t/t2501-cwd-empty.sh ##
      @@ t/t2501-cwd-empty.sh: test_expect_success 'clean does not remove cwd incidentally' '
     - 	test_path_is_dir untracked
       '
       
     --test_expect_failure 'stash does not remove cwd incidentally' '
     -+test_expect_success 'stash does not remove cwd incidentally' '
     - 	git checkout foo/bar/baz &&
     - 	test_when_finished "git clean -fdx" &&
     + test_expect_success 'stash does not remove cwd incidentally' '
     +-	test_incidental_untracked_dir_removal failure \
     ++	test_incidental_untracked_dir_removal success \
     + 		git stash --include-untracked
     + '
       
  8:  1815f18592b !  9:  2c73a09a2e8 dir: avoid incidentally removing the original_cwd in remove_path()
     @@ dir.h: int get_sparse_checkout_patterns(struct pattern_list *pl);
      
       ## t/t2501-cwd-empty.sh ##
      @@ t/t2501-cwd-empty.sh: test_expect_success 'revert fails if cwd needs to be removed' '
     - 	test_path_is_dir dirORfile
       '
       
     --test_expect_failure 'rm does not remove cwd incidentally' '
     -+test_expect_success 'rm does not remove cwd incidentally' '
     - 	test_when_finished "git reset --hard" &&
     - 	git checkout foo/bar/baz &&
     + test_expect_success 'rm does not clean cwd incidentally' '
     +-	test_incidental_dir_removal failure git rm bar/baz.t
     ++	test_incidental_dir_removal success git rm bar/baz.t
     + '
       
     -@@ t/t2501-cwd-empty.sh: test_expect_failure 'rm does not remove cwd incidentally' '
     - 	test_path_is_dir foo
     + test_expect_success 'apply does not remove cwd incidentally' '
     + 	git diff HEAD HEAD~1 >patch &&
     +-	test_incidental_dir_removal failure git apply ../patch
     ++	test_incidental_dir_removal success git apply ../patch
       '
       
     --test_expect_failure 'apply does not remove cwd incidentally' '
     -+test_expect_success 'apply does not remove cwd incidentally' '
     - 	test_when_finished "git reset --hard" &&
     - 	git checkout foo/bar/baz &&
     + test_incidental_untracked_dir_removal () {
     +@@ t/t2501-cwd-empty.sh: test_expect_success '`rm -rf dir` even with only tracked files will remove somet
     + 	) &&
     + 
     + 	test_path_is_missing a/b/c/tracked &&
     +-	## We would prefer if a/b was still present, though empty, since it
     +-	## was the current working directory
     +-	#test_path_is_dir a/b
     +-	## But the current behavior is that it not only deletes the directory
     +-	## a/b as requested, but also goes and deletes a
     +-	test_path_is_missing a
     ++	test_path_is_missing a/b/c &&
     ++	test_path_is_dir a/b
     + '
       
     + test_expect_success 'git version continues working from a deleted dir' '
  9:  adaad7aeaac ! 10:  d4e50b4053d dir: new flag to remove_dir_recurse() to spare the original_cwd
     @@ dir.h: int get_sparse_checkout_patterns(struct pattern_list *pl);
        * of the above REMOVE_DIR_* constants. Return 0 on success.
      
       ## t/t2501-cwd-empty.sh ##
     -@@ t/t2501-cwd-empty.sh: test_expect_success 'stash does not remove cwd incidentally' '
     - 	test_path_is_dir untracked
     - '
     +@@ t/t2501-cwd-empty.sh: test_submodule_removal () {
     + 	test_status=
     + 	test $path_status = dir && test_status=test_must_fail
     + 
     +-	# Actually, while path_status == dir && test_status=test_must_fail
     +-	# reflect our desired behavior, current behavior is:
     +-	path_status=missing
     +-	test_status=
     +-
     + 	test_when_finished "git reset --hard HEAD~1" &&
     + 	test_when_finished "rm -rf .git/modules/my_submodule" &&
       
     -+test_expect_success 'rm -r leaves submodule if cwd inside' '
     -+	test_when_finished "git reset --hard HEAD~1" &&
     -+	test_when_finished "rm -rf .git/modules/my_submodule" &&
     -+
     -+	git checkout foo/bar/baz &&
     -+
     -+	git init my_submodule &&
     -+	touch my_submodule/file &&
     -+	git -C my_submodule add file &&
     -+	git -C my_submodule commit -m "initial commit" &&
     -+	git submodule add ./my_submodule &&
     -+	git commit -m "Add the submodule" &&
     -+
     -+	(
     -+		cd my_submodule &&
     -+		test_must_fail git --git-dir=../.git --work-tree=.. rm -r ../my_submodule/
     -+	) &&
     -+
     -+	test_path_is_dir my_submodule
     -+'
     -+
     -+test_expect_success 'rm -rf removes submodule even if cwd inside' '
     -+	test_when_finished "git reset --hard HEAD~1" &&
     -+	test_when_finished "rm -rf .git/modules/my_submodule" &&
     -+
     -+	git checkout foo/bar/baz &&
     -+
     -+	git init my_submodule &&
     -+	touch my_submodule/file &&
     -+	git -C my_submodule add file &&
     -+	git -C my_submodule commit -m "initial commit" &&
     -+	git submodule add ./my_submodule &&
     -+	git commit -m "Add the submodule" &&
     -+
     -+	(
     -+		cd my_submodule &&
     -+		git --git-dir=../.git --work-tree=.. rm -rf ../my_submodule/
     -+	) &&
     -+
     -+	test_path_is_missing my_submodule
     -+'
     -+
     - test_done
  -:  ----------- > 11:  7eb6281be4b t2501: simplify the tests since we can now assume desired behavior

-- 
gitgitgadget

[PATCH v3 01/11] t2501: add various tests for removing the current working directory

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Numerous commands will remove directories left empty as a "convenience"
after removing files within them.  That is normally fine, but removing
the current working directory can be rather inconvenient since it can
cause confusion for the user when they run subsequent commands.  For
example, after one git process has removed the current working
directory, git status/log/diff will all abort with the message:

    fatal: Unable to read current working directory: No such file or directory

We also have code paths that, when a file needs to be placed where a
directory is (due to e.g. checkout, merge, reset, whatever), will check
if this is okay and error out if not.  These rules include:
  * all tracked files under that directory are intended to be removed by
    the operation
  * none of the tracked files under that directory have uncommitted
    modification
  * there are no untracked files under that directory
However, if we end up remove the current working directory, we can cause
user confusion when they run subsequent commands, so we would prefer if
there was a fourth rule added to this list: avoid removing the current
working directory.

Since there are several code paths that can result in the current
working directory being removed, add several tests of various different
codepaths.  To make it clearer what the difference between the current
behavior and the behavior at the end of the series, code both of them
into the tests and have the appropriate behavior be selected by a flag.
Subsequent commits will toggle the flag from current to desired
behavior.

Also add a few tests suggested during the review of earlier rounds of
this patch series.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh | 333 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 333 insertions(+)
 create mode 100755 t/t2501-cwd-empty.sh

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
new file mode 100755
index 00000000000..67feb6fd200
--- /dev/null
+++ b/t/t2501-cwd-empty.sh
@@ -0,0 +1,333 @@
+#!/bin/sh
+
+test_description='Test handling of the current working directory becoming empty'
+
+. ./test-lib.sh
+
+test_expect_success setup '
+	test_commit init &&
+
+	git branch fd_conflict &&
+
+	mkdir -p foo/bar &&
+	test_commit foo/bar/baz &&
+
+	git revert HEAD &&
+	git tag reverted &&
+
+	git checkout fd_conflict &&
+	mkdir dirORfile &&
+	test_commit dirORfile/foo &&
+
+	git rm -r dirORfile &&
+	echo not-a-directory >dirORfile &&
+	git add dirORfile &&
+	git commit -m dirORfile &&
+
+	git switch -c df_conflict HEAD~1 &&
+	test_commit random_file &&
+
+	git switch -c undo_fd_conflict fd_conflict &&
+	git revert HEAD
+'
+
+test_incidental_dir_removal () {
+	works=$1 &&
+	shift &&
+
+	test_when_finished "git reset --hard" &&
+
+	git checkout foo/bar/baz^{commit} &&
+	test_path_is_dir foo/bar &&
+
+	(
+		cd foo &&
+		"$@" &&
+
+		# Although we want pwd & git status to pass, test for existing
+		# rather than desired behavior.
+		if [[ $works == "success" ]]; then
+			pwd -P &&
+			git status --porcelain
+		else
+			! pwd -P &&
+			test_might_fail git status --porcelain
+		fi
+	) &&
+	test_path_is_missing foo/bar/baz &&
+	test_path_is_missing foo/bar &&
+
+	# Although we want dir to be present, test for existing rather
+	# than desired behavior.
+	if [[ $works == "success" ]]; then
+		test_path_is_dir foo
+	else
+		test_path_is_missing foo
+	fi
+}
+
+test_required_dir_removal () {
+	works=$1 &&
+	shift &&
+
+	git checkout df_conflict^{commit} &&
+	test_when_finished "git clean -fdx" &&
+
+	(
+		cd dirORfile &&
+
+		# We'd like for the command to fail (much as it would if there
+		# was an untracked file there), and for pwd & git status to
+		# succeed afterwards.  But test for existing rather than
+		# desired behavior.
+		if [[ $works == "success" ]]; then
+			test_must_fail "$@" 2>../error &&
+			grep "Refusing to remove.*current working directory" ../error &&
+			pwd -P &&
+			git status --porcelain
+		else
+			"$@" &&
+			! pwd -P &&
+			test_might_fail git status --porcelain
+		fi
+	) &&
+
+	# Although we want dirORfile to be present, test for existing rather
+	# than desired behavior.
+	if [[ $works == "success" ]]; then
+		test_path_is_dir dirORfile
+	else
+		test_path_is_file dirORfile
+	fi
+}
+
+test_expect_success 'checkout does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git checkout init
+'
+
+test_expect_success 'checkout fails if cwd needs to be removed' '
+	test_required_dir_removal failure git checkout fd_conflict
+'
+
+test_expect_success 'reset --hard does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git reset --hard init
+'
+
+test_expect_success 'reset --hard fails if cwd needs to be removed' '
+	test_required_dir_removal failure git reset --hard fd_conflict
+'
+
+test_expect_success 'merge does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git merge reverted
+'
+
+# This file uses some simple merges where
+#   Base: 'dirORfile/' exists
+#   Side1: random other file changed
+#   Side2: 'dirORfile/' removed, 'dirORfile' added
+# this should resolve cleanly, but merge-recursive throws merge conflicts
+# because it's dumb.  Add a special test for checking merge-recursive (and
+# merge-ort), then after this just hard require ort for all remaining tests.
+#
+test_expect_success 'merge fails if cwd needs to be removed; recursive friendly' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir dirORfile &&
+	(
+		cd dirORfile &&
+
+		# We would rather this failed, but we test for existing
+		# rather than desired behavior
+		git merge fd_conflict 2>../error
+	) &&
+
+	## Here is the behavior we would rather have:
+	#test_path_is_dir dirORfile &&
+	#grep "Refusing to remove the current working directory" error
+	## But instead we test for existing behavior
+	test_path_is_file dirORfile &&
+	test_must_be_empty error
+'
+
+GIT_TEST_MERGE_ALGORITHM=ort
+
+test_expect_success 'merge fails if cwd needs to be removed' '
+	test_required_dir_removal failure git merge fd_conflict
+'
+
+test_expect_success 'cherry-pick does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git cherry-pick reverted
+'
+
+test_expect_success 'cherry-pick fails if cwd needs to be removed' '
+	test_required_dir_removal failure git cherry-pick fd_conflict
+'
+
+test_expect_success 'rebase does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git rebase reverted
+'
+
+test_expect_success 'rebase fails if cwd needs to be removed' '
+	test_required_dir_removal failure git rebase fd_conflict
+'
+
+test_expect_success 'revert does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git revert HEAD
+'
+
+test_expect_success 'revert fails if cwd needs to be removed' '
+	test_required_dir_removal failure git revert undo_fd_conflict
+'
+
+test_expect_success 'rm does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git rm bar/baz.t
+'
+
+test_expect_success 'apply does not remove cwd incidentally' '
+	git diff HEAD HEAD~1 >patch &&
+	test_incidental_dir_removal failure git apply ../patch
+'
+
+test_incidental_untracked_dir_removal () {
+	works=$1 &&
+	shift &&
+
+	test_when_finished "git reset --hard" &&
+
+	git checkout foo/bar/baz^{commit} &&
+	mkdir -p untracked &&
+	mkdir empty
+	>untracked/random &&
+
+	(
+		cd untracked &&
+		"$@" &&
+
+		# Although we want pwd & git status to pass, test for existing
+		# rather than desired behavior.
+		if [[ $works == "success" ]]; then
+			pwd -P &&
+			git status --porcelain
+		else
+			! pwd -P &&
+			test_might_fail git status --porcelain
+		fi
+	) &&
+	test_path_is_missing empty &&
+	test_path_is_missing untracked/random &&
+
+	# Although we want dir to be present, test for existing rather
+	# than desired behavior.
+	if [[ $works == "success" ]]; then
+		test_path_is_dir untracked
+	else
+		test_path_is_missing untracked
+	fi
+}
+
+test_expect_success 'clean does not remove cwd incidentally' '
+	test_incidental_untracked_dir_removal failure \
+		git -C .. clean -fd -e warnings . >warnings
+'
+
+test_expect_success 'stash does not remove cwd incidentally' '
+	test_incidental_untracked_dir_removal failure \
+		git stash --include-untracked
+'
+
+test_expect_success '`rm -rf dir` only removes a subset of dir' '
+	test_when_finished "rm -rf a/" &&
+
+	mkdir -p a/b/c &&
+	>a/b/c/untracked &&
+	>a/b/c/tracked &&
+	git add a/b/c/tracked &&
+
+	(
+		cd a/b &&
+		git rm -rf ../b
+	) &&
+
+	test_path_is_dir a/b &&
+	test_path_is_missing a/b/c/tracked &&
+	test_path_is_file a/b/c/untracked
+'
+
+test_expect_success '`rm -rf dir` even with only tracked files will remove something else' '
+	test_when_finished "rm -rf a/" &&
+
+	mkdir -p a/b/c &&
+	>a/b/c/tracked &&
+	git add a/b/c/tracked &&
+
+	(
+		cd a/b &&
+		git rm -rf ../b
+	) &&
+
+	test_path_is_missing a/b/c/tracked &&
+	## We would prefer if a/b was still present, though empty, since it
+	## was the current working directory
+	#test_path_is_dir a/b
+	## But the current behavior is that it not only deletes the directory
+	## a/b as requested, but also goes and deletes a
+	test_path_is_missing a
+'
+
+test_expect_success 'git version continues working from a deleted dir' '
+	mkdir tmp &&
+	(
+		cd tmp &&
+		rm -rf ../tmp &&
+		git version
+	)
+'
+
+test_submodule_removal () {
+	path_status=$1 &&
+	shift &&
+
+	test_status=
+	test $path_status = dir && test_status=test_must_fail
+
+	# Actually, while path_status == dir && test_status=test_must_fail
+	# reflect our desired behavior, current behavior is:
+	path_status=missing
+	test_status=
+
+	test_when_finished "git reset --hard HEAD~1" &&
+	test_when_finished "rm -rf .git/modules/my_submodule" &&
+
+	git checkout foo/bar/baz &&
+
+	git init my_submodule &&
+	touch my_submodule/file &&
+	git -C my_submodule add file &&
+	git -C my_submodule commit -m "initial commit" &&
+	git submodule add ./my_submodule &&
+	git commit -m "Add the submodule" &&
+
+	(
+		cd my_submodule &&
+		$test_status "$@"
+	) &&
+
+	test_path_is_${path_status} my_submodule
+}
+
+test_expect_success 'rm -r with -C leaves submodule if cwd inside' '
+	test_submodule_removal dir git -C .. rm -r my_submodule/
+'
+
+test_expect_success 'rm -r leaves submodule if cwd inside' '
+	test_submodule_removal dir \
+		git --git-dir=../.git --work-tree=.. rm -r ../my_submodule/
+'
+
+test_expect_success 'rm -rf removes submodule even if cwd inside' '
+	test_submodule_removal missing \
+		git --git-dir=../.git --work-tree=.. rm -rf ../my_submodule/
+'
+
+test_done
-- 
gitgitgadget

[PATCH v3 02/11] setup: introduce startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Removing the current working directory causes all subsequent git
commands run from that directory to get confused and fail with a message
about being unable to read the current working directory:

    $ git status
    fatal: Unable to read current working directory: No such file or directory

Non-git commands likely have similar warnings or even errors, e.g.

    $ bash -c 'echo hello'
    shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    hello

This confuses end users, particularly since the command they get the
error from is not the one that caused the problem; the problem came from
the side-effect of some previous command.

We would like to avoid removing the current working directory of our
parent process; towards this end, introduce a new variable,
startup_info->original_cwd, that tracks the current working directory
that we inherited from our parent process.  For convenience of later
comparisons, we prefer that this new variable store a path relative to
the toplevel working directory (thus much like 'prefix'), except without
the trailing slash.

Subsequent commits will make use of this new variable.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 cache.h       |  2 ++
 common-main.c |  4 ++++
 setup.c       | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 71 insertions(+)

diff --git a/cache.h b/cache.h
index eba12487b99..92e181ea759 100644
--- a/cache.h
+++ b/cache.h
@@ -1834,8 +1834,10 @@ void overlay_tree_on_index(struct index_state *istate,
 struct startup_info {
 	int have_repository;
 	const char *prefix;
+	const char *original_cwd;
 };
 extern struct startup_info *startup_info;
+extern const char *tmp_original_cwd;
 
 /* merge.c */
 struct commit_list;
diff --git a/common-main.c b/common-main.c
index 71e21dd20a3..aa8d5aba5bb 100644
--- a/common-main.c
+++ b/common-main.c
@@ -26,6 +26,7 @@ static void restore_sigpipe_to_default(void)
 int main(int argc, const char **argv)
 {
 	int result;
+	struct strbuf tmp = STRBUF_INIT;
 
 	trace2_initialize_clock();
 
@@ -49,6 +50,9 @@ int main(int argc, const char **argv)
 	trace2_cmd_start(argv);
 	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
 
+	if (!strbuf_getcwd(&tmp))
+		tmp_original_cwd = strbuf_detach(&tmp, NULL);
+
 	result = cmd_main(argc, argv);
 
 	trace2_cmd_exit(result);
diff --git a/setup.c b/setup.c
index 347d7181ae9..44f5bd38f7b 100644
--- a/setup.c
+++ b/setup.c
@@ -12,6 +12,7 @@ static int work_tree_config_is_bogus;
 
 static struct startup_info the_startup_info;
 struct startup_info *startup_info = &the_startup_info;
+const char *tmp_original_cwd;
 
 /*
  * The input parameter must contain an absolute path, and it must already be
@@ -432,6 +433,69 @@ void setup_work_tree(void)
 	initialized = 1;
 }
 
+static void setup_original_cwd(void)
+{
+	struct strbuf tmp = STRBUF_INIT;
+	const char *worktree = NULL;
+	int offset = -1;
+
+	if (!tmp_original_cwd)
+		return;
+
+	/*
+	 * startup_info->original_cwd points to the current working
+	 * directory we inherited from our parent process, which is a
+	 * directory we want to avoid removing.
+	 *
+	 * For convience, we would like to have the path relative to the
+	 * worktree instead of an absolute path.
+	 *
+	 * Yes, startup_info->original_cwd is usually the same as 'prefix',
+	 * but differs in two ways:
+	 *   - prefix has a trailing '/'
+	 *   - if the user passes '-C' to git, that modifies the prefix but
+	 *     not startup_info->original_cwd.
+	 */
+
+	/* Normalize the directory */
+	strbuf_realpath(&tmp, tmp_original_cwd, 1);
+	free((char*)tmp_original_cwd);
+	tmp_original_cwd = NULL;
+	startup_info->original_cwd = strbuf_detach(&tmp, NULL);
+
+	/*
+	 * Get our worktree; we only protect the current working directory
+	 * if it's in the worktree.
+	 */
+	worktree = get_git_work_tree();
+	if (!worktree)
+		goto no_prevention_needed;
+
+	offset = dir_inside_of(startup_info->original_cwd, worktree);
+	if (offset >= 0) {
+		/*
+		 * If startup_info->original_cwd == worktree, that is already
+		 * protected and we don't need original_cwd as a secondary
+		 * protection measure.
+		 */
+		if (!*(startup_info->original_cwd + offset))
+			goto no_prevention_needed;
+
+		/*
+		 * original_cwd was inside worktree; precompose it just as
+		 * we do prefix so that built up paths will match
+		 */
+		startup_info->original_cwd = \
+			precompose_string_if_needed(startup_info->original_cwd
+						    + offset);
+	}
+	return;
+
+no_prevention_needed:
+	free((char*)startup_info->original_cwd);
+	startup_info->original_cwd = NULL;
+}
+
 static int read_worktree_config(const char *var, const char *value, void *vdata)
 {
 	struct repository_format *data = vdata;
@@ -1330,6 +1394,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
 		setenv(GIT_PREFIX_ENVIRONMENT, "", 1);
 	}
 
+	setup_original_cwd();
 
 	strbuf_release(&dir);
 	strbuf_release(&gitdir);
-- 
gitgitgadget

[PATCH v3 03/11] unpack-trees: refuse to remove startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

In the past, when a directory needs to be removed to make room for a
file, we have always errored out when that directory contains any
untracked (but not ignored) files.  Add an extra condition on that: also
error out if the directory is the current working directory we inherited
from our parent process.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh | 20 +++++++-------------
 unpack-trees.c       | 17 +++++++++++++----
 unpack-trees.h       |  1 +
 3 files changed, 21 insertions(+), 17 deletions(-)

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 67feb6fd200..52399d1906f 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -106,7 +106,7 @@ test_expect_success 'checkout does not clean cwd incidentally' '
 '
 
 test_expect_success 'checkout fails if cwd needs to be removed' '
-	test_required_dir_removal failure git checkout fd_conflict
+	test_required_dir_removal success git checkout fd_conflict
 '
 
 test_expect_success 'reset --hard does not clean cwd incidentally' '
@@ -137,23 +137,17 @@ test_expect_success 'merge fails if cwd needs to be removed; recursive friendly'
 	(
 		cd dirORfile &&
 
-		# We would rather this failed, but we test for existing
-		# rather than desired behavior
-		git merge fd_conflict 2>../error
+		test_must_fail git merge fd_conflict 2>../error
 	) &&
 
-	## Here is the behavior we would rather have:
-	#test_path_is_dir dirORfile &&
-	#grep "Refusing to remove the current working directory" error
-	## But instead we test for existing behavior
-	test_path_is_file dirORfile &&
-	test_must_be_empty error
+	test_path_is_dir dirORfile &&
+	grep "Refusing to remove the current working directory" error
 '
 
 GIT_TEST_MERGE_ALGORITHM=ort
 
 test_expect_success 'merge fails if cwd needs to be removed' '
-	test_required_dir_removal failure git merge fd_conflict
+	test_required_dir_removal success git merge fd_conflict
 '
 
 test_expect_success 'cherry-pick does not clean cwd incidentally' '
@@ -161,7 +155,7 @@ test_expect_success 'cherry-pick does not clean cwd incidentally' '
 '
 
 test_expect_success 'cherry-pick fails if cwd needs to be removed' '
-	test_required_dir_removal failure git cherry-pick fd_conflict
+	test_required_dir_removal success git cherry-pick fd_conflict
 '
 
 test_expect_success 'rebase does not clean cwd incidentally' '
@@ -177,7 +171,7 @@ test_expect_success 'revert does not clean cwd incidentally' '
 '
 
 test_expect_success 'revert fails if cwd needs to be removed' '
-	test_required_dir_removal failure git revert undo_fd_conflict
+	test_required_dir_removal success git revert undo_fd_conflict
 '
 
 test_expect_success 'rm does not clean cwd incidentally' '
diff --git a/unpack-trees.c b/unpack-trees.c
index 89ca95ce90b..6bc16f3a714 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -36,6 +36,9 @@ static const char *unpack_plumbing_errors[NB_UNPACK_TREES_WARNING_TYPES] = {
 	/* ERROR_NOT_UPTODATE_DIR */
 	"Updating '%s' would lose untracked files in it",
 
+	/* ERROR_CWD_IN_THE_WAY */
+	"Refusing to remove '%s' since it is the current working directory.",
+
 	/* ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN */
 	"Untracked working tree file '%s' would be overwritten by merge.",
 
@@ -131,6 +134,9 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 	msgs[ERROR_NOT_UPTODATE_DIR] =
 		_("Updating the following directories would lose untracked files in them:\n%s");
 
+	msgs[ERROR_CWD_IN_THE_WAY] =
+		_("Refusing to remove the current working directory:\n%s");
+
 	if (!strcmp(cmd, "checkout"))
 		msg = advice_enabled(ADVICE_COMMIT_BEFORE_MERGE)
 		      ? _("The following untracked working tree files would be removed by checkout:\n%%s"
@@ -2146,10 +2152,7 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,
 		cnt++;
 	}
 
-	/*
-	 * Then we need to make sure that we do not lose a locally
-	 * present file that is not ignored.
-	 */
+	/* Do not lose a locally present file that is not ignored. */
 	pathbuf = xstrfmt("%.*s/", namelen, ce->name);
 
 	memset(&d, 0, sizeof(d));
@@ -2160,6 +2163,12 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,
 	free(pathbuf);
 	if (i)
 		return add_rejected_path(o, ERROR_NOT_UPTODATE_DIR, ce->name);
+
+	/* Do not lose startup_info->original_cwd */
+	if (startup_info->original_cwd &&
+	    !strcmp(startup_info->original_cwd, ce->name))
+		return add_rejected_path(o, ERROR_CWD_IN_THE_WAY, ce->name);
+
 	return cnt;
 }
 
diff --git a/unpack-trees.h b/unpack-trees.h
index 71ffb7eeb0c..efb9edfbb27 100644
--- a/unpack-trees.h
+++ b/unpack-trees.h
@@ -19,6 +19,7 @@ enum unpack_trees_error_types {
 	ERROR_WOULD_OVERWRITE = 0,
 	ERROR_NOT_UPTODATE_FILE,
 	ERROR_NOT_UPTODATE_DIR,
+	ERROR_CWD_IN_THE_WAY,
 	ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN,
 	ERROR_WOULD_LOSE_UNTRACKED_REMOVED,
 	ERROR_BIND_OVERLAP,
-- 
gitgitgadget

[PATCH v3 04/11] unpack-trees: add special cwd handling

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

When running commands such as `git reset --hard` from a subdirectory, if
that subdirectory is in the way of adding needed files, bail with an
error message.

Note that this change looks kind of like it duplicates the new lines of
code from the previous commit in verify_clean_subdirectory().  However,
when we are preserving untracked files, we would rather any error
messages about untracked files being in the way take precedence over
error messages about a subdirectory that happens to be the_original_cwd
being in the way.  But in the UNPACK_RESET_OVERWRITE_UNTRACKED case,
there is no untracked checking to be done, so we simply add a special
case near the top of verify_absent_1.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh |  2 +-
 unpack-trees.c       | 13 +++++++++++--
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 52399d1906f..886a391a63d 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -114,7 +114,7 @@ test_expect_success 'reset --hard does not clean cwd incidentally' '
 '
 
 test_expect_success 'reset --hard fails if cwd needs to be removed' '
-	test_required_dir_removal failure git reset --hard fd_conflict
+	test_required_dir_removal success git reset --hard fd_conflict
 '
 
 test_expect_success 'merge does not clean cwd incidentally' '
diff --git a/unpack-trees.c b/unpack-trees.c
index 6bc16f3a714..5852807d2fb 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -2261,10 +2261,19 @@ static int verify_absent_1(const struct cache_entry *ce,
 	int len;
 	struct stat st;
 
-	if (o->index_only || !o->update ||
-	    o->reset == UNPACK_RESET_OVERWRITE_UNTRACKED)
+	if (o->index_only || !o->update)
 		return 0;
 
+	if (o->reset == UNPACK_RESET_OVERWRITE_UNTRACKED) {
+		/* Avoid nuking startup_info->original_cwd... */
+		if (startup_info->original_cwd &&
+		    !strcmp(startup_info->original_cwd, ce->name))
+			return add_rejected_path(o, ERROR_CWD_IN_THE_WAY,
+						 ce->name);
+		/* ...but nuke anything else. */
+		return 0;
+	}
+
 	len = check_leading_path(ce->name, ce_namelen(ce), 0);
 	if (!len)
 		return 0;
-- 
gitgitgadget

[PATCH v3 05/11] symlinks: do not include startup_info->original_cwd in dir removal

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

symlinks has a pair of schedule_dir_for_removal() and
remove_scheduled_dirs() functions that ensure that directories made
empty by removing other files also themselves get removed.  However, we
want to exclude startup_info->original_cwd and leave it around.  This
avoids the user getting confused by subsequent git commands (and non-git
commands) that would otherwise report confusing messages about being
unable to read the current working directory.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 symlinks.c           |  8 +++++++-
 t/t2501-cwd-empty.sh | 10 +++++-----
 2 files changed, 12 insertions(+), 6 deletions(-)

diff --git a/symlinks.c b/symlinks.c
index 5232d02020c..c667baa949b 100644
--- a/symlinks.c
+++ b/symlinks.c
@@ -279,7 +279,9 @@ static void do_remove_scheduled_dirs(int new_len)
 {
 	while (removal.len > new_len) {
 		removal.buf[removal.len] = '\0';
-		if (rmdir(removal.buf))
+		if ((startup_info->original_cwd &&
+		     !strcmp(removal.buf, startup_info->original_cwd)) ||
+		    rmdir(removal.buf))
 			break;
 		do {
 			removal.len--;
@@ -293,6 +295,10 @@ void schedule_dir_for_removal(const char *name, int len)
 {
 	int match_len, last_slash, i, previous_slash;
 
+	if (startup_info->original_cwd &&
+	    !strcmp(name, startup_info->original_cwd))
+		return;	/* Do not remove the current working directory */
+
 	match_len = last_slash = i =
 		longest_path_match(name, len, removal.buf, removal.len,
 				   &previous_slash);
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 886a391a63d..496f4c6a6e9 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -102,7 +102,7 @@ test_required_dir_removal () {
 }
 
 test_expect_success 'checkout does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git checkout init
+	test_incidental_dir_removal success git checkout init
 '
 
 test_expect_success 'checkout fails if cwd needs to be removed' '
@@ -110,7 +110,7 @@ test_expect_success 'checkout fails if cwd needs to be removed' '
 '
 
 test_expect_success 'reset --hard does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git reset --hard init
+	test_incidental_dir_removal success git reset --hard init
 '
 
 test_expect_success 'reset --hard fails if cwd needs to be removed' '
@@ -118,7 +118,7 @@ test_expect_success 'reset --hard fails if cwd needs to be removed' '
 '
 
 test_expect_success 'merge does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git merge reverted
+	test_incidental_dir_removal success git merge reverted
 '
 
 # This file uses some simple merges where
@@ -151,7 +151,7 @@ test_expect_success 'merge fails if cwd needs to be removed' '
 '
 
 test_expect_success 'cherry-pick does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git cherry-pick reverted
+	test_incidental_dir_removal success git cherry-pick reverted
 '
 
 test_expect_success 'cherry-pick fails if cwd needs to be removed' '
@@ -167,7 +167,7 @@ test_expect_success 'rebase fails if cwd needs to be removed' '
 '
 
 test_expect_success 'revert does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git revert HEAD
+	test_incidental_dir_removal success git revert HEAD
 '
 
 test_expect_success 'revert fails if cwd needs to be removed' '
-- 
gitgitgadget

[PATCH v3 06/11] clean: do not attempt to remove startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/clean.c      | 44 +++++++++++++++++++++++++++++++++++---------
 t/t2501-cwd-empty.sh |  5 +++--
 2 files changed, 38 insertions(+), 11 deletions(-)

diff --git a/builtin/clean.c b/builtin/clean.c
index 98a2860409b..3ff02bbbffe 100644
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@ -36,6 +36,8 @@ static const char *msg_skip_git_dir = N_("Skipping repository %s\n");
 static const char *msg_would_skip_git_dir = N_("Would skip repository %s\n");
 static const char *msg_warn_remove_failed = N_("failed to remove %s");
 static const char *msg_warn_lstat_failed = N_("could not lstat %s\n");
+static const char *msg_skip_cwd = N_("Refusing to remove current working directory\n");
+static const char *msg_would_skip_cwd = N_("Would refuse to remove current working directory\n");
 
 enum color_clean {
 	CLEAN_COLOR_RESET = 0,
@@ -153,6 +155,8 @@ static int remove_dirs(struct strbuf *path, const char *prefix, int force_flag,
 {
 	DIR *dir;
 	struct strbuf quoted = STRBUF_INIT;
+	struct strbuf realpath = STRBUF_INIT;
+	struct strbuf real_ocwd = STRBUF_INIT;
 	struct dirent *e;
 	int res = 0, ret = 0, gone = 1, original_len = path->len, len;
 	struct string_list dels = STRING_LIST_INIT_DUP;
@@ -231,16 +235,36 @@ static int remove_dirs(struct strbuf *path, const char *prefix, int force_flag,
 	strbuf_setlen(path, original_len);
 
 	if (*dir_gone) {
-		res = dry_run ? 0 : rmdir(path->buf);
-		if (!res)
-			*dir_gone = 1;
-		else {
-			int saved_errno = errno;
-			quote_path(path->buf, prefix, &quoted, 0);
-			errno = saved_errno;
-			warning_errno(_(msg_warn_remove_failed), quoted.buf);
+		/*
+		 * Normalize path components in path->buf, e.g. change '\' to
+		 * '/' on Windows.
+		 */
+		strbuf_realpath(&realpath, path->buf, 1);
+
+		/*
+		 * path and realpath are absolute; for comparison, we would
+		 * like to transform startup_info->original_cwd to an absolute
+		 * path too.
+		 */
+		 if (startup_info->original_cwd)
+			 strbuf_realpath(&real_ocwd,
+					 startup_info->original_cwd, 1);
+
+		if (!strbuf_cmp(&realpath, &real_ocwd)) {
+			printf("%s", dry_run ? _(msg_would_skip_cwd) : _(msg_skip_cwd));
 			*dir_gone = 0;
-			ret = 1;
+		} else {
+			res = dry_run ? 0 : rmdir(path->buf);
+			if (!res)
+				*dir_gone = 1;
+			else {
+				int saved_errno = errno;
+				quote_path(path->buf, prefix, &quoted, 0);
+				errno = saved_errno;
+				warning_errno(_(msg_warn_remove_failed), quoted.buf);
+				*dir_gone = 0;
+				ret = 1;
+			}
 		}
 	}
 
@@ -250,6 +274,8 @@ static int remove_dirs(struct strbuf *path, const char *prefix, int force_flag,
 			printf(dry_run ?  _(msg_would_remove) : _(msg_remove), dels.items[i].string);
 	}
 out:
+	strbuf_release(&realpath);
+	strbuf_release(&real_ocwd);
 	strbuf_release(&quoted);
 	string_list_clear(&dels, 0);
 	return ret;
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 496f4c6a6e9..be04b673f6a 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -221,8 +221,9 @@ test_incidental_untracked_dir_removal () {
 }
 
 test_expect_success 'clean does not remove cwd incidentally' '
-	test_incidental_untracked_dir_removal failure \
-		git -C .. clean -fd -e warnings . >warnings
+	test_incidental_untracked_dir_removal success \
+		git -C .. clean -fd -e warnings . >warnings &&
+	grep "Refusing to remove current working directory" warnings
 '
 
 test_expect_success 'stash does not remove cwd incidentally' '
-- 
gitgitgadget

[PATCH v3 07/11] rebase: do not attempt to remove startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Since rebase spawns a `checkout` subprocess, make sure we run that from
the startup_info->original_cwd directory, so that the checkout process
knows to protect that directory.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 sequencer.c          | 3 +++
 t/t2501-cwd-empty.sh | 4 ++--
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index ea96837cde3..b71f7b8a0a6 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -4228,6 +4228,9 @@ static int run_git_checkout(struct repository *r, struct replay_opts *opts,
 
 	cmd.git_cmd = 1;
 
+	if (startup_info->original_cwd &&
+	    !is_absolute_path(startup_info->original_cwd))
+		cmd.dir = startup_info->original_cwd;
 	strvec_push(&cmd.args, "checkout");
 	strvec_push(&cmd.args, commit);
 	strvec_pushf(&cmd.env_array, GIT_REFLOG_ACTION "=%s", action);
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index be04b673f6a..acf9646ec6e 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -159,11 +159,11 @@ test_expect_success 'cherry-pick fails if cwd needs to be removed' '
 '
 
 test_expect_success 'rebase does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git rebase reverted
+	test_incidental_dir_removal success git rebase reverted
 '
 
 test_expect_success 'rebase fails if cwd needs to be removed' '
-	test_required_dir_removal failure git rebase fd_conflict
+	test_required_dir_removal success git rebase fd_conflict
 '
 
 test_expect_success 'revert does not clean cwd incidentally' '
-- 
gitgitgadget

[PATCH v3 08/11] stash: do not attempt to remove startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Since stash spawns a `clean` subprocess, make sure we run that from the
startup_info->original_cwd directory, so that the `clean` processs knows
to protect that directory.  Also, since the `clean` command might no
longer run from the toplevel, pass the ':/' magic pathspec to ensure we
still clean from the toplevel.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/stash.c      | 5 ++++-
 t/t2501-cwd-empty.sh | 2 +-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/builtin/stash.c b/builtin/stash.c
index a0ccc8654df..4d89cc7f969 100644
--- a/builtin/stash.c
+++ b/builtin/stash.c
@@ -1485,8 +1485,11 @@ static int do_push_stash(const struct pathspec *ps, const char *stash_msg, int q
 			struct child_process cp = CHILD_PROCESS_INIT;
 
 			cp.git_cmd = 1;
+			if (startup_info->original_cwd &&
+			    !is_absolute_path(startup_info->original_cwd))
+				cp.dir = startup_info->original_cwd;
 			strvec_pushl(&cp.args, "clean", "--force",
-				     "--quiet", "-d", NULL);
+				     "--quiet", "-d", ":/", NULL);
 			if (include_untracked == INCLUDE_ALL_FILES)
 				strvec_push(&cp.args, "-x");
 			if (run_command(&cp)) {
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index acf9646ec6e..6788a0d267f 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -227,7 +227,7 @@ test_expect_success 'clean does not remove cwd incidentally' '
 '
 
 test_expect_success 'stash does not remove cwd incidentally' '
-	test_incidental_untracked_dir_removal failure \
+	test_incidental_untracked_dir_removal success \
 		git stash --include-untracked
 '
 
-- 
gitgitgadget

[PATCH v3 09/11] dir: avoid incidentally removing the original_cwd in remove_path()

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Modern git often tries to avoid leaving empty directories around when
removing files.  Originally, it did not bother.  This behavior started
with commit 80e21a9ed809 (merge-recursive::removeFile: remove empty
directories, 2005-11-19), stating the reason simply as:

    When the last file in a directory is removed as the result of a
    merge, try to rmdir the now-empty directory.

This was reimplemented in C and renamed to remove_path() in commit
e1b3a2cad7 ("Build-in merge-recursive", 2008-02-07), but was still
internal to merge-recursive.

This trend towards removing leading empty directories continued with
commit d9b814cc97f1 (Add builtin "git rm" command, 2006-05-19), which
stated the reasoning as:

    The other question is what to do with leading directories. The old
    "git rm" script didn't do anything, which is somewhat inconsistent.
    This one will actually clean up directories that have become empty
    as a result of removing the last file, but maybe we want to have a
    flag to decide the behaviour?

remove_path() in dir.c was added in 4a92d1bfb784 (Add remove_path: a
function to remove as much as possible of a path, 2008-09-27), because
it was noted that we had two separate implementations of the same idea
AND both were buggy.  It described the purpose of the function as

    a function to remove as much as possible of a path

Why remove as much as possible?  Well, at the time we probably would
have said something like:

  * removing leading directories makes things feel tidy
  * removing leading directories doesn't hurt anything so long as they
    had no files in them.

But I don't believe those reasons hold when the empty directory happens
to be the current working directory we inherited from our parent
process.  Leaving the parent process in a deleted directory can cause
user confusion when subsequent processes fail: any git command, for
example, will immediately fail with

    fatal: Unable to read current working directory: No such file or directory

Other commands may similarly get confused.  Modify remove_path() so that
the empty leading directories it also deletes does not include the
current working directory we inherited from our parent process.  I have
looked through every caller of remove_path() in the current codebase to
make sure that all should take this change.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 dir.c                |  3 +++
 dir.h                |  6 +++++-
 t/t2501-cwd-empty.sh | 12 ++++--------
 3 files changed, 12 insertions(+), 9 deletions(-)

diff --git a/dir.c b/dir.c
index 94489298f4c..97d6b71c872 100644
--- a/dir.c
+++ b/dir.c
@@ -3327,6 +3327,9 @@ int remove_path(const char *name)
 		slash = dirs + (slash - name);
 		do {
 			*slash = '\0';
+			if (startup_info->original_cwd &&
+			    !strcmp(startup_info->original_cwd, dirs))
+				break;
 		} while (rmdir(dirs) == 0 && (slash = strrchr(dirs, '/')));
 		free(dirs);
 	}
diff --git a/dir.h b/dir.h
index 83f46c0fb4c..d6a5d03bec2 100644
--- a/dir.h
+++ b/dir.h
@@ -504,7 +504,11 @@ int get_sparse_checkout_patterns(struct pattern_list *pl);
  */
 int remove_dir_recursively(struct strbuf *path, int flag);
 
-/* tries to remove the path with empty directories along it, ignores ENOENT */
+/*
+ * Tries to remove the path, along with leading empty directories so long as
+ * those empty directories are not startup_info->original_cwd.  Ignores
+ * ENOENT.
+ */
 int remove_path(const char *path);
 
 int fspathcmp(const char *a, const char *b);
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 6788a0d267f..cba817ff734 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -175,12 +175,12 @@ test_expect_success 'revert fails if cwd needs to be removed' '
 '
 
 test_expect_success 'rm does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git rm bar/baz.t
+	test_incidental_dir_removal success git rm bar/baz.t
 '
 
 test_expect_success 'apply does not remove cwd incidentally' '
 	git diff HEAD HEAD~1 >patch &&
-	test_incidental_dir_removal failure git apply ../patch
+	test_incidental_dir_removal success git apply ../patch
 '
 
 test_incidental_untracked_dir_removal () {
@@ -262,12 +262,8 @@ test_expect_success '`rm -rf dir` even with only tracked files will remove somet
 	) &&
 
 	test_path_is_missing a/b/c/tracked &&
-	## We would prefer if a/b was still present, though empty, since it
-	## was the current working directory
-	#test_path_is_dir a/b
-	## But the current behavior is that it not only deletes the directory
-	## a/b as requested, but also goes and deletes a
-	test_path_is_missing a
+	test_path_is_missing a/b/c &&
+	test_path_is_dir a/b
 '
 
 test_expect_success 'git version continues working from a deleted dir' '
-- 
gitgitgadget

[PATCH v3 10/11] dir: new flag to remove_dir_recurse() to spare the original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

remove_dir_recurse(), and its non-static wrapper called
remove_dir_recursively(), both take flags for modifying its behavior.
As with the previous commits, we would generally like to protect
the original_cwd, but we want to forced user commands (e.g. 'git rm -rf
...') or other special cases to remove it.  Add a flag for this purpose.
After reading through every caller of remove_dir_recursively() in the
current codebase, there was only one that should be adjusted and that
one only in a very unusual circumstance.  Add a pair of new testcases to
highlight that very specific case involving submodules && --git-dir &&
--work-tree.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/rm.c         |  3 ++-
 dir.c                | 12 +++++++++---
 dir.h                |  3 +++
 t/t2501-cwd-empty.sh |  5 -----
 4 files changed, 14 insertions(+), 9 deletions(-)

diff --git a/builtin/rm.c b/builtin/rm.c
index 3d0967cdc11..b4132e5d8ee 100644
--- a/builtin/rm.c
+++ b/builtin/rm.c
@@ -399,12 +399,13 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
 	if (!index_only) {
 		int removed = 0, gitmodules_modified = 0;
 		struct strbuf buf = STRBUF_INIT;
+		int flag = force ? REMOVE_DIR_PURGE_ORIGINAL_CWD : 0;
 		for (i = 0; i < list.nr; i++) {
 			const char *path = list.entry[i].name;
 			if (list.entry[i].is_submodule) {
 				strbuf_reset(&buf);
 				strbuf_addstr(&buf, path);
-				if (remove_dir_recursively(&buf, 0))
+				if (remove_dir_recursively(&buf, flag))
 					die(_("could not remove '%s'"), path);
 
 				removed = 1;
diff --git a/dir.c b/dir.c
index 97d6b71c872..52064345a6b 100644
--- a/dir.c
+++ b/dir.c
@@ -3204,6 +3204,7 @@ static int remove_dir_recurse(struct strbuf *path, int flag, int *kept_up)
 	int ret = 0, original_len = path->len, len, kept_down = 0;
 	int only_empty = (flag & REMOVE_DIR_EMPTY_ONLY);
 	int keep_toplevel = (flag & REMOVE_DIR_KEEP_TOPLEVEL);
+	int purge_original_cwd = (flag & REMOVE_DIR_PURGE_ORIGINAL_CWD);
 	struct object_id submodule_head;
 
 	if ((flag & REMOVE_DIR_KEEP_NESTED_GIT) &&
@@ -3259,9 +3260,14 @@ static int remove_dir_recurse(struct strbuf *path, int flag, int *kept_up)
 	closedir(dir);
 
 	strbuf_setlen(path, original_len);
-	if (!ret && !keep_toplevel && !kept_down)
-		ret = (!rmdir(path->buf) || errno == ENOENT) ? 0 : -1;
-	else if (kept_up)
+	if (!ret && !keep_toplevel && !kept_down) {
+		if (!purge_original_cwd &&
+		    startup_info->original_cwd &&
+		    !strcmp(startup_info->original_cwd, path->buf))
+			ret = -1; /* Do not remove current working directory */
+		else
+			ret = (!rmdir(path->buf) || errno == ENOENT) ? 0 : -1;
+	} else if (kept_up)
 		/*
 		 * report the uplevel that it is not an error that we
 		 * did not rmdir() our directory.
diff --git a/dir.h b/dir.h
index d6a5d03bec2..8e02dfb505d 100644
--- a/dir.h
+++ b/dir.h
@@ -495,6 +495,9 @@ int get_sparse_checkout_patterns(struct pattern_list *pl);
 /* Remove the contents of path, but leave path itself. */
 #define REMOVE_DIR_KEEP_TOPLEVEL 04
 
+/* Remove the_original_cwd too */
+#define REMOVE_DIR_PURGE_ORIGINAL_CWD 0x08
+
 /*
  * Remove path and its contents, recursively. flags is a combination
  * of the above REMOVE_DIR_* constants. Return 0 on success.
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index cba817ff734..8f299fd3b19 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -282,11 +282,6 @@ test_submodule_removal () {
 	test_status=
 	test $path_status = dir && test_status=test_must_fail
 
-	# Actually, while path_status == dir && test_status=test_must_fail
-	# reflect our desired behavior, current behavior is:
-	path_status=missing
-	test_status=
-
 	test_when_finished "git reset --hard HEAD~1" &&
 	test_when_finished "rm -rf .git/modules/my_submodule" &&
 
-- 
gitgitgadget

[PATCH v3 11/11] t2501: simplify the tests since we can now assume desired behavior

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

We no longer are dealing with a mixture of previous and desired
behavior, so simplify the tests a bit.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh | 105 +++++++++++--------------------------------
 1 file changed, 27 insertions(+), 78 deletions(-)

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 8f299fd3b19..88e27b16e11 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -32,9 +32,6 @@ test_expect_success setup '
 '
 
 test_incidental_dir_removal () {
-	works=$1 &&
-	shift &&
-
 	test_when_finished "git reset --hard" &&
 
 	git checkout foo/bar/baz^{commit} &&
@@ -44,81 +41,49 @@ test_incidental_dir_removal () {
 		cd foo &&
 		"$@" &&
 
-		# Although we want pwd & git status to pass, test for existing
-		# rather than desired behavior.
-		if [[ $works == "success" ]]; then
-			pwd -P &&
-			git status --porcelain
-		else
-			! pwd -P &&
-			test_might_fail git status --porcelain
-		fi
+		pwd -P &&
+		git status --porcelain
 	) &&
 	test_path_is_missing foo/bar/baz &&
 	test_path_is_missing foo/bar &&
 
-	# Although we want dir to be present, test for existing rather
-	# than desired behavior.
-	if [[ $works == "success" ]]; then
-		test_path_is_dir foo
-	else
-		test_path_is_missing foo
-	fi
+	test_path_is_dir foo
 }
 
 test_required_dir_removal () {
-	works=$1 &&
-	shift &&
-
 	git checkout df_conflict^{commit} &&
 	test_when_finished "git clean -fdx" &&
 
 	(
 		cd dirORfile &&
 
-		# We'd like for the command to fail (much as it would if there
-		# was an untracked file there), and for pwd & git status to
-		# succeed afterwards.  But test for existing rather than
-		# desired behavior.
-		if [[ $works == "success" ]]; then
-			test_must_fail "$@" 2>../error &&
-			grep "Refusing to remove.*current working directory" ../error &&
-			pwd -P &&
-			git status --porcelain
-		else
-			"$@" &&
-			! pwd -P &&
-			test_might_fail git status --porcelain
-		fi
+		test_must_fail "$@" 2>../error &&
+		grep "Refusing to remove.*current working directory" ../error &&
+		pwd -P &&
+		git status --porcelain
 	) &&
 
-	# Although we want dirORfile to be present, test for existing rather
-	# than desired behavior.
-	if [[ $works == "success" ]]; then
-		test_path_is_dir dirORfile
-	else
-		test_path_is_file dirORfile
-	fi
+	test_path_is_dir dirORfile
 }
 
 test_expect_success 'checkout does not clean cwd incidentally' '
-	test_incidental_dir_removal success git checkout init
+	test_incidental_dir_removal git checkout init
 '
 
 test_expect_success 'checkout fails if cwd needs to be removed' '
-	test_required_dir_removal success git checkout fd_conflict
+	test_required_dir_removal git checkout fd_conflict
 '
 
 test_expect_success 'reset --hard does not clean cwd incidentally' '
-	test_incidental_dir_removal success git reset --hard init
+	test_incidental_dir_removal git reset --hard init
 '
 
 test_expect_success 'reset --hard fails if cwd needs to be removed' '
-	test_required_dir_removal success git reset --hard fd_conflict
+	test_required_dir_removal git reset --hard fd_conflict
 '
 
 test_expect_success 'merge does not clean cwd incidentally' '
-	test_incidental_dir_removal success git merge reverted
+	test_incidental_dir_removal git merge reverted
 '
 
 # This file uses some simple merges where
@@ -147,46 +112,43 @@ test_expect_success 'merge fails if cwd needs to be removed; recursive friendly'
 GIT_TEST_MERGE_ALGORITHM=ort
 
 test_expect_success 'merge fails if cwd needs to be removed' '
-	test_required_dir_removal success git merge fd_conflict
+	test_required_dir_removal git merge fd_conflict
 '
 
 test_expect_success 'cherry-pick does not clean cwd incidentally' '
-	test_incidental_dir_removal success git cherry-pick reverted
+	test_incidental_dir_removal git cherry-pick reverted
 '
 
 test_expect_success 'cherry-pick fails if cwd needs to be removed' '
-	test_required_dir_removal success git cherry-pick fd_conflict
+	test_required_dir_removal git cherry-pick fd_conflict
 '
 
 test_expect_success 'rebase does not clean cwd incidentally' '
-	test_incidental_dir_removal success git rebase reverted
+	test_incidental_dir_removal git rebase reverted
 '
 
 test_expect_success 'rebase fails if cwd needs to be removed' '
-	test_required_dir_removal success git rebase fd_conflict
+	test_required_dir_removal git rebase fd_conflict
 '
 
 test_expect_success 'revert does not clean cwd incidentally' '
-	test_incidental_dir_removal success git revert HEAD
+	test_incidental_dir_removal git revert HEAD
 '
 
 test_expect_success 'revert fails if cwd needs to be removed' '
-	test_required_dir_removal success git revert undo_fd_conflict
+	test_required_dir_removal git revert undo_fd_conflict
 '
 
 test_expect_success 'rm does not clean cwd incidentally' '
-	test_incidental_dir_removal success git rm bar/baz.t
+	test_incidental_dir_removal git rm bar/baz.t
 '
 
 test_expect_success 'apply does not remove cwd incidentally' '
 	git diff HEAD HEAD~1 >patch &&
-	test_incidental_dir_removal success git apply ../patch
+	test_incidental_dir_removal git apply ../patch
 '
 
 test_incidental_untracked_dir_removal () {
-	works=$1 &&
-	shift &&
-
 	test_when_finished "git reset --hard" &&
 
 	git checkout foo/bar/baz^{commit} &&
@@ -198,36 +160,23 @@ test_incidental_untracked_dir_removal () {
 		cd untracked &&
 		"$@" &&
 
-		# Although we want pwd & git status to pass, test for existing
-		# rather than desired behavior.
-		if [[ $works == "success" ]]; then
-			pwd -P &&
-			git status --porcelain
-		else
-			! pwd -P &&
-			test_might_fail git status --porcelain
-		fi
+		pwd -P &&
+		git status --porcelain
 	) &&
 	test_path_is_missing empty &&
 	test_path_is_missing untracked/random &&
 
-	# Although we want dir to be present, test for existing rather
-	# than desired behavior.
-	if [[ $works == "success" ]]; then
-		test_path_is_dir untracked
-	else
-		test_path_is_missing untracked
-	fi
+	test_path_is_dir untracked
 }
 
 test_expect_success 'clean does not remove cwd incidentally' '
-	test_incidental_untracked_dir_removal success \
+	test_incidental_untracked_dir_removal \
 		git -C .. clean -fd -e warnings . >warnings &&
 	grep "Refusing to remove current working directory" warnings
 '
 
 test_expect_success 'stash does not remove cwd incidentally' '
-	test_incidental_untracked_dir_removal success \
+	test_incidental_untracked_dir_removal \
 		git stash --include-untracked
 '
 
-- 
gitgitgadget

Re: [PATCH v3 01/11] t2501: add various tests for removing the current working directory

[Ævar Arnfjörð Bjarmason]

On Fri, Nov 26 2021, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>

> +		# Although we want pwd & git status to pass, test for existing
> +		# rather than desired behavior.
> +		if [[ $works == "success" ]]; then

Wasn't "[[" bash-specific or something? In any case a more regular "if
test "$works" = "success" would work here.

> +			pwd -P &&

I wonder if this is doing what we want everywhere cf. 482e1488a9b
(t0001: fix broken not-quite getcwd(3) test in bed67874e2, 2021-07-30),
but haven't looked much/thought about it.

> +test_expect_success 'checkout does not clean cwd incidentally' '
> +	test_incidental_dir_removal failure git checkout init
> +'
> +
> +test_expect_success 'checkout fails if cwd needs to be removed' '
> +	test_required_dir_removal failure git checkout fd_conflict
> +'
> +
> +test_expect_success 'reset --hard does not clean cwd incidentally' '
> +	test_incidental_dir_removal failure git reset --hard init
> +'
> +
> +test_expect_success 'reset --hard fails if cwd needs to be removed' '
> +	test_required_dir_removal failure git reset --hard fd_conflict
> +'
> +
> +test_expect_success 'merge does not clean cwd incidentally' '
> +	test_incidental_dir_removal failure git merge reverted
> +'

This testing the current behavior (and below) looks much better, thanks!

Re: [PATCH v3 02/11] setup: introduce startup_info->original_cwd

[Ævar Arnfjörð Bjarmason]

On Fri, Nov 26 2021, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
> [...]
> +	/*
> +	 * Get our worktree; we only protect the current working directory
> +	 * if it's in the worktree.
> +	 */
> +	worktree = get_git_work_tree();
> +	if (!worktree)
> +		goto no_prevention_needed;
> +
> +	offset = dir_inside_of(startup_info->original_cwd, worktree);
> +	if (offset >= 0) {

Nit: Easier to read as:

    offset = [...]
    if (offset < 0)
    	return;

I.e. the reader can skip that whole "offset >= 0" block and anything
after if it <0, which also reduces the indentation. We run into the
"return' below.

> +		/*
> +		 * If startup_info->original_cwd == worktree, that is already
> +		 * protected and we don't need original_cwd as a secondary
> +		 * protection measure.
> +		 */
> +		if (!*(startup_info->original_cwd + offset))
> +			goto no_prevention_needed;
> +
> +		/*
> +		 * original_cwd was inside worktree; precompose it just as
> +		 * we do prefix so that built up paths will match
> +		 */
> +		startup_info->original_cwd = \

TIL you can use backslashes like that in C outside of macros, but it's
not needed here, better without?

> +			precompose_string_if_needed(startup_info->original_cwd
> +						    + offset);
> +	}
> +	return;

Re: [PATCH v3 02/11] setup: introduce startup_info->original_cwd

[Ævar Arnfjörð Bjarmason]

On Fri, Nov 26 2021, Elijah Newren via GitGitGadget wrote:

[Just some more "I haven't really looked at this in all that much
detail" commentary, so maybe it's stupid, sorry]

> From: Elijah Newren <newren@gmail.com>
>
> Removing the current working directory causes all subsequent git
> commands run from that directory to get confused and fail with a message
> about being unable to read the current working directory:
>
>     $ git status
>     fatal: Unable to read current working directory: No such file or directory
>
> Non-git commands likely have similar warnings or even errors, e.g.
>
>     $ bash -c 'echo hello'
>     shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
>     hello

Is that really realistic? Any "normal" command would use "pwd" or look
at $PWD, both of which "work", this error is only because we're starting
a new shell.

I wonder if it was just because you ran into our bin-wrappers edge case,
but that should be really obscure for any real users.

> This confuses end users, particularly since the command they get the
> error from is not the one that caused the problem; the problem came from
> the side-effect of some previous command.
>
> We would like to avoid removing the current working directory of our
> parent process; towards this end, introduce a new variable,
> startup_info->original_cwd, that tracks the current working directory
> that we inherited from our parent process.  For convenience of later
> comparisons, we prefer that this new variable store a path relative to
> the toplevel working directory (thus much like 'prefix'), except without
> the trailing slash.

I'm still not clear at all on why we need a "original_cwd" at all then
as opposed to just using "prefix" (or adding "the_prefix" if passing it
down is painful). I.e. we discover our relative path, we resolve the
relative path to the root, can't we use that as the "don't remove our
CWD" guard?

Does our prefix change at some point, then maybe "orig_prefix" would
make more sense?

This is with the context that I haven't dug into your code, so maybe
there's some obvious reason I'm missing...

Re: [PATCH v3 02/11] setup: introduce startup_info->original_cwd

[Elijah Newren]

On Sat, Nov 27, 2021 at 2:39 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
>
> On Fri, Nov 26 2021, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> > [...]
> > +     /*
> > +      * Get our worktree; we only protect the current working directory
> > +      * if it's in the worktree.
> > +      */
> > +     worktree = get_git_work_tree();
> > +     if (!worktree)
> > +             goto no_prevention_needed;
> > +
> > +     offset = dir_inside_of(startup_info->original_cwd, worktree);
> > +     if (offset >= 0) {
>
> Nit: Easier to read as:
>
>     offset = [...]
>     if (offset < 0)
>         return;
>
> I.e. the reader can skip that whole "offset >= 0" block and anything
> after if it <0, which also reduces the indentation. We run into the
> "return' below.

Whoops; the return should be inside the braces.  I'll fix.

> > +             /*
> > +              * If startup_info->original_cwd == worktree, that is already
> > +              * protected and we don't need original_cwd as a secondary
> > +              * protection measure.
> > +              */
> > +             if (!*(startup_info->original_cwd + offset))
> > +                     goto no_prevention_needed;
> > +
> > +             /*
> > +              * original_cwd was inside worktree; precompose it just as
> > +              * we do prefix so that built up paths will match
> > +              */
> > +             startup_info->original_cwd = \
>
> TIL you can use backslashes like that in C outside of macros, but it's
> not needed here, better without?

Line is too long without it, so better with it.

>
> > +                     precompose_string_if_needed(startup_info->original_cwd
> > +                                                 + offset);
> > +     }
> > +     return;

Re: [PATCH v3 02/11] setup: introduce startup_info->original_cwd

[Elijah Newren]

On Sat, Nov 27, 2021 at 6:00 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Fri, Nov 26 2021, Elijah Newren via GitGitGadget wrote:
>
> [Just some more "I haven't really looked at this in all that much
> detail" commentary, so maybe it's stupid, sorry]
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > Removing the current working directory causes all subsequent git
> > commands run from that directory to get confused and fail with a message
> > about being unable to read the current working directory:
> >
> >     $ git status
> >     fatal: Unable to read current working directory: No such file or directory
> >
> > Non-git commands likely have similar warnings or even errors, e.g.
> >
> >     $ bash -c 'echo hello'
> >     shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
> >     hello
>
> Is that really realistic?

I have 321 shell scripts (or symlinks thereto) in /usr/bin/ on my
Fedora laptop, and 951 shell scripts in /usr/bin/ on my Ubuntu
workstation at $DAYJOB.  That's not even considering stuff in other
directories.  Note that I didn't place any of those scripts in
/usr/bin/; they came from the distribution and/or corporate packages
from others.  Any invocation of bash will see the above "error"
message.  Granted, bash calls it an 'error' but continues anyway, but
I thought it showed that there were clearly programs besides git where
users could run into problems.

> Any "normal" command would use "pwd" or look
> at $PWD, both of which "work", this error is only because we're starting
> a new shell.

<grin>

Yeah, good point.  Who would use an unusual program like git?  Or a
shell script?  Or a java program?  Or emacs?  Or other programs like
them?  Besides, git, bash, java, and emacs are all relatively young
commands with small development teams.  Maybe we should just rely on
users only using commands that get the pwd/$PWD detail right; perhaps
limiting to commands that are more mature and have bigger development
teams than those four.

Silly users.

;-)

Sorry, couldn't resist a little friendly teasing.


However, for "pwd"...by "work", do you mean "doesn't necessarily
work"?  On my machine:

$ mkdir gone
$ cd gone
$ rmdir ../gone
$ pwd -P
pwd: error retrieving current directory: getcwd: cannot access parent
directories: No such file or directory
$ pwd
pwd: error retrieving current directory: getcwd: cannot access parent
directories: No such file or directory
$ echo $PWD
/home/newren/floss/git/gone

If I do not run `pwd -P` first then a plain `pwd` works.  But your
advice to use `pwd` seems misguided for programs, since they'll hit
this problem if users have run a previous `pwd -P`.  (The $PWD trick
would have worked, as shown above)

> I wonder if it was just because you ran into our bin-wrappers edge case,
> but that should be really obscure for any real users.

Yes, I agree our bin-wrappers is a really obscure edge case not worth
considering.  I don't see how you jump from there to painting all
shell scripts combined with the same brush, though.

> > This confuses end users, particularly since the command they get the
> > error from is not the one that caused the problem; the problem came from
> > the side-effect of some previous command.
> >
> > We would like to avoid removing the current working directory of our
> > parent process; towards this end, introduce a new variable,
> > startup_info->original_cwd, that tracks the current working directory
> > that we inherited from our parent process.  For convenience of later
> > comparisons, we prefer that this new variable store a path relative to
> > the toplevel working directory (thus much like 'prefix'), except without
> > the trailing slash.
>
> I'm still not clear at all on why we need a "original_cwd" at all then
> as opposed to just using "prefix" (or adding "the_prefix" if passing it
> down is painful). I.e. we discover our relative path, we resolve the
> relative path to the root, can't we use that as the "don't remove our
> CWD" guard?
>
> Does our prefix change at some point, then maybe "orig_prefix" would
> make more sense?

No; see the code comment in the same patch:

+       /*
+        * startup_info->original_cwd points to the current working
+        * directory we inherited from our parent process, which is a
+        * directory we want to avoid removing.
+        *
+        * For convience, we would like to have the path relative to the
+        * worktree instead of an absolute path.
+        *
+        * Yes, startup_info->original_cwd is usually the same as 'prefix',
+        * but differs in two ways:
+        *   - prefix has a trailing '/'
+        *   - if the user passes '-C' to git, that modifies the prefix but
+        *     not startup_info->original_cwd.
+        */

It's never equal to prefix, even though it's usually semantically
referring to the same directory.  However, even if it weren't for the
trailing slash issue, the -C case means it is not appropriate to think
of it as "orig_prefix" either.

Re: [PATCH v3 01/11] t2501: add various tests for removing the current working directory

[Elijah Newren]

On Sat, Nov 27, 2021 at 2:35 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Fri, Nov 26 2021, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
>
> > +             # Although we want pwd & git status to pass, test for existing
> > +             # rather than desired behavior.
> > +             if [[ $works == "success" ]]; then
>
> Wasn't "[[" bash-specific or something? In any case a more regular "if
> test "$works" = "success" would work here.

Thanks; will fix.

> > +                     pwd -P &&
>
> I wonder if this is doing what we want everywhere cf. 482e1488a9b
> (t0001: fix broken not-quite getcwd(3) test in bed67874e2, 2021-07-30),
> but haven't looked much/thought about it.

I can change to `test-tool getcwd` for the obscure platforms; thanks
for the tip.

> > +test_expect_success 'checkout does not clean cwd incidentally' '
> > +     test_incidental_dir_removal failure git checkout init
> > +'
> > +
> > +test_expect_success 'checkout fails if cwd needs to be removed' '
> > +     test_required_dir_removal failure git checkout fd_conflict
> > +'
> > +
> > +test_expect_success 'reset --hard does not clean cwd incidentally' '
> > +     test_incidental_dir_removal failure git reset --hard init
> > +'
> > +
> > +test_expect_success 'reset --hard fails if cwd needs to be removed' '
> > +     test_required_dir_removal failure git reset --hard fd_conflict
> > +'
> > +
> > +test_expect_success 'merge does not clean cwd incidentally' '
> > +     test_incidental_dir_removal failure git merge reverted
> > +'
>
> This testing the current behavior (and below) looks much better, thanks!

:-)

Re: [PATCH v3 02/11] setup: introduce startup_info->original_cwd

[Ævar Arnfjörð Bjarmason]

On Sat, Nov 27 2021, Elijah Newren wrote:

> On Sat, Nov 27, 2021 at 6:00 AM Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>>
>> On Fri, Nov 26 2021, Elijah Newren via GitGitGadget wrote:
>>
>> [Just some more "I haven't really looked at this in all that much
>> detail" commentary, so maybe it's stupid, sorry]
>>
>> > From: Elijah Newren <newren@gmail.com>
>> >
>> > Removing the current working directory causes all subsequent git
>> > commands run from that directory to get confused and fail with a message
>> > about being unable to read the current working directory:
>> >
>> >     $ git status
>> >     fatal: Unable to read current working directory: No such file or directory
>> >
>> > Non-git commands likely have similar warnings or even errors, e.g.
>> >
>> >     $ bash -c 'echo hello'
>> >     shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
>> >     hello
>>
>> Is that really realistic?
>
> I have 321 shell scripts (or symlinks thereto) in /usr/bin/ on my
> Fedora laptop, and 951 shell scripts in /usr/bin/ on my Ubuntu
> workstation at $DAYJOB.  That's not even considering stuff in other
> directories.  Note that I didn't place any of those scripts in
> /usr/bin/; they came from the distribution and/or corporate packages
> from others.  Any invocation of bash will see the above "error"
> message.  Granted, bash calls it an 'error' but continues anyway, but
> I thought it showed that there were clearly programs besides git where
> users could run into problems.
>
>> Any "normal" command would use "pwd" or look
>> at $PWD, both of which "work", this error is only because we're starting
>> a new shell.
>
> <grin>
>
> Yeah, good point.  Who would use an unusual program like git?  Or a
> shell script?  Or a java program?  Or emacs?  Or other programs like
> them?  Besides, git, bash, java, and emacs are all relatively young
> commands with small development teams.  Maybe we should just rely on
> users only using commands that get the pwd/$PWD detail right; perhaps
> limiting to commands that are more mature and have bigger development
> teams than those four.
>
> Silly users.
>
> ;-)
>
> Sorry, couldn't resist a little friendly teasing.
>
>
> However, for "pwd"...by "work", do you mean "doesn't necessarily
> work"?  On my machine:
>
> $ mkdir gone
> $ cd gone
> $ rmdir ../gone
> $ pwd -P
> pwd: error retrieving current directory: getcwd: cannot access parent
> directories: No such file or directory
> $ pwd
> pwd: error retrieving current directory: getcwd: cannot access parent
> directories: No such file or directory
> $ echo $PWD
> /home/newren/floss/git/gone
>
> If I do not run `pwd -P` first then a plain `pwd` works.  But your
> advice to use `pwd` seems misguided for programs, since they'll hit
> this problem if users have run a previous `pwd -P`.  (The $PWD trick
> would have worked, as shown above)
>
>> I wonder if it was just because you ran into our bin-wrappers edge case,
>> but that should be really obscure for any real users.

For some reason I was under the misimpression that the "#!/bin/bash"
part of the bin-wrappers and other scripts somehow immunized them from
the $PWD/"pwd" reset, and it was only the programs they invoked (like
git in the bin-wrappers) that didn't get the values passed along.

But that's clearly incorrect as you demonstrate above, so the only thing
that'll work OK (seemingly) is running "pwd" (but not "pwd -P"), or
looking at $PWD in your terminal shell itself.

Invoking non-POSIX shell programs "works" in that they can use the same
trick, after the dance of "mkdir x && cd x && rm -rf ..x" e.g. Perl
says:
    
    $ perl -MCwd=getcwd -wE 'say $ENV{PWD}; say getcwd'
    /home/avar/g/git/x
    Use of uninitialized value in say at -e line 1

This "pwd -P" behavior isn't just something weird in your shell & mine,
it semse to be mandated by POSIX:
https://pubs.opengroup.org/onlinepubs/007904875/utilities/pwd.html

All of which is to say I'm much more sympathetic to this approach
now. I.e. it seemed like purely a way to work around fixable breakages
in our own and other programs. Well, I guess "don't use shellscript if
you care about this edge case" is a "fix", but not a trivial one.

I do still think a better starting point would be fixing the setup.c
dying in our own code, and see where that leaves us, but up to you
etc. I'm not going to work on it any time soon.

> Yes, I agree our bin-wrappers is a really obscure edge case not worth
> considering.  I don't see how you jump from there to painting all
> shell scripts combined with the same brush, though.

*nod*

>> > This confuses end users, particularly since the command they get the
>> > error from is not the one that caused the problem; the problem came from
>> > the side-effect of some previous command.
>> >
>> > We would like to avoid removing the current working directory of our
>> > parent process; towards this end, introduce a new variable,
>> > startup_info->original_cwd, that tracks the current working directory
>> > that we inherited from our parent process.  For convenience of later
>> > comparisons, we prefer that this new variable store a path relative to
>> > the toplevel working directory (thus much like 'prefix'), except without
>> > the trailing slash.
>>
>> I'm still not clear at all on why we need a "original_cwd" at all then
>> as opposed to just using "prefix" (or adding "the_prefix" if passing it
>> down is painful). I.e. we discover our relative path, we resolve the
>> relative path to the root, can't we use that as the "don't remove our
>> CWD" guard?
>>
>> Does our prefix change at some point, then maybe "orig_prefix" would
>> make more sense?
>
> No; see the code comment in the same patch:
>
> +       /*
> +        * startup_info->original_cwd points to the current working
> +        * directory we inherited from our parent process, which is a
> +        * directory we want to avoid removing.
> +        *
> +        * For convience, we would like to have the path relative to the
> +        * worktree instead of an absolute path.
> +        *
> +        * Yes, startup_info->original_cwd is usually the same as 'prefix',
> +        * but differs in two ways:
> +        *   - prefix has a trailing '/'
> +        *   - if the user passes '-C' to git, that modifies the prefix but
> +        *     not startup_info->original_cwd.
> +        */
>
> It's never equal to prefix, even though it's usually semantically
> referring to the same directory.  However, even if it weren't for the
> trailing slash issue, the -C case means it is not appropriate to think
> of it as "orig_prefix" either.

Ah, with -C of e.g. t/helper we'll first chdir(t/helper), and then run
the usual setup.c dance to find that we need to chdir() again to the
(equivalent of) "../../". But our prefix stays at "t/helper".

I'm a bit confused about the trailing slash case, isn't the prefix
always going to point to a directory? Why would t/helper v.s. t/helper/
matter?

I think it won't matter for rmdir(2) et al, but maybe I'm wrong.

What got me confused about the "prefix" v.s. "original_cwd" is that I
was assuming they'd be the same. The commentary on setup_git_directory()
says as much, i.e.:

    Returns the "prefix", a path to the current working directory
    relative to the work tree root, or NULL,

But of course we know that's a white lie, it's not the $PWD/getcwd(). So
you're only trying to save the user in cases of e.g. (in t/helper):

    git rm ../helper

If they actually run it while in t/helper, but would like to explicitly
omit the case of (at the top-level):

    git -C t/helper rm ../helper

That's fair enough I guess. I'd just assumed those cases would be
treated the same way.

Even in that case, I can't think of a case where this "original_cwd"
wouldn't be made redundant by some boolean flag to accompany "prefix" to
indicate that we've chdir'd out of the prefix (or not).

Re: [PATCH v2 2/9] setup: introduce startup_info->original_cwd

[Derrick Stolee]

On 11/25/2021 3:39 AM, Elijah Newren via GitGitGadget wrote:
> From: Elijah Newren <newren@gmail.com>

> @@ -866,6 +866,8 @@ int cmd_main(int argc, const char **argv)
>  
>  	trace_command_performance(argv);
>  
> +	startup_info->original_cwd = xgetcwd();
> +

I see this initial assignment in cmd_main()...

> +static void setup_original_cwd(void)
> +{
> +	struct strbuf tmp = STRBUF_INIT;
> +	const char *worktree = NULL;
> +	int offset = -1;
> +
> +	/*
> +	 * startup_info->original_cwd wass set early on in cmd_main(), unless
> +	 * we're an auxiliary tool like git-remote-http or test-tool.
> +	 */
> +	if (!startup_info->original_cwd)
> +		return;

...which is assumed to be run before this method was called...

> @@ -1330,6 +1378,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
>  		setenv(GIT_PREFIX_ENVIRONMENT, "", 1);
>  	}
>  
> +	setup_original_cwd();

...here in setup_git_directory_gently().

Why do we need that assignment in cmd_main()? Could we instead
let setup_original_cwd() do the initial assignment? Or is it
possible that a chdir has happened already before this point?

Thanks,
-Stolee

Re: [PATCH v2 3/9] unpack-trees: refuse to remove startup_info->original_cwd

[Derrick Stolee]

On 11/25/2021 3:39 AM, Elijah Newren via GitGitGadget wrote:
> From: Elijah Newren <newren@gmail.com>

> @@ -36,6 +36,9 @@ static const char *unpack_plumbing_errors[NB_UNPACK_TREES_WARNING_TYPES] = {
>  	/* ERROR_NOT_UPTODATE_DIR */
>  	"Updating '%s' would lose untracked files in it",
>  
> +	/* ERROR_CWD_IN_THE_WAY */
> +	"Refusing to remove '%s' since it is the current working directory.",
> +
>  	/* ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN */
>  	"Untracked working tree file '%s' would be overwritten by merge.",

Your new message includes a hard stop (".") which is non-standard. I
see that the message after yours has one, but the preceding one does
not. Since the file you are in is not consistent, I would choose to
drop the hard stop here.

Thanks,
-Stolee

Re: [PATCH v2 4/9] unpack-trees: add special cwd handling

[Derrick Stolee]

On 11/25/2021 3:39 AM, Elijah Newren via GitGitGadget wrote:
> From: Elijah Newren <newren@gmail.com>
> 
> When running commands such as `git reset --hard` from a subdirectory, if
> that subdirectory is in the way of adding needed files, bail with an
> error message.
...
> -test_expect_failure 'reset --hard fails if cwd needs to be removed' '
> +test_expect_success 'reset --hard fails if cwd needs to be removed' '
>  	git checkout foo/bar/baz &&
>  	test_when_finished "git clean -fdx" &&

This test doesn't include any condition on the final state of
the worktree. I could imagine that an implementation could be
writing to the worktree incrementally and then bail halfway
through. A clean "git status" would give us some confidence that
the worktree and index are in a good state after the command
fails.

> +	if (o->reset == UNPACK_RESET_OVERWRITE_UNTRACKED) {
> +		/* Avoid nuking startup_info->original_cwd... */
> +		if (startup_info->original_cwd &&
> +		    !strcmp(startup_info->original_cwd, ce->name))
> +			return add_rejected_path(o, ERROR_CWD_IN_THE_WAY,
> +						 ce->name);
> +		/* ...but nuke anything else. */
> +		return 0;
> +	}
> +

This implementation appears to do the right thing in the case
I'm describing: we check which entries would need to change and
we reject them before doing any real work.

Thanks,
-Stolee

Re: [PATCH v2 2/9] setup: introduce startup_info->original_cwd

[Elijah Newren]

On Mon, Nov 29, 2021 at 6:05 AM Derrick Stolee <stolee@gmail.com> wrote:
>
> On 11/25/2021 3:39 AM, Elijah Newren via GitGitGadget wrote:
> > From: Elijah Newren <newren@gmail.com>
>
> > @@ -866,6 +866,8 @@ int cmd_main(int argc, const char **argv)
> >
> >       trace_command_performance(argv);
> >
> > +     startup_info->original_cwd = xgetcwd();
> > +
>
> I see this initial assignment in cmd_main()...

It looks like you accidentally responded to v2 when there's a v3
(something I occasionally do too).  v3 changes this to put it in
common-main instead of here, as suggested by Ævar, but to answer the
question...

> > +static void setup_original_cwd(void)
> > +{
> > +     struct strbuf tmp = STRBUF_INIT;
> > +     const char *worktree = NULL;
> > +     int offset = -1;
> > +
> > +     /*
> > +      * startup_info->original_cwd wass set early on in cmd_main(), unless
> > +      * we're an auxiliary tool like git-remote-http or test-tool.
> > +      */
> > +     if (!startup_info->original_cwd)
> > +             return;
>
> ...which is assumed to be run before this method was called...
>
> > @@ -1330,6 +1378,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
> >               setenv(GIT_PREFIX_ENVIRONMENT, "", 1);
> >       }
> >
> > +     setup_original_cwd();
>
> ...here in setup_git_directory_gently().
>
> Why do we need that assignment in cmd_main()? Could we instead
> let setup_original_cwd() do the initial assignment? Or is it
> possible that a chdir has happened already before this point?

In v1, I made that mistake.  Then I realized that when users pass the
-C option to git, there is a chdir() call immediately upon parsing of
the -C option.  So I had to move the strbuf_getcwd() call earlier.

Re: [PATCH v2 3/9] unpack-trees: refuse to remove startup_info->original_cwd

[Elijah Newren]

On Mon, Nov 29, 2021 at 6:10 AM Derrick Stolee <stolee@gmail.com> wrote:
>
> On 11/25/2021 3:39 AM, Elijah Newren via GitGitGadget wrote:
> > From: Elijah Newren <newren@gmail.com>
>
> > @@ -36,6 +36,9 @@ static const char *unpack_plumbing_errors[NB_UNPACK_TREES_WARNING_TYPES] = {
> >       /* ERROR_NOT_UPTODATE_DIR */
> >       "Updating '%s' would lose untracked files in it",
> >
> > +     /* ERROR_CWD_IN_THE_WAY */
> > +     "Refusing to remove '%s' since it is the current working directory.",
> > +
> >       /* ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN */
> >       "Untracked working tree file '%s' would be overwritten by merge.",
>
> Your new message includes a hard stop (".") which is non-standard. I
> see that the message after yours has one, but the preceding one does
> not. Since the file you are in is not consistent, I would choose to
> drop the hard stop here.

Seems pretty reasonable, but 9 out of 10 of the messages in that list
contain the hard stop.  True, that's still inconsistent, but adding
the hard stop feels more internally consistent.  I think I'd rather
have someone else post a subsequent cleanup to remove the hard stops
and lowercase the first letter and whatever else needs to be changed
with these messages.

Re: [PATCH v2 4/9] unpack-trees: add special cwd handling

[Elijah Newren]

On Mon, Nov 29, 2021 at 6:14 AM Derrick Stolee <stolee@gmail.com> wrote:
>
> On 11/25/2021 3:39 AM, Elijah Newren via GitGitGadget wrote:
> > From: Elijah Newren <newren@gmail.com>
> >
> > When running commands such as `git reset --hard` from a subdirectory, if
> > that subdirectory is in the way of adding needed files, bail with an
> > error message.
> ...
> > -test_expect_failure 'reset --hard fails if cwd needs to be removed' '
> > +test_expect_success 'reset --hard fails if cwd needs to be removed' '
> >       git checkout foo/bar/baz &&
> >       test_when_finished "git clean -fdx" &&
>
> This test doesn't include any condition on the final state of
> the worktree. I could imagine that an implementation could be
> writing to the worktree incrementally and then bail halfway
> through. A clean "git status" would give us some confidence that
> the worktree and index are in a good state after the command
> fails.

Makes sense; I can add a
   git diff --exit-code HEAD &&
to the test.

> > +     if (o->reset == UNPACK_RESET_OVERWRITE_UNTRACKED) {
> > +             /* Avoid nuking startup_info->original_cwd... */
> > +             if (startup_info->original_cwd &&
> > +                 !strcmp(startup_info->original_cwd, ce->name))
> > +                     return add_rejected_path(o, ERROR_CWD_IN_THE_WAY,
> > +                                              ce->name);
> > +             /* ...but nuke anything else. */
> > +             return 0;
> > +     }
> > +
>
> This implementation appears to do the right thing in the case
> I'm describing: we check which entries would need to change and
> we reject them before doing any real work.

Thanks for taking a look.

Re: [PATCH v2 2/9] setup: introduce startup_info->original_cwd

[Junio C Hamano]

Derrick Stolee <stolee@gmail.com> writes:

>> @@ -1330,6 +1378,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
>>  		setenv(GIT_PREFIX_ENVIRONMENT, "", 1);
>>  	}
>>  
>> +	setup_original_cwd();
>
> ...here in setup_git_directory_gently().
>
> Why do we need that assignment in cmd_main()? Could we instead
> let setup_original_cwd() do the initial assignment? Or is it
> possible that a chdir has happened already before this point?

Since setup_git_directory() is used to figure out the prefix, which
is relative path to the directory, in which the command was started,
viewed from the top-level of the working tree, it would be a bug if
the caller did any chdir(2) before it called the function.

I didn't look at or thought about how chdir(2) done inside this
function affects what setup_original_cwd() wants to do, though.

Thanks.

Re: [PATCH v2 2/9] setup: introduce startup_info->original_cwd

[Derrick Stolee]

On 11/29/2021 12:18 PM, Elijah Newren wrote:
> On Mon, Nov 29, 2021 at 6:05 AM Derrick Stolee <stolee@gmail.com> wrote:
>>
>> On 11/25/2021 3:39 AM, Elijah Newren via GitGitGadget wrote:
>>> From: Elijah Newren <newren@gmail.com>
>>
>>> @@ -866,6 +866,8 @@ int cmd_main(int argc, const char **argv)
>>>
>>>       trace_command_performance(argv);
>>>
>>> +     startup_info->original_cwd = xgetcwd();
>>> +
>>
>> I see this initial assignment in cmd_main()...
> 
> It looks like you accidentally responded to v2 when there's a v3
> (something I occasionally do too).  v3 changes this to put it in
> common-main instead of here, as suggested by Ævar, but to answer the
> question...

Yes, sorry about that. My inbox was delayed in showing me that a v3
existed until I was halfway through reviewing v2. (It then only
showed me half of the patches from v3, so something was causing
a delay.)

>>> +static void setup_original_cwd(void)
>>> +{
>>> +     struct strbuf tmp = STRBUF_INIT;
>>> +     const char *worktree = NULL;
>>> +     int offset = -1;
>>> +
>>> +     /*
>>> +      * startup_info->original_cwd wass set early on in cmd_main(), unless
>>> +      * we're an auxiliary tool like git-remote-http or test-tool.
>>> +      */
>>> +     if (!startup_info->original_cwd)
>>> +             return;
>>
>> ...which is assumed to be run before this method was called...
>>
>>> @@ -1330,6 +1378,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
>>>               setenv(GIT_PREFIX_ENVIRONMENT, "", 1);
>>>       }
>>>
>>> +     setup_original_cwd();
>>
>> ...here in setup_git_directory_gently().
>>
>> Why do we need that assignment in cmd_main()? Could we instead
>> let setup_original_cwd() do the initial assignment? Or is it
>> possible that a chdir has happened already before this point?
> 
> In v1, I made that mistake.  Then I realized that when users pass the
> -C option to git, there is a chdir() call immediately upon parsing of
> the -C option.  So I had to move the strbuf_getcwd() call earlier.

Ok. I wonder if we could setup_original_cwd() earlier than that
parsing, but I'm sure you've already explored that option.

Thanks,
-Stolee

Re: [PATCH v3 07/11] rebase: do not attempt to remove startup_info->original_cwd

[Derrick Stolee]

On 11/26/2021 5:40 PM, Elijah Newren via GitGitGadget wrote:
> From: Elijah Newren <newren@gmail.com>
> 
> Since rebase spawns a `checkout` subprocess, make sure we run that from
> the startup_info->original_cwd directory, so that the checkout process
> knows to protect that directory.
> 
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  sequencer.c          | 3 +++
>  t/t2501-cwd-empty.sh | 4 ++--
>  2 files changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/sequencer.c b/sequencer.c
> index ea96837cde3..b71f7b8a0a6 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -4228,6 +4228,9 @@ static int run_git_checkout(struct repository *r, struct replay_opts *opts,
>  
>  	cmd.git_cmd = 1;
>  
> +	if (startup_info->original_cwd &&
> +	    !is_absolute_path(startup_info->original_cwd))
> +		cmd.dir = startup_info->original_cwd;

I was initially confused by the "!is_absolute_path()" because
it seemed to me like it would be natural to store an absolute
path there, but I see this comment in patch 2:

+	 * For convience, we would like to have the path relative to the
+	 * worktree instead of an absolute path.

So it seems that we won't store it as an absolute path. Is
there any value in this condition, then?

This assignment of cmd.dir to the relative path has a lot
of baked-in knowledge of this variable _and_ the current
state (Git chdir()'d to the root of the worktree). If the
path is always relative, then it should be a BUG() if we
see an absolute path. Also, it seems like we would want
cmd.dir to be a concatenation of the worktree root and the
original_cwd.

Or perhaps I'm being overly cautious and this could all be
resolved with a comment about the expected state of the
working directory and original_cwd. The tests will catch if
any of those expectations change.

Thanks,
-Stolee

Re: [PATCH v3 00/11] Avoid removing the current working directory, even if it becomes empty

[Derrick Stolee]

On 11/26/2021 5:40 PM, Elijah Newren via GitGitGadget wrote:
> Traditionally, if folks run git commands such as checkout or rebase from a
> subdirectory, that git command could remove their current working directory
> and result in subsequent git and non-git commands either getting confused or
> printing messages that confuse the user (e.g. "fatal: Unable to read current
> working directory: No such file or directory"). Many commands either
> silently avoid removing directories that are not empty (i.e. those that have
> untracked or modified files in them)[1], or show an error and abort,
> depending on which is more appropriate for the command in question. With
> this series, we augment the reasons to avoid removing directories to include
> not just has-untracked-or-modified-files, but also to avoid removing the
> original_cwd as well.

As I was reviewing v2, this version popped up in my inbox. Sorry about that.

The only actionable comment from my review of v2 was the addition of a check
that the worktree is in the expected state after commands are aborted due to
trying to remove the current working directory. Your suggested

	git diff --exit-code HEAD

should work for this. I might even add a "git rev-parse HEAD" to make sure
we are on the right ref, but that's perhaps too specific to something like
'git reset --hard <branch>'.

> Changes since v2:
> 
>  * the series is now only about the working tree. So if the original cwd is
>    outside the worktree (or we're in a bare repo), then the new code is a
>    no-op.
>  * fixed ugly early die() possibility (uses strbuf_getcwd() instead of
>    xgetcwd())
>  * modified the initial tests to show both expected and desired behavior.
>    subsequent patches fix the tests. One new patch added at the end which
>    simplifies the tests to only check for desired behavior.
>  * NULLify startup_info->original_cwd when it matches the toplevel worktree;
>    that is already protected and we don't need secondary protection for it.
>    This simplified some other codepaths so we don't have to check for
>    startup_info->original_cwd == "".
>  * clarified some commit messages

Looking at these changes I like all of them.

> Range-diff vs v2:
> 
>   1:  38a120f5c03 !  1:  4b0044656b0 t2501: add various tests for removing the current working directory

I like this new test structure. Using test_expect_success to document existing
behavior is a good strategy.

>   2:  f6129a8ac9c !  2:  200ddece05d setup: introduce startup_info->original_cwd>   3:  e74975e83cc !  3:  68ae90546fe unpack-trees: refuse to remove startup_info->original_cwd
>   4:  e06806e3a32 !  4:  1bb8905900c unpack-trees: add special cwd handling
>   5:  46728f74ea1 !  5:  8a33d74e7cf symlinks: do not include startup_info->original_cwd in dir removal
>   6:  01ce9444dae !  6:  11e4ec881bb clean: do not attempt to remove startup_info->original_cwd
These changes looked good.

>   -:  ----------- >  7:  39b1f3a225e rebase: do not attempt to remove startup_info->original_cwd

I had a small comment on this one, only because I think there is a condition
in your 'if' statement that is either unhelpful or is hiding something.

>   7:  edec0894ca2 !  8:  0110462a19c stash: do not attempt to remove startup_info->original_cwd
...
>        			cp.git_cmd = 1;
>       +			if (startup_info->original_cwd &&
>      -+			    *startup_info->original_cwd &&
>       +			    !is_absolute_path(startup_info->original_cwd))
>       +				cp.dir = startup_info->original_cwd;

And here is a similar use of is_absolute_path() that could be resolved the
same way as whatever you choose for the v3 patch 7.

>   8:  1815f18592b !  9:  2c73a09a2e8 dir: avoid incidentally removing the original_cwd in remove_path()
>   9:  adaad7aeaac ! 10:  d4e50b4053d dir: new flag to remove_dir_recurse() to spare the original_cwd
>   -:  ----------- > 11:  7eb6281be4b t2501: simplify the tests since we can now assume desired behavior

This last test cleanup is good to have.

Thanks,
-Stolee

Re: [PATCH v3 07/11] rebase: do not attempt to remove startup_info->original_cwd

[Elijah Newren]

On Mon, Nov 29, 2021 at 9:50 AM Derrick Stolee <stolee@gmail.com> wrote:
>
> On 11/26/2021 5:40 PM, Elijah Newren via GitGitGadget wrote:
> > From: Elijah Newren <newren@gmail.com>
> >
> > Since rebase spawns a `checkout` subprocess, make sure we run that from
> > the startup_info->original_cwd directory, so that the checkout process
> > knows to protect that directory.
> >
> > Signed-off-by: Elijah Newren <newren@gmail.com>
> > ---
> >  sequencer.c          | 3 +++
> >  t/t2501-cwd-empty.sh | 4 ++--
> >  2 files changed, 5 insertions(+), 2 deletions(-)
> >
> > diff --git a/sequencer.c b/sequencer.c
> > index ea96837cde3..b71f7b8a0a6 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -4228,6 +4228,9 @@ static int run_git_checkout(struct repository *r, struct replay_opts *opts,
> >
> >       cmd.git_cmd = 1;
> >
> > +     if (startup_info->original_cwd &&
> > +         !is_absolute_path(startup_info->original_cwd))
> > +             cmd.dir = startup_info->original_cwd;
>
> I was initially confused by the "!is_absolute_path()" because
> it seemed to me like it would be natural to store an absolute
> path there, but I see this comment in patch 2:
>
> +        * For convience, we would like to have the path relative to the
> +        * worktree instead of an absolute path.
>
> So it seems that we won't store it as an absolute path. Is
> there any value in this condition, then?

Good catch.  This is leftover from an earlier round; I'll remove it.
From stash too.

> This assignment of cmd.dir to the relative path has a lot
> of baked-in knowledge of this variable _and_ the current
> state (Git chdir()'d to the root of the worktree). If the
> path is always relative, then it should be a BUG() if we
> see an absolute path. Also, it seems like we would want
> cmd.dir to be a concatenation of the worktree root and the
> original_cwd.
>
> Or perhaps I'm being overly cautious and this could all be
> resolved with a comment about the expected state of the
> working directory and original_cwd. The tests will catch if
> any of those expectations change.

Yeah, with the extra condition in there I should have added a good
explanation about why I had it in there.  But I think a simple

if (startup_info->original_cwd)
    cmd.dir = startup_info->original_cwd;

should be good enough?  Or do you still want a comment for that?

Re: [PATCH v3 07/11] rebase: do not attempt to remove startup_info->original_cwd

[Derrick Stolee]

On 11/29/2021 2:22 PM, Elijah Newren wrote:
> On Mon, Nov 29, 2021 at 9:50 AM Derrick Stolee <stolee@gmail.com> wrote:
>>
>> On 11/26/2021 5:40 PM, Elijah Newren via GitGitGadget wrote:
>>>       cmd.git_cmd = 1;
>>>
>>> +     if (startup_info->original_cwd &&
>>> +         !is_absolute_path(startup_info->original_cwd))
>>> +             cmd.dir = startup_info->original_cwd;
>>
>> I was initially confused by the "!is_absolute_path()" because
>> it seemed to me like it would be natural to store an absolute
>> path there, but I see this comment in patch 2:
>>
>> +        * For convience, we would like to have the path relative to the
>> +        * worktree instead of an absolute path.
>>
>> So it seems that we won't store it as an absolute path. Is
>> there any value in this condition, then?
> 
> Good catch.  This is leftover from an earlier round; I'll remove it.
> From stash too.
> 
>> This assignment of cmd.dir to the relative path has a lot
>> of baked-in knowledge of this variable _and_ the current
>> state (Git chdir()'d to the root of the worktree). If the
>> path is always relative, then it should be a BUG() if we
>> see an absolute path. Also, it seems like we would want
>> cmd.dir to be a concatenation of the worktree root and the
>> original_cwd.
>>
>> Or perhaps I'm being overly cautious and this could all be
>> resolved with a comment about the expected state of the
>> working directory and original_cwd. The tests will catch if
>> any of those expectations change.
> 
> Yeah, with the extra condition in there I should have added a good
> explanation about why I had it in there.  But I think a simple
> 
> if (startup_info->original_cwd)
>     cmd.dir = startup_info->original_cwd;
> 
> should be good enough?  Or do you still want a comment for that?
 
This is the code I would expect to see. No comment needed.

Thanks,
-Stolee

Re: [PATCH v3 02/11] setup: introduce startup_info->original_cwd

[Elijah Newren]

On Mon, Nov 29, 2021 at 12:41 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
>
> On Sat, Nov 27 2021, Elijah Newren wrote:
>
> > On Sat, Nov 27, 2021 at 6:00 AM Ævar Arnfjörð Bjarmason
> > <avarab@gmail.com> wrote:
> >>
> >> On Fri, Nov 26 2021, Elijah Newren via GitGitGadget wrote:
> >>
> >> [Just some more "I haven't really looked at this in all that much
> >> detail" commentary, so maybe it's stupid, sorry]
> >>
> >> > From: Elijah Newren <newren@gmail.com>
> >> >
> >> > Removing the current working directory causes all subsequent git
> >> > commands run from that directory to get confused and fail with a message
> >> > about being unable to read the current working directory:
> >> >
> >> >     $ git status
> >> >     fatal: Unable to read current working directory: No such file or directory
> >> >
> >> > Non-git commands likely have similar warnings or even errors, e.g.
> >> >
> >> >     $ bash -c 'echo hello'
> >> >     shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
> >> >     hello
> >>
> >> Is that really realistic?
> >
> > I have 321 shell scripts (or symlinks thereto) in /usr/bin/ on my
> > Fedora laptop, and 951 shell scripts in /usr/bin/ on my Ubuntu
> > workstation at $DAYJOB.  That's not even considering stuff in other
> > directories.  Note that I didn't place any of those scripts in
> > /usr/bin/; they came from the distribution and/or corporate packages
> > from others.  Any invocation of bash will see the above "error"
> > message.  Granted, bash calls it an 'error' but continues anyway, but
> > I thought it showed that there were clearly programs besides git where
> > users could run into problems.
> >
> >> Any "normal" command would use "pwd" or look
> >> at $PWD, both of which "work", this error is only because we're starting
> >> a new shell.
> >
> > <grin>
> >
> > Yeah, good point.  Who would use an unusual program like git?  Or a
> > shell script?  Or a java program?  Or emacs?  Or other programs like
> > them?  Besides, git, bash, java, and emacs are all relatively young
> > commands with small development teams.  Maybe we should just rely on
> > users only using commands that get the pwd/$PWD detail right; perhaps
> > limiting to commands that are more mature and have bigger development
> > teams than those four.
> >
> > Silly users.
> >
> > ;-)
> >
> > Sorry, couldn't resist a little friendly teasing.
> >
> >
> > However, for "pwd"...by "work", do you mean "doesn't necessarily
> > work"?  On my machine:
> >
> > $ mkdir gone
> > $ cd gone
> > $ rmdir ../gone
> > $ pwd -P
> > pwd: error retrieving current directory: getcwd: cannot access parent
> > directories: No such file or directory
> > $ pwd
> > pwd: error retrieving current directory: getcwd: cannot access parent
> > directories: No such file or directory
> > $ echo $PWD
> > /home/newren/floss/git/gone
> >
> > If I do not run `pwd -P` first then a plain `pwd` works.  But your
> > advice to use `pwd` seems misguided for programs, since they'll hit
> > this problem if users have run a previous `pwd -P`.  (The $PWD trick
> > would have worked, as shown above)
> >
> >> I wonder if it was just because you ran into our bin-wrappers edge case,
> >> but that should be really obscure for any real users.
>
> For some reason I was under the misimpression that the "#!/bin/bash"
> part of the bin-wrappers and other scripts somehow immunized them from
> the $PWD/"pwd" reset, and it was only the programs they invoked (like
> git in the bin-wrappers) that didn't get the values passed along.
>
> But that's clearly incorrect as you demonstrate above, so the only thing
> that'll work OK (seemingly) is running "pwd" (but not "pwd -P"), or
> looking at $PWD in your terminal shell itself.

And further, even running a plain "pwd" seems to not work either if
something else has already run "pwd -P" already, as I pointed out
above.  I have no clue how one 'pwd -P' call can prevent future plain
'pwd' calls from working, but that's the behavior I observed.

> Invoking non-POSIX shell programs "works" in that they can use the same
> trick, after the dance of "mkdir x && cd x && rm -rf ..x" e.g. Perl
> says:
>
>     $ perl -MCwd=getcwd -wE 'say $ENV{PWD}; say getcwd'
>     /home/avar/g/git/x
>     Use of uninitialized value in say at -e line 1
>
> This "pwd -P" behavior isn't just something weird in your shell & mine,
> it semse to be mandated by POSIX:
> https://pubs.opengroup.org/onlinepubs/007904875/utilities/pwd.html
>
> All of which is to say I'm much more sympathetic to this approach
> now. I.e. it seemed like purely a way to work around fixable breakages
> in our own and other programs. Well, I guess "don't use shellscript if
> you care about this edge case" is a "fix", but not a trivial one.
>
> I do still think a better starting point would be fixing the setup.c
> dying in our own code, and see where that leaves us, but up to you
> etc. I'm not going to work on it any time soon.
>
> > Yes, I agree our bin-wrappers is a really obscure edge case not worth
> > considering.  I don't see how you jump from there to painting all
> > shell scripts combined with the same brush, though.
>
> *nod*
>
> >> > This confuses end users, particularly since the command they get the
> >> > error from is not the one that caused the problem; the problem came from
> >> > the side-effect of some previous command.
> >> >
> >> > We would like to avoid removing the current working directory of our
> >> > parent process; towards this end, introduce a new variable,
> >> > startup_info->original_cwd, that tracks the current working directory
> >> > that we inherited from our parent process.  For convenience of later
> >> > comparisons, we prefer that this new variable store a path relative to
> >> > the toplevel working directory (thus much like 'prefix'), except without
> >> > the trailing slash.
> >>
> >> I'm still not clear at all on why we need a "original_cwd" at all then
> >> as opposed to just using "prefix" (or adding "the_prefix" if passing it
> >> down is painful). I.e. we discover our relative path, we resolve the
> >> relative path to the root, can't we use that as the "don't remove our
> >> CWD" guard?
> >>
> >> Does our prefix change at some point, then maybe "orig_prefix" would
> >> make more sense?
> >
> > No; see the code comment in the same patch:
> >
> > +       /*
> > +        * startup_info->original_cwd points to the current working
> > +        * directory we inherited from our parent process, which is a
> > +        * directory we want to avoid removing.
> > +        *
> > +        * For convience, we would like to have the path relative to the
> > +        * worktree instead of an absolute path.
> > +        *
> > +        * Yes, startup_info->original_cwd is usually the same as 'prefix',
> > +        * but differs in two ways:
> > +        *   - prefix has a trailing '/'
> > +        *   - if the user passes '-C' to git, that modifies the prefix but
> > +        *     not startup_info->original_cwd.
> > +        */
> >
> > It's never equal to prefix, even though it's usually semantically
> > referring to the same directory.  However, even if it weren't for the
> > trailing slash issue, the -C case means it is not appropriate to think
> > of it as "orig_prefix" either.
>
> Ah, with -C of e.g. t/helper we'll first chdir(t/helper), and then run
> the usual setup.c dance to find that we need to chdir() again to the
> (equivalent of) "../../". But our prefix stays at "t/helper".
>
> I'm a bit confused about the trailing slash case, isn't the prefix
> always going to point to a directory? Why would t/helper v.s. t/helper/
> matter?

Because when you compare two paths, you either need to canonicalize
them in some fashion or do a bunch of tricky work to see if they are
"equal".

> I think it won't matter for rmdir(2) et al, but maybe I'm wrong.

Correct, rmdir won't care, but strcmp() certainly does.  Since we call
strcmp() before deciding whether to remove, and we do that based on
paths built up from prefix + tree traversal + entry name (which won't
have the trailing '/') we want original_cmp to be canonicalized the
same way (i.e. without the trailing '/').

(In contrast, prefix is used as a starting point to build up other
paths, so having a trailing slash was more convenient for it.)

> What got me confused about the "prefix" v.s. "original_cwd" is that I
> was assuming they'd be the same. The commentary on setup_git_directory()
> says as much, i.e.:
>
>     Returns the "prefix", a path to the current working directory
>     relative to the work tree root, or NULL,
>
> But of course we know that's a white lie, it's not the $PWD/getcwd().

I don't think it is a white lie.  This code runs after the chdir()
from the parsing of the global -C option, and before the chdir() to
the worktree root (which occurs later in setup.c).  So the statement
is true, you just have to understand what "current" means in "the
current working directory".  In this context, "current" means *as of
the time the setup.c code where this comment is written starts
running*, which is a pretty logical thing for current to mean.  If you
try reading this code unaware of the other chdir() calls we make,
though, it certainly has the potential to trip you up.

> So you're only trying to save the user in cases of e.g. (in t/helper):
>
>     git rm ../helper
>
> If they actually run it while in t/helper, but would like to explicitly
> omit the case of (at the top-level):
>
>     git -C t/helper rm ../helper
>
> That's fair enough I guess. I'd just assumed those cases would be
> treated the same way.

I don't see why you'd assume that.  The whole point was preserving the
cwd of our parent process so that any subsequent commands that happen
to run in that parent process don't get confused by having their cwd
no longer exist.  In the former command you gave, the cwd of the
parent process was $WORKTREE/t/helper; in the latter, it was
$WORKTREE.  So, they should preserve different things.

> Even in that case, I can't think of a case where this "original_cwd"
> wouldn't be made redundant by some boolean flag to accompany "prefix" to
> indicate that we've chdir'd out of the prefix (or not).

I can't think of a case where a boolean would provide any useful
information.  Don't we always chdir out of the prefix whenever prefix
!= NULL?

Maybe I'm missing something, so let me give you a simple case where
you can explain how this boolean could help.  Let's say you started at
the worktree root and then ran these two commands:

   $ cd subdir/
   $ git -C subsubdir rm lastfile

After finishing the setup phase of the git command, the state should
be the following:
   prefix = subdir/subsubdir/
   original_cwd = subdir
   getcwd = subdir/../  (because setup.c chdirs to the worktree root;
the actual answer would be a more canonicalized path, though)

In this above case, how can we infer the value of original_cwd from
other data if we don't record it?

[PATCH v4 00/11] Avoid removing the current working directory, even if it becomes empty

[Elijah Newren via GitGitGadget]

Traditionally, if folks run git commands such as checkout or rebase from a
subdirectory, that git command could remove their current working directory
and result in subsequent git and non-git commands either getting confused or
printing messages that confuse the user (e.g. "fatal: Unable to read current
working directory: No such file or directory"). Many commands either
silently avoid removing directories that are not empty (i.e. those that have
untracked or modified files in them)[1], or show an error and abort,
depending on which is more appropriate for the command in question. With
this series, we augment the reasons to avoid removing directories to include
not just has-untracked-or-modified-files, but also to avoid removing the
original_cwd as well.

Peff and Junio provided some good pros/cons, if it helps:

 * Pros: Peff (original suggester of the idea)[2], and Junio[3]
 * Cons: Peff [2, again -- see the "P.S."], and Junio[4]

[1] well, with a few exceptions; see
https://lore.kernel.org/git/pull.1036.v3.git.1632760428.gitgitgadget@gmail.com/
[2] https://lore.kernel.org/git/YS8eEtwQvF7TaLCb@coredump.intra.peff.net/
[3] https://lore.kernel.org/git/xmqqo86elyht.fsf@gitster.g/ [4]
https://lore.kernel.org/git/xmqqo8691gr8.fsf@gitster.g/

Changes since v3:

 * fixed one codepath from v2 so that the series really is only about the
   working tree
 * used test-tool getcwd instead of pwd -P as suggested by Ævar for some
   less common platforms
 * fixed bashism
 * check for clean index/worktree after verifying that expected-to-abort
   codepaths do abort, to make it clearer that we expect an early abort
 * remove a leftover (and confusing) is_absolute_dir() check in sequencer
   and stash from an earlier round of the series

Changes since v2:

 * the series is now only about the working tree. So if the original cwd is
   outside the worktree (or we're in a bare repo), then the new code is a
   no-op.
 * fixed ugly early die() possibility (uses strbuf_getcwd() instead of
   xgetcwd())
 * modified the initial tests to show both expected and desired behavior.
   subsequent patches fix the tests. One new patch added at the end which
   simplifies the tests to only check for desired behavior.
 * NULLify startup_info->original_cwd when it matches the toplevel worktree;
   that is already protected and we don't need secondary protection for it.
   This simplified some other codepaths so we don't have to check for
   startup_info->original_cwd == "".
 * clarified some commit messages

Changes since v1:

 * clarified multiple commit messages
 * renamed the_cwd to startup_info->original_cwd to make it clearer that
   it's our parent process'es cwd that really matters, which we inherited at
   program startup. Also pulls it out of the global namespace.
 * Normalize the path for startup_info->original_cwd, and ensure that it's
   actually the original cwd even if -C is passed to git.
 * small code cleanups suggested by René and Ævar
 * split the final patch (which got the most comments) into two -- one for
   each function being modified. Also, add a bunch more history to the first
   of the two resulting commit messages
 * no longer has a content conflict with so/stash-staged
 * add another value for the flags parameter that remove_dir_recursively()
   takes so that it can opt into either the old or the new behavior. Use
   that for the one special corner case I could find where it matters, and
   add a few tests around it to highlight the utility of the flag.

Elijah Newren (11):
  t2501: add various tests for removing the current working directory
  setup: introduce startup_info->original_cwd
  unpack-trees: refuse to remove startup_info->original_cwd
  unpack-trees: add special cwd handling
  symlinks: do not include startup_info->original_cwd in dir removal
  clean: do not attempt to remove startup_info->original_cwd
  rebase: do not attempt to remove startup_info->original_cwd
  stash: do not attempt to remove startup_info->original_cwd
  dir: avoid incidentally removing the original_cwd in remove_path()
  dir: new flag to remove_dir_recurse() to spare the original_cwd
  t2501: simplify the tests since we can now assume desired behavior

 builtin/clean.c      |  44 +++++--
 builtin/rm.c         |   3 +-
 builtin/stash.c      |   4 +-
 cache.h              |   2 +
 common-main.c        |   4 +
 dir.c                |  15 ++-
 dir.h                |   9 +-
 sequencer.c          |   2 +
 setup.c              |  65 ++++++++++
 symlinks.c           |   8 +-
 t/t2501-cwd-empty.sh | 277 +++++++++++++++++++++++++++++++++++++++++++
 unpack-trees.c       |  30 ++++-
 unpack-trees.h       |   1 +
 13 files changed, 442 insertions(+), 22 deletions(-)
 create mode 100755 t/t2501-cwd-empty.sh


base-commit: 88d915a634b449147855041d44875322de2b286d
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1140%2Fnewren%2Fcwd_removal-v4
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1140/newren/cwd_removal-v4
Pull-Request: https://github.com/git/git/pull/1140

Range-diff vs v3:

  1:  4b0044656b0 !  1:  a45b3f08802 t2501: add various tests for removing the current working directory
     @@ t/t2501-cwd-empty.sh (new)
      +
      +		# Although we want pwd & git status to pass, test for existing
      +		# rather than desired behavior.
     -+		if [[ $works == "success" ]]; then
     -+			pwd -P &&
     ++		if test "$works" == "success"
     ++		then
     ++			test-tool getcwd &&
      +			git status --porcelain
      +		else
     -+			! pwd -P &&
     ++			! test-tool getcwd &&
      +			test_might_fail git status --porcelain
      +		fi
      +	) &&
     @@ t/t2501-cwd-empty.sh (new)
      +
      +	# Although we want dir to be present, test for existing rather
      +	# than desired behavior.
     -+	if [[ $works == "success" ]]; then
     ++	if test "$works" == "success"
     ++	then
      +		test_path_is_dir foo
      +	else
      +		test_path_is_missing foo
     @@ t/t2501-cwd-empty.sh (new)
      +		cd dirORfile &&
      +
      +		# We'd like for the command to fail (much as it would if there
     -+		# was an untracked file there), and for pwd & git status to
     -+		# succeed afterwards.  But test for existing rather than
     -+		# desired behavior.
     -+		if [[ $works == "success" ]]; then
     ++		# was an untracked file there), and for the index and worktree
     ++		# to be left clean with pwd and git status working afterwards.
     ++		# But test for existing rather than desired behavior.
     ++		if test "$works" == "success"
     ++		then
      +			test_must_fail "$@" 2>../error &&
      +			grep "Refusing to remove.*current working directory" ../error &&
     -+			pwd -P &&
     ++
     ++			git diff --exit-code HEAD &&
     ++
     ++			test-tool getcwd &&
      +			git status --porcelain
      +		else
      +			"$@" &&
     -+			! pwd -P &&
     ++			! test-tool getcwd &&
      +			test_might_fail git status --porcelain
      +		fi
      +	) &&
      +
      +	# Although we want dirORfile to be present, test for existing rather
      +	# than desired behavior.
     -+	if [[ $works == "success" ]]; then
     ++	if test "$works" == "success"
     ++	then
      +		test_path_is_dir dirORfile
      +	else
      +		test_path_is_file dirORfile
     @@ t/t2501-cwd-empty.sh (new)
      +
      +		# Although we want pwd & git status to pass, test for existing
      +		# rather than desired behavior.
     -+		if [[ $works == "success" ]]; then
     -+			pwd -P &&
     ++		if test "$works" == "success"
     ++		then
     ++			test-tool getcwd &&
      +			git status --porcelain
      +		else
     -+			! pwd -P &&
     ++			! test-tool getcwd &&
      +			test_might_fail git status --porcelain
      +		fi
      +	) &&
     @@ t/t2501-cwd-empty.sh (new)
      +
      +	# Although we want dir to be present, test for existing rather
      +	# than desired behavior.
     -+	if [[ $works == "success" ]]; then
     ++	if test "$works" == "success"
     ++	then
      +		test_path_is_dir untracked
      +	else
      +		test_path_is_missing untracked
     @@ t/t2501-cwd-empty.sh (new)
      +	shift &&
      +
      +	test_status=
     -+	test $path_status = dir && test_status=test_must_fail
     ++	test "$path_status" = dir && test_status=test_must_fail
      +
      +	# Actually, while path_status == dir && test_status=test_must_fail
      +	# reflect our desired behavior, current behavior is:
  2:  200ddece05d !  2:  ca9f632bd11 setup: introduce startup_info->original_cwd
     @@ setup.c: void setup_work_tree(void)
      +		startup_info->original_cwd = \
      +			precompose_string_if_needed(startup_info->original_cwd
      +						    + offset);
     ++		return;
      +	}
     -+	return;
      +
      +no_prevention_needed:
      +	free((char*)startup_info->original_cwd);
  3:  68ae90546fe =  3:  41a82eff41e unpack-trees: refuse to remove startup_info->original_cwd
  4:  1bb8905900c =  4:  2e2ea02f97b unpack-trees: add special cwd handling
  5:  8a33d74e7cf =  5:  f444a541da4 symlinks: do not include startup_info->original_cwd in dir removal
  6:  11e4ec881bb =  6:  1990e36bb41 clean: do not attempt to remove startup_info->original_cwd
  7:  39b1f3a225e !  7:  1035ee7f9ce rebase: do not attempt to remove startup_info->original_cwd
     @@ sequencer.c: static int run_git_checkout(struct repository *r, struct replay_opt
       
       	cmd.git_cmd = 1;
       
     -+	if (startup_info->original_cwd &&
     -+	    !is_absolute_path(startup_info->original_cwd))
     ++	if (startup_info->original_cwd)
      +		cmd.dir = startup_info->original_cwd;
       	strvec_push(&cmd.args, "checkout");
       	strvec_push(&cmd.args, commit);
  8:  0110462a19c !  8:  a2be40a22d1 stash: do not attempt to remove startup_info->original_cwd
     @@ builtin/stash.c: static int do_push_stash(const struct pathspec *ps, const char
       			struct child_process cp = CHILD_PROCESS_INIT;
       
       			cp.git_cmd = 1;
     -+			if (startup_info->original_cwd &&
     -+			    !is_absolute_path(startup_info->original_cwd))
     ++			if (startup_info->original_cwd)
      +				cp.dir = startup_info->original_cwd;
       			strvec_pushl(&cp.args, "clean", "--force",
      -				     "--quiet", "-d", NULL);
  9:  2c73a09a2e8 =  9:  834031be9e0 dir: avoid incidentally removing the original_cwd in remove_path()
 10:  d4e50b4053d ! 10:  d5750fcb6d5 dir: new flag to remove_dir_recurse() to spare the original_cwd
     @@ dir.h: int get_sparse_checkout_patterns(struct pattern_list *pl);
       ## t/t2501-cwd-empty.sh ##
      @@ t/t2501-cwd-empty.sh: test_submodule_removal () {
       	test_status=
     - 	test $path_status = dir && test_status=test_must_fail
     + 	test "$path_status" = dir && test_status=test_must_fail
       
      -	# Actually, while path_status == dir && test_status=test_must_fail
      -	# reflect our desired behavior, current behavior is:
 11:  7eb6281be4b ! 11:  21ff99a767c t2501: simplify the tests since we can now assume desired behavior
     @@ t/t2501-cwd-empty.sh: test_incidental_dir_removal () {
       
      -		# Although we want pwd & git status to pass, test for existing
      -		# rather than desired behavior.
     --		if [[ $works == "success" ]]; then
     --			pwd -P &&
     +-		if test "$works" == "success"
     +-		then
     +-			test-tool getcwd &&
      -			git status --porcelain
      -		else
     --			! pwd -P &&
     +-			! test-tool getcwd &&
      -			test_might_fail git status --porcelain
      -		fi
     -+		pwd -P &&
     ++		# Make sure foo still exists, and commands needing it work
     ++		test-tool getcwd &&
      +		git status --porcelain
       	) &&
       	test_path_is_missing foo/bar/baz &&
     @@ t/t2501-cwd-empty.sh: test_incidental_dir_removal () {
       
      -	# Although we want dir to be present, test for existing rather
      -	# than desired behavior.
     --	if [[ $works == "success" ]]; then
     +-	if test "$works" == "success"
     +-	then
      -		test_path_is_dir foo
      -	else
      -		test_path_is_missing foo
     @@ t/t2501-cwd-empty.sh: test_incidental_dir_removal () {
       		cd dirORfile &&
       
      -		# We'd like for the command to fail (much as it would if there
     --		# was an untracked file there), and for pwd & git status to
     --		# succeed afterwards.  But test for existing rather than
     --		# desired behavior.
     --		if [[ $works == "success" ]]; then
     +-		# was an untracked file there), and for the index and worktree
     +-		# to be left clean with pwd and git status working afterwards.
     +-		# But test for existing rather than desired behavior.
     +-		if test "$works" == "success"
     +-		then
      -			test_must_fail "$@" 2>../error &&
      -			grep "Refusing to remove.*current working directory" ../error &&
     --			pwd -P &&
     +-
     +-			git diff --exit-code HEAD &&
     +-
     +-			test-tool getcwd &&
      -			git status --porcelain
      -		else
      -			"$@" &&
     --			! pwd -P &&
     +-			! test-tool getcwd &&
      -			test_might_fail git status --porcelain
      -		fi
     ++		# Ensure command refuses to run
      +		test_must_fail "$@" 2>../error &&
      +		grep "Refusing to remove.*current working directory" ../error &&
     -+		pwd -P &&
     ++
     ++		# ...and that the index and working tree are left clean
     ++		git diff --exit-code HEAD &&
     ++
     ++		# Ensure that getcwd and git status do not error out (which
     ++		# they might if the current working directory had been removed)
     ++		test-tool getcwd &&
      +		git status --porcelain
       	) &&
       
      -	# Although we want dirORfile to be present, test for existing rather
      -	# than desired behavior.
     --	if [[ $works == "success" ]]; then
     +-	if test "$works" == "success"
     +-	then
      -		test_path_is_dir dirORfile
      -	else
      -		test_path_is_file dirORfile
     @@ t/t2501-cwd-empty.sh: test_incidental_untracked_dir_removal () {
       
      -		# Although we want pwd & git status to pass, test for existing
      -		# rather than desired behavior.
     --		if [[ $works == "success" ]]; then
     --			pwd -P &&
     +-		if test "$works" == "success"
     +-		then
     +-			test-tool getcwd &&
      -			git status --porcelain
      -		else
     --			! pwd -P &&
     +-			! test-tool getcwd &&
      -			test_might_fail git status --porcelain
      -		fi
     -+		pwd -P &&
     ++		# Make sure untracked still exists, and commands needing it work
     ++		test-tool getcwd &&
      +		git status --porcelain
       	) &&
       	test_path_is_missing empty &&
     @@ t/t2501-cwd-empty.sh: test_incidental_untracked_dir_removal () {
       
      -	# Although we want dir to be present, test for existing rather
      -	# than desired behavior.
     --	if [[ $works == "success" ]]; then
     +-	if test "$works" == "success"
     +-	then
      -		test_path_is_dir untracked
      -	else
      -		test_path_is_missing untracked

-- 
gitgitgadget

[PATCH v4 01/11] t2501: add various tests for removing the current working directory

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Numerous commands will remove directories left empty as a "convenience"
after removing files within them.  That is normally fine, but removing
the current working directory can be rather inconvenient since it can
cause confusion for the user when they run subsequent commands.  For
example, after one git process has removed the current working
directory, git status/log/diff will all abort with the message:

    fatal: Unable to read current working directory: No such file or directory

We also have code paths that, when a file needs to be placed where a
directory is (due to e.g. checkout, merge, reset, whatever), will check
if this is okay and error out if not.  These rules include:
  * all tracked files under that directory are intended to be removed by
    the operation
  * none of the tracked files under that directory have uncommitted
    modification
  * there are no untracked files under that directory
However, if we end up remove the current working directory, we can cause
user confusion when they run subsequent commands, so we would prefer if
there was a fourth rule added to this list: avoid removing the current
working directory.

Since there are several code paths that can result in the current
working directory being removed, add several tests of various different
codepaths.  To make it clearer what the difference between the current
behavior and the behavior at the end of the series, code both of them
into the tests and have the appropriate behavior be selected by a flag.
Subsequent commits will toggle the flag from current to desired
behavior.

Also add a few tests suggested during the review of earlier rounds of
this patch series.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh | 342 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 342 insertions(+)
 create mode 100755 t/t2501-cwd-empty.sh

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
new file mode 100755
index 00000000000..fd83fe921d5
--- /dev/null
+++ b/t/t2501-cwd-empty.sh
@@ -0,0 +1,342 @@
+#!/bin/sh
+
+test_description='Test handling of the current working directory becoming empty'
+
+. ./test-lib.sh
+
+test_expect_success setup '
+	test_commit init &&
+
+	git branch fd_conflict &&
+
+	mkdir -p foo/bar &&
+	test_commit foo/bar/baz &&
+
+	git revert HEAD &&
+	git tag reverted &&
+
+	git checkout fd_conflict &&
+	mkdir dirORfile &&
+	test_commit dirORfile/foo &&
+
+	git rm -r dirORfile &&
+	echo not-a-directory >dirORfile &&
+	git add dirORfile &&
+	git commit -m dirORfile &&
+
+	git switch -c df_conflict HEAD~1 &&
+	test_commit random_file &&
+
+	git switch -c undo_fd_conflict fd_conflict &&
+	git revert HEAD
+'
+
+test_incidental_dir_removal () {
+	works=$1 &&
+	shift &&
+
+	test_when_finished "git reset --hard" &&
+
+	git checkout foo/bar/baz^{commit} &&
+	test_path_is_dir foo/bar &&
+
+	(
+		cd foo &&
+		"$@" &&
+
+		# Although we want pwd & git status to pass, test for existing
+		# rather than desired behavior.
+		if test "$works" == "success"
+		then
+			test-tool getcwd &&
+			git status --porcelain
+		else
+			! test-tool getcwd &&
+			test_might_fail git status --porcelain
+		fi
+	) &&
+	test_path_is_missing foo/bar/baz &&
+	test_path_is_missing foo/bar &&
+
+	# Although we want dir to be present, test for existing rather
+	# than desired behavior.
+	if test "$works" == "success"
+	then
+		test_path_is_dir foo
+	else
+		test_path_is_missing foo
+	fi
+}
+
+test_required_dir_removal () {
+	works=$1 &&
+	shift &&
+
+	git checkout df_conflict^{commit} &&
+	test_when_finished "git clean -fdx" &&
+
+	(
+		cd dirORfile &&
+
+		# We'd like for the command to fail (much as it would if there
+		# was an untracked file there), and for the index and worktree
+		# to be left clean with pwd and git status working afterwards.
+		# But test for existing rather than desired behavior.
+		if test "$works" == "success"
+		then
+			test_must_fail "$@" 2>../error &&
+			grep "Refusing to remove.*current working directory" ../error &&
+
+			git diff --exit-code HEAD &&
+
+			test-tool getcwd &&
+			git status --porcelain
+		else
+			"$@" &&
+			! test-tool getcwd &&
+			test_might_fail git status --porcelain
+		fi
+	) &&
+
+	# Although we want dirORfile to be present, test for existing rather
+	# than desired behavior.
+	if test "$works" == "success"
+	then
+		test_path_is_dir dirORfile
+	else
+		test_path_is_file dirORfile
+	fi
+}
+
+test_expect_success 'checkout does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git checkout init
+'
+
+test_expect_success 'checkout fails if cwd needs to be removed' '
+	test_required_dir_removal failure git checkout fd_conflict
+'
+
+test_expect_success 'reset --hard does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git reset --hard init
+'
+
+test_expect_success 'reset --hard fails if cwd needs to be removed' '
+	test_required_dir_removal failure git reset --hard fd_conflict
+'
+
+test_expect_success 'merge does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git merge reverted
+'
+
+# This file uses some simple merges where
+#   Base: 'dirORfile/' exists
+#   Side1: random other file changed
+#   Side2: 'dirORfile/' removed, 'dirORfile' added
+# this should resolve cleanly, but merge-recursive throws merge conflicts
+# because it's dumb.  Add a special test for checking merge-recursive (and
+# merge-ort), then after this just hard require ort for all remaining tests.
+#
+test_expect_success 'merge fails if cwd needs to be removed; recursive friendly' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir dirORfile &&
+	(
+		cd dirORfile &&
+
+		# We would rather this failed, but we test for existing
+		# rather than desired behavior
+		git merge fd_conflict 2>../error
+	) &&
+
+	## Here is the behavior we would rather have:
+	#test_path_is_dir dirORfile &&
+	#grep "Refusing to remove the current working directory" error
+	## But instead we test for existing behavior
+	test_path_is_file dirORfile &&
+	test_must_be_empty error
+'
+
+GIT_TEST_MERGE_ALGORITHM=ort
+
+test_expect_success 'merge fails if cwd needs to be removed' '
+	test_required_dir_removal failure git merge fd_conflict
+'
+
+test_expect_success 'cherry-pick does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git cherry-pick reverted
+'
+
+test_expect_success 'cherry-pick fails if cwd needs to be removed' '
+	test_required_dir_removal failure git cherry-pick fd_conflict
+'
+
+test_expect_success 'rebase does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git rebase reverted
+'
+
+test_expect_success 'rebase fails if cwd needs to be removed' '
+	test_required_dir_removal failure git rebase fd_conflict
+'
+
+test_expect_success 'revert does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git revert HEAD
+'
+
+test_expect_success 'revert fails if cwd needs to be removed' '
+	test_required_dir_removal failure git revert undo_fd_conflict
+'
+
+test_expect_success 'rm does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git rm bar/baz.t
+'
+
+test_expect_success 'apply does not remove cwd incidentally' '
+	git diff HEAD HEAD~1 >patch &&
+	test_incidental_dir_removal failure git apply ../patch
+'
+
+test_incidental_untracked_dir_removal () {
+	works=$1 &&
+	shift &&
+
+	test_when_finished "git reset --hard" &&
+
+	git checkout foo/bar/baz^{commit} &&
+	mkdir -p untracked &&
+	mkdir empty
+	>untracked/random &&
+
+	(
+		cd untracked &&
+		"$@" &&
+
+		# Although we want pwd & git status to pass, test for existing
+		# rather than desired behavior.
+		if test "$works" == "success"
+		then
+			test-tool getcwd &&
+			git status --porcelain
+		else
+			! test-tool getcwd &&
+			test_might_fail git status --porcelain
+		fi
+	) &&
+	test_path_is_missing empty &&
+	test_path_is_missing untracked/random &&
+
+	# Although we want dir to be present, test for existing rather
+	# than desired behavior.
+	if test "$works" == "success"
+	then
+		test_path_is_dir untracked
+	else
+		test_path_is_missing untracked
+	fi
+}
+
+test_expect_success 'clean does not remove cwd incidentally' '
+	test_incidental_untracked_dir_removal failure \
+		git -C .. clean -fd -e warnings . >warnings
+'
+
+test_expect_success 'stash does not remove cwd incidentally' '
+	test_incidental_untracked_dir_removal failure \
+		git stash --include-untracked
+'
+
+test_expect_success '`rm -rf dir` only removes a subset of dir' '
+	test_when_finished "rm -rf a/" &&
+
+	mkdir -p a/b/c &&
+	>a/b/c/untracked &&
+	>a/b/c/tracked &&
+	git add a/b/c/tracked &&
+
+	(
+		cd a/b &&
+		git rm -rf ../b
+	) &&
+
+	test_path_is_dir a/b &&
+	test_path_is_missing a/b/c/tracked &&
+	test_path_is_file a/b/c/untracked
+'
+
+test_expect_success '`rm -rf dir` even with only tracked files will remove something else' '
+	test_when_finished "rm -rf a/" &&
+
+	mkdir -p a/b/c &&
+	>a/b/c/tracked &&
+	git add a/b/c/tracked &&
+
+	(
+		cd a/b &&
+		git rm -rf ../b
+	) &&
+
+	test_path_is_missing a/b/c/tracked &&
+	## We would prefer if a/b was still present, though empty, since it
+	## was the current working directory
+	#test_path_is_dir a/b
+	## But the current behavior is that it not only deletes the directory
+	## a/b as requested, but also goes and deletes a
+	test_path_is_missing a
+'
+
+test_expect_success 'git version continues working from a deleted dir' '
+	mkdir tmp &&
+	(
+		cd tmp &&
+		rm -rf ../tmp &&
+		git version
+	)
+'
+
+test_submodule_removal () {
+	path_status=$1 &&
+	shift &&
+
+	test_status=
+	test "$path_status" = dir && test_status=test_must_fail
+
+	# Actually, while path_status == dir && test_status=test_must_fail
+	# reflect our desired behavior, current behavior is:
+	path_status=missing
+	test_status=
+
+	test_when_finished "git reset --hard HEAD~1" &&
+	test_when_finished "rm -rf .git/modules/my_submodule" &&
+
+	git checkout foo/bar/baz &&
+
+	git init my_submodule &&
+	touch my_submodule/file &&
+	git -C my_submodule add file &&
+	git -C my_submodule commit -m "initial commit" &&
+	git submodule add ./my_submodule &&
+	git commit -m "Add the submodule" &&
+
+	(
+		cd my_submodule &&
+		$test_status "$@"
+	) &&
+
+	test_path_is_${path_status} my_submodule
+}
+
+test_expect_success 'rm -r with -C leaves submodule if cwd inside' '
+	test_submodule_removal dir git -C .. rm -r my_submodule/
+'
+
+test_expect_success 'rm -r leaves submodule if cwd inside' '
+	test_submodule_removal dir \
+		git --git-dir=../.git --work-tree=.. rm -r ../my_submodule/
+'
+
+test_expect_success 'rm -rf removes submodule even if cwd inside' '
+	test_submodule_removal missing \
+		git --git-dir=../.git --work-tree=.. rm -rf ../my_submodule/
+'
+
+test_done
-- 
gitgitgadget

[PATCH v4 02/11] setup: introduce startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Removing the current working directory causes all subsequent git
commands run from that directory to get confused and fail with a message
about being unable to read the current working directory:

    $ git status
    fatal: Unable to read current working directory: No such file or directory

Non-git commands likely have similar warnings or even errors, e.g.

    $ bash -c 'echo hello'
    shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    hello

This confuses end users, particularly since the command they get the
error from is not the one that caused the problem; the problem came from
the side-effect of some previous command.

We would like to avoid removing the current working directory of our
parent process; towards this end, introduce a new variable,
startup_info->original_cwd, that tracks the current working directory
that we inherited from our parent process.  For convenience of later
comparisons, we prefer that this new variable store a path relative to
the toplevel working directory (thus much like 'prefix'), except without
the trailing slash.

Subsequent commits will make use of this new variable.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 cache.h       |  2 ++
 common-main.c |  4 ++++
 setup.c       | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 71 insertions(+)

diff --git a/cache.h b/cache.h
index eba12487b99..92e181ea759 100644
--- a/cache.h
+++ b/cache.h
@@ -1834,8 +1834,10 @@ void overlay_tree_on_index(struct index_state *istate,
 struct startup_info {
 	int have_repository;
 	const char *prefix;
+	const char *original_cwd;
 };
 extern struct startup_info *startup_info;
+extern const char *tmp_original_cwd;
 
 /* merge.c */
 struct commit_list;
diff --git a/common-main.c b/common-main.c
index 71e21dd20a3..aa8d5aba5bb 100644
--- a/common-main.c
+++ b/common-main.c
@@ -26,6 +26,7 @@ static void restore_sigpipe_to_default(void)
 int main(int argc, const char **argv)
 {
 	int result;
+	struct strbuf tmp = STRBUF_INIT;
 
 	trace2_initialize_clock();
 
@@ -49,6 +50,9 @@ int main(int argc, const char **argv)
 	trace2_cmd_start(argv);
 	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
 
+	if (!strbuf_getcwd(&tmp))
+		tmp_original_cwd = strbuf_detach(&tmp, NULL);
+
 	result = cmd_main(argc, argv);
 
 	trace2_cmd_exit(result);
diff --git a/setup.c b/setup.c
index 347d7181ae9..af3b8c09abe 100644
--- a/setup.c
+++ b/setup.c
@@ -12,6 +12,7 @@ static int work_tree_config_is_bogus;
 
 static struct startup_info the_startup_info;
 struct startup_info *startup_info = &the_startup_info;
+const char *tmp_original_cwd;
 
 /*
  * The input parameter must contain an absolute path, and it must already be
@@ -432,6 +433,69 @@ void setup_work_tree(void)
 	initialized = 1;
 }
 
+static void setup_original_cwd(void)
+{
+	struct strbuf tmp = STRBUF_INIT;
+	const char *worktree = NULL;
+	int offset = -1;
+
+	if (!tmp_original_cwd)
+		return;
+
+	/*
+	 * startup_info->original_cwd points to the current working
+	 * directory we inherited from our parent process, which is a
+	 * directory we want to avoid removing.
+	 *
+	 * For convience, we would like to have the path relative to the
+	 * worktree instead of an absolute path.
+	 *
+	 * Yes, startup_info->original_cwd is usually the same as 'prefix',
+	 * but differs in two ways:
+	 *   - prefix has a trailing '/'
+	 *   - if the user passes '-C' to git, that modifies the prefix but
+	 *     not startup_info->original_cwd.
+	 */
+
+	/* Normalize the directory */
+	strbuf_realpath(&tmp, tmp_original_cwd, 1);
+	free((char*)tmp_original_cwd);
+	tmp_original_cwd = NULL;
+	startup_info->original_cwd = strbuf_detach(&tmp, NULL);
+
+	/*
+	 * Get our worktree; we only protect the current working directory
+	 * if it's in the worktree.
+	 */
+	worktree = get_git_work_tree();
+	if (!worktree)
+		goto no_prevention_needed;
+
+	offset = dir_inside_of(startup_info->original_cwd, worktree);
+	if (offset >= 0) {
+		/*
+		 * If startup_info->original_cwd == worktree, that is already
+		 * protected and we don't need original_cwd as a secondary
+		 * protection measure.
+		 */
+		if (!*(startup_info->original_cwd + offset))
+			goto no_prevention_needed;
+
+		/*
+		 * original_cwd was inside worktree; precompose it just as
+		 * we do prefix so that built up paths will match
+		 */
+		startup_info->original_cwd = \
+			precompose_string_if_needed(startup_info->original_cwd
+						    + offset);
+		return;
+	}
+
+no_prevention_needed:
+	free((char*)startup_info->original_cwd);
+	startup_info->original_cwd = NULL;
+}
+
 static int read_worktree_config(const char *var, const char *value, void *vdata)
 {
 	struct repository_format *data = vdata;
@@ -1330,6 +1394,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
 		setenv(GIT_PREFIX_ENVIRONMENT, "", 1);
 	}
 
+	setup_original_cwd();
 
 	strbuf_release(&dir);
 	strbuf_release(&gitdir);
-- 
gitgitgadget

[PATCH v4 03/11] unpack-trees: refuse to remove startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

In the past, when a directory needs to be removed to make room for a
file, we have always errored out when that directory contains any
untracked (but not ignored) files.  Add an extra condition on that: also
error out if the directory is the current working directory we inherited
from our parent process.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh | 20 +++++++-------------
 unpack-trees.c       | 17 +++++++++++++----
 unpack-trees.h       |  1 +
 3 files changed, 21 insertions(+), 17 deletions(-)

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index fd83fe921d5..55081f5c980 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -113,7 +113,7 @@ test_expect_success 'checkout does not clean cwd incidentally' '
 '
 
 test_expect_success 'checkout fails if cwd needs to be removed' '
-	test_required_dir_removal failure git checkout fd_conflict
+	test_required_dir_removal success git checkout fd_conflict
 '
 
 test_expect_success 'reset --hard does not clean cwd incidentally' '
@@ -144,23 +144,17 @@ test_expect_success 'merge fails if cwd needs to be removed; recursive friendly'
 	(
 		cd dirORfile &&
 
-		# We would rather this failed, but we test for existing
-		# rather than desired behavior
-		git merge fd_conflict 2>../error
+		test_must_fail git merge fd_conflict 2>../error
 	) &&
 
-	## Here is the behavior we would rather have:
-	#test_path_is_dir dirORfile &&
-	#grep "Refusing to remove the current working directory" error
-	## But instead we test for existing behavior
-	test_path_is_file dirORfile &&
-	test_must_be_empty error
+	test_path_is_dir dirORfile &&
+	grep "Refusing to remove the current working directory" error
 '
 
 GIT_TEST_MERGE_ALGORITHM=ort
 
 test_expect_success 'merge fails if cwd needs to be removed' '
-	test_required_dir_removal failure git merge fd_conflict
+	test_required_dir_removal success git merge fd_conflict
 '
 
 test_expect_success 'cherry-pick does not clean cwd incidentally' '
@@ -168,7 +162,7 @@ test_expect_success 'cherry-pick does not clean cwd incidentally' '
 '
 
 test_expect_success 'cherry-pick fails if cwd needs to be removed' '
-	test_required_dir_removal failure git cherry-pick fd_conflict
+	test_required_dir_removal success git cherry-pick fd_conflict
 '
 
 test_expect_success 'rebase does not clean cwd incidentally' '
@@ -184,7 +178,7 @@ test_expect_success 'revert does not clean cwd incidentally' '
 '
 
 test_expect_success 'revert fails if cwd needs to be removed' '
-	test_required_dir_removal failure git revert undo_fd_conflict
+	test_required_dir_removal success git revert undo_fd_conflict
 '
 
 test_expect_success 'rm does not clean cwd incidentally' '
diff --git a/unpack-trees.c b/unpack-trees.c
index 89ca95ce90b..6bc16f3a714 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -36,6 +36,9 @@ static const char *unpack_plumbing_errors[NB_UNPACK_TREES_WARNING_TYPES] = {
 	/* ERROR_NOT_UPTODATE_DIR */
 	"Updating '%s' would lose untracked files in it",
 
+	/* ERROR_CWD_IN_THE_WAY */
+	"Refusing to remove '%s' since it is the current working directory.",
+
 	/* ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN */
 	"Untracked working tree file '%s' would be overwritten by merge.",
 
@@ -131,6 +134,9 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 	msgs[ERROR_NOT_UPTODATE_DIR] =
 		_("Updating the following directories would lose untracked files in them:\n%s");
 
+	msgs[ERROR_CWD_IN_THE_WAY] =
+		_("Refusing to remove the current working directory:\n%s");
+
 	if (!strcmp(cmd, "checkout"))
 		msg = advice_enabled(ADVICE_COMMIT_BEFORE_MERGE)
 		      ? _("The following untracked working tree files would be removed by checkout:\n%%s"
@@ -2146,10 +2152,7 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,
 		cnt++;
 	}
 
-	/*
-	 * Then we need to make sure that we do not lose a locally
-	 * present file that is not ignored.
-	 */
+	/* Do not lose a locally present file that is not ignored. */
 	pathbuf = xstrfmt("%.*s/", namelen, ce->name);
 
 	memset(&d, 0, sizeof(d));
@@ -2160,6 +2163,12 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,
 	free(pathbuf);
 	if (i)
 		return add_rejected_path(o, ERROR_NOT_UPTODATE_DIR, ce->name);
+
+	/* Do not lose startup_info->original_cwd */
+	if (startup_info->original_cwd &&
+	    !strcmp(startup_info->original_cwd, ce->name))
+		return add_rejected_path(o, ERROR_CWD_IN_THE_WAY, ce->name);
+
 	return cnt;
 }
 
diff --git a/unpack-trees.h b/unpack-trees.h
index 71ffb7eeb0c..efb9edfbb27 100644
--- a/unpack-trees.h
+++ b/unpack-trees.h
@@ -19,6 +19,7 @@ enum unpack_trees_error_types {
 	ERROR_WOULD_OVERWRITE = 0,
 	ERROR_NOT_UPTODATE_FILE,
 	ERROR_NOT_UPTODATE_DIR,
+	ERROR_CWD_IN_THE_WAY,
 	ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN,
 	ERROR_WOULD_LOSE_UNTRACKED_REMOVED,
 	ERROR_BIND_OVERLAP,
-- 
gitgitgadget

[PATCH v4 04/11] unpack-trees: add special cwd handling

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

When running commands such as `git reset --hard` from a subdirectory, if
that subdirectory is in the way of adding needed files, bail with an
error message.

Note that this change looks kind of like it duplicates the new lines of
code from the previous commit in verify_clean_subdirectory().  However,
when we are preserving untracked files, we would rather any error
messages about untracked files being in the way take precedence over
error messages about a subdirectory that happens to be the_original_cwd
being in the way.  But in the UNPACK_RESET_OVERWRITE_UNTRACKED case,
there is no untracked checking to be done, so we simply add a special
case near the top of verify_absent_1.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh |  2 +-
 unpack-trees.c       | 13 +++++++++++--
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 55081f5c980..10e4654042c 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -121,7 +121,7 @@ test_expect_success 'reset --hard does not clean cwd incidentally' '
 '
 
 test_expect_success 'reset --hard fails if cwd needs to be removed' '
-	test_required_dir_removal failure git reset --hard fd_conflict
+	test_required_dir_removal success git reset --hard fd_conflict
 '
 
 test_expect_success 'merge does not clean cwd incidentally' '
diff --git a/unpack-trees.c b/unpack-trees.c
index 6bc16f3a714..5852807d2fb 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -2261,10 +2261,19 @@ static int verify_absent_1(const struct cache_entry *ce,
 	int len;
 	struct stat st;
 
-	if (o->index_only || !o->update ||
-	    o->reset == UNPACK_RESET_OVERWRITE_UNTRACKED)
+	if (o->index_only || !o->update)
 		return 0;
 
+	if (o->reset == UNPACK_RESET_OVERWRITE_UNTRACKED) {
+		/* Avoid nuking startup_info->original_cwd... */
+		if (startup_info->original_cwd &&
+		    !strcmp(startup_info->original_cwd, ce->name))
+			return add_rejected_path(o, ERROR_CWD_IN_THE_WAY,
+						 ce->name);
+		/* ...but nuke anything else. */
+		return 0;
+	}
+
 	len = check_leading_path(ce->name, ce_namelen(ce), 0);
 	if (!len)
 		return 0;
-- 
gitgitgadget

[PATCH v4 05/11] symlinks: do not include startup_info->original_cwd in dir removal

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

symlinks has a pair of schedule_dir_for_removal() and
remove_scheduled_dirs() functions that ensure that directories made
empty by removing other files also themselves get removed.  However, we
want to exclude startup_info->original_cwd and leave it around.  This
avoids the user getting confused by subsequent git commands (and non-git
commands) that would otherwise report confusing messages about being
unable to read the current working directory.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 symlinks.c           |  8 +++++++-
 t/t2501-cwd-empty.sh | 10 +++++-----
 2 files changed, 12 insertions(+), 6 deletions(-)

diff --git a/symlinks.c b/symlinks.c
index 5232d02020c..c667baa949b 100644
--- a/symlinks.c
+++ b/symlinks.c
@@ -279,7 +279,9 @@ static void do_remove_scheduled_dirs(int new_len)
 {
 	while (removal.len > new_len) {
 		removal.buf[removal.len] = '\0';
-		if (rmdir(removal.buf))
+		if ((startup_info->original_cwd &&
+		     !strcmp(removal.buf, startup_info->original_cwd)) ||
+		    rmdir(removal.buf))
 			break;
 		do {
 			removal.len--;
@@ -293,6 +295,10 @@ void schedule_dir_for_removal(const char *name, int len)
 {
 	int match_len, last_slash, i, previous_slash;
 
+	if (startup_info->original_cwd &&
+	    !strcmp(name, startup_info->original_cwd))
+		return;	/* Do not remove the current working directory */
+
 	match_len = last_slash = i =
 		longest_path_match(name, len, removal.buf, removal.len,
 				   &previous_slash);
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 10e4654042c..50619de10aa 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -109,7 +109,7 @@ test_required_dir_removal () {
 }
 
 test_expect_success 'checkout does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git checkout init
+	test_incidental_dir_removal success git checkout init
 '
 
 test_expect_success 'checkout fails if cwd needs to be removed' '
@@ -117,7 +117,7 @@ test_expect_success 'checkout fails if cwd needs to be removed' '
 '
 
 test_expect_success 'reset --hard does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git reset --hard init
+	test_incidental_dir_removal success git reset --hard init
 '
 
 test_expect_success 'reset --hard fails if cwd needs to be removed' '
@@ -125,7 +125,7 @@ test_expect_success 'reset --hard fails if cwd needs to be removed' '
 '
 
 test_expect_success 'merge does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git merge reverted
+	test_incidental_dir_removal success git merge reverted
 '
 
 # This file uses some simple merges where
@@ -158,7 +158,7 @@ test_expect_success 'merge fails if cwd needs to be removed' '
 '
 
 test_expect_success 'cherry-pick does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git cherry-pick reverted
+	test_incidental_dir_removal success git cherry-pick reverted
 '
 
 test_expect_success 'cherry-pick fails if cwd needs to be removed' '
@@ -174,7 +174,7 @@ test_expect_success 'rebase fails if cwd needs to be removed' '
 '
 
 test_expect_success 'revert does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git revert HEAD
+	test_incidental_dir_removal success git revert HEAD
 '
 
 test_expect_success 'revert fails if cwd needs to be removed' '
-- 
gitgitgadget

[PATCH v4 06/11] clean: do not attempt to remove startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/clean.c      | 44 +++++++++++++++++++++++++++++++++++---------
 t/t2501-cwd-empty.sh |  5 +++--
 2 files changed, 38 insertions(+), 11 deletions(-)

diff --git a/builtin/clean.c b/builtin/clean.c
index 98a2860409b..3ff02bbbffe 100644
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@ -36,6 +36,8 @@ static const char *msg_skip_git_dir = N_("Skipping repository %s\n");
 static const char *msg_would_skip_git_dir = N_("Would skip repository %s\n");
 static const char *msg_warn_remove_failed = N_("failed to remove %s");
 static const char *msg_warn_lstat_failed = N_("could not lstat %s\n");
+static const char *msg_skip_cwd = N_("Refusing to remove current working directory\n");
+static const char *msg_would_skip_cwd = N_("Would refuse to remove current working directory\n");
 
 enum color_clean {
 	CLEAN_COLOR_RESET = 0,
@@ -153,6 +155,8 @@ static int remove_dirs(struct strbuf *path, const char *prefix, int force_flag,
 {
 	DIR *dir;
 	struct strbuf quoted = STRBUF_INIT;
+	struct strbuf realpath = STRBUF_INIT;
+	struct strbuf real_ocwd = STRBUF_INIT;
 	struct dirent *e;
 	int res = 0, ret = 0, gone = 1, original_len = path->len, len;
 	struct string_list dels = STRING_LIST_INIT_DUP;
@@ -231,16 +235,36 @@ static int remove_dirs(struct strbuf *path, const char *prefix, int force_flag,
 	strbuf_setlen(path, original_len);
 
 	if (*dir_gone) {
-		res = dry_run ? 0 : rmdir(path->buf);
-		if (!res)
-			*dir_gone = 1;
-		else {
-			int saved_errno = errno;
-			quote_path(path->buf, prefix, &quoted, 0);
-			errno = saved_errno;
-			warning_errno(_(msg_warn_remove_failed), quoted.buf);
+		/*
+		 * Normalize path components in path->buf, e.g. change '\' to
+		 * '/' on Windows.
+		 */
+		strbuf_realpath(&realpath, path->buf, 1);
+
+		/*
+		 * path and realpath are absolute; for comparison, we would
+		 * like to transform startup_info->original_cwd to an absolute
+		 * path too.
+		 */
+		 if (startup_info->original_cwd)
+			 strbuf_realpath(&real_ocwd,
+					 startup_info->original_cwd, 1);
+
+		if (!strbuf_cmp(&realpath, &real_ocwd)) {
+			printf("%s", dry_run ? _(msg_would_skip_cwd) : _(msg_skip_cwd));
 			*dir_gone = 0;
-			ret = 1;
+		} else {
+			res = dry_run ? 0 : rmdir(path->buf);
+			if (!res)
+				*dir_gone = 1;
+			else {
+				int saved_errno = errno;
+				quote_path(path->buf, prefix, &quoted, 0);
+				errno = saved_errno;
+				warning_errno(_(msg_warn_remove_failed), quoted.buf);
+				*dir_gone = 0;
+				ret = 1;
+			}
 		}
 	}
 
@@ -250,6 +274,8 @@ static int remove_dirs(struct strbuf *path, const char *prefix, int force_flag,
 			printf(dry_run ?  _(msg_would_remove) : _(msg_remove), dels.items[i].string);
 	}
 out:
+	strbuf_release(&realpath);
+	strbuf_release(&real_ocwd);
 	strbuf_release(&quoted);
 	string_list_clear(&dels, 0);
 	return ret;
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 50619de10aa..6ab82e05d98 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -230,8 +230,9 @@ test_incidental_untracked_dir_removal () {
 }
 
 test_expect_success 'clean does not remove cwd incidentally' '
-	test_incidental_untracked_dir_removal failure \
-		git -C .. clean -fd -e warnings . >warnings
+	test_incidental_untracked_dir_removal success \
+		git -C .. clean -fd -e warnings . >warnings &&
+	grep "Refusing to remove current working directory" warnings
 '
 
 test_expect_success 'stash does not remove cwd incidentally' '
-- 
gitgitgadget

[PATCH v4 07/11] rebase: do not attempt to remove startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Since rebase spawns a `checkout` subprocess, make sure we run that from
the startup_info->original_cwd directory, so that the checkout process
knows to protect that directory.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 sequencer.c          | 2 ++
 t/t2501-cwd-empty.sh | 4 ++--
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index ea96837cde3..83f257e7fa4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -4228,6 +4228,8 @@ static int run_git_checkout(struct repository *r, struct replay_opts *opts,
 
 	cmd.git_cmd = 1;
 
+	if (startup_info->original_cwd)
+		cmd.dir = startup_info->original_cwd;
 	strvec_push(&cmd.args, "checkout");
 	strvec_push(&cmd.args, commit);
 	strvec_pushf(&cmd.env_array, GIT_REFLOG_ACTION "=%s", action);
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 6ab82e05d98..4f7dba9ab3d 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -166,11 +166,11 @@ test_expect_success 'cherry-pick fails if cwd needs to be removed' '
 '
 
 test_expect_success 'rebase does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git rebase reverted
+	test_incidental_dir_removal success git rebase reverted
 '
 
 test_expect_success 'rebase fails if cwd needs to be removed' '
-	test_required_dir_removal failure git rebase fd_conflict
+	test_required_dir_removal success git rebase fd_conflict
 '
 
 test_expect_success 'revert does not clean cwd incidentally' '
-- 
gitgitgadget

[PATCH v4 08/11] stash: do not attempt to remove startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Since stash spawns a `clean` subprocess, make sure we run that from the
startup_info->original_cwd directory, so that the `clean` processs knows
to protect that directory.  Also, since the `clean` command might no
longer run from the toplevel, pass the ':/' magic pathspec to ensure we
still clean from the toplevel.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/stash.c      | 4 +++-
 t/t2501-cwd-empty.sh | 2 +-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/builtin/stash.c b/builtin/stash.c
index a0ccc8654df..de0e432a4ff 100644
--- a/builtin/stash.c
+++ b/builtin/stash.c
@@ -1485,8 +1485,10 @@ static int do_push_stash(const struct pathspec *ps, const char *stash_msg, int q
 			struct child_process cp = CHILD_PROCESS_INIT;
 
 			cp.git_cmd = 1;
+			if (startup_info->original_cwd)
+				cp.dir = startup_info->original_cwd;
 			strvec_pushl(&cp.args, "clean", "--force",
-				     "--quiet", "-d", NULL);
+				     "--quiet", "-d", ":/", NULL);
 			if (include_untracked == INCLUDE_ALL_FILES)
 				strvec_push(&cp.args, "-x");
 			if (run_command(&cp)) {
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 4f7dba9ab3d..e2586a63eca 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -236,7 +236,7 @@ test_expect_success 'clean does not remove cwd incidentally' '
 '
 
 test_expect_success 'stash does not remove cwd incidentally' '
-	test_incidental_untracked_dir_removal failure \
+	test_incidental_untracked_dir_removal success \
 		git stash --include-untracked
 '
 
-- 
gitgitgadget

[PATCH v4 09/11] dir: avoid incidentally removing the original_cwd in remove_path()

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Modern git often tries to avoid leaving empty directories around when
removing files.  Originally, it did not bother.  This behavior started
with commit 80e21a9ed809 (merge-recursive::removeFile: remove empty
directories, 2005-11-19), stating the reason simply as:

    When the last file in a directory is removed as the result of a
    merge, try to rmdir the now-empty directory.

This was reimplemented in C and renamed to remove_path() in commit
e1b3a2cad7 ("Build-in merge-recursive", 2008-02-07), but was still
internal to merge-recursive.

This trend towards removing leading empty directories continued with
commit d9b814cc97f1 (Add builtin "git rm" command, 2006-05-19), which
stated the reasoning as:

    The other question is what to do with leading directories. The old
    "git rm" script didn't do anything, which is somewhat inconsistent.
    This one will actually clean up directories that have become empty
    as a result of removing the last file, but maybe we want to have a
    flag to decide the behaviour?

remove_path() in dir.c was added in 4a92d1bfb784 (Add remove_path: a
function to remove as much as possible of a path, 2008-09-27), because
it was noted that we had two separate implementations of the same idea
AND both were buggy.  It described the purpose of the function as

    a function to remove as much as possible of a path

Why remove as much as possible?  Well, at the time we probably would
have said something like:

  * removing leading directories makes things feel tidy
  * removing leading directories doesn't hurt anything so long as they
    had no files in them.

But I don't believe those reasons hold when the empty directory happens
to be the current working directory we inherited from our parent
process.  Leaving the parent process in a deleted directory can cause
user confusion when subsequent processes fail: any git command, for
example, will immediately fail with

    fatal: Unable to read current working directory: No such file or directory

Other commands may similarly get confused.  Modify remove_path() so that
the empty leading directories it also deletes does not include the
current working directory we inherited from our parent process.  I have
looked through every caller of remove_path() in the current codebase to
make sure that all should take this change.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 dir.c                |  3 +++
 dir.h                |  6 +++++-
 t/t2501-cwd-empty.sh | 12 ++++--------
 3 files changed, 12 insertions(+), 9 deletions(-)

diff --git a/dir.c b/dir.c
index 94489298f4c..97d6b71c872 100644
--- a/dir.c
+++ b/dir.c
@@ -3327,6 +3327,9 @@ int remove_path(const char *name)
 		slash = dirs + (slash - name);
 		do {
 			*slash = '\0';
+			if (startup_info->original_cwd &&
+			    !strcmp(startup_info->original_cwd, dirs))
+				break;
 		} while (rmdir(dirs) == 0 && (slash = strrchr(dirs, '/')));
 		free(dirs);
 	}
diff --git a/dir.h b/dir.h
index 83f46c0fb4c..d6a5d03bec2 100644
--- a/dir.h
+++ b/dir.h
@@ -504,7 +504,11 @@ int get_sparse_checkout_patterns(struct pattern_list *pl);
  */
 int remove_dir_recursively(struct strbuf *path, int flag);
 
-/* tries to remove the path with empty directories along it, ignores ENOENT */
+/*
+ * Tries to remove the path, along with leading empty directories so long as
+ * those empty directories are not startup_info->original_cwd.  Ignores
+ * ENOENT.
+ */
 int remove_path(const char *path);
 
 int fspathcmp(const char *a, const char *b);
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index e2586a63eca..30b8ffaa11b 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -182,12 +182,12 @@ test_expect_success 'revert fails if cwd needs to be removed' '
 '
 
 test_expect_success 'rm does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git rm bar/baz.t
+	test_incidental_dir_removal success git rm bar/baz.t
 '
 
 test_expect_success 'apply does not remove cwd incidentally' '
 	git diff HEAD HEAD~1 >patch &&
-	test_incidental_dir_removal failure git apply ../patch
+	test_incidental_dir_removal success git apply ../patch
 '
 
 test_incidental_untracked_dir_removal () {
@@ -271,12 +271,8 @@ test_expect_success '`rm -rf dir` even with only tracked files will remove somet
 	) &&
 
 	test_path_is_missing a/b/c/tracked &&
-	## We would prefer if a/b was still present, though empty, since it
-	## was the current working directory
-	#test_path_is_dir a/b
-	## But the current behavior is that it not only deletes the directory
-	## a/b as requested, but also goes and deletes a
-	test_path_is_missing a
+	test_path_is_missing a/b/c &&
+	test_path_is_dir a/b
 '
 
 test_expect_success 'git version continues working from a deleted dir' '
-- 
gitgitgadget

[PATCH v4 10/11] dir: new flag to remove_dir_recurse() to spare the original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

remove_dir_recurse(), and its non-static wrapper called
remove_dir_recursively(), both take flags for modifying its behavior.
As with the previous commits, we would generally like to protect
the original_cwd, but we want to forced user commands (e.g. 'git rm -rf
...') or other special cases to remove it.  Add a flag for this purpose.
After reading through every caller of remove_dir_recursively() in the
current codebase, there was only one that should be adjusted and that
one only in a very unusual circumstance.  Add a pair of new testcases to
highlight that very specific case involving submodules && --git-dir &&
--work-tree.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/rm.c         |  3 ++-
 dir.c                | 12 +++++++++---
 dir.h                |  3 +++
 t/t2501-cwd-empty.sh |  5 -----
 4 files changed, 14 insertions(+), 9 deletions(-)

diff --git a/builtin/rm.c b/builtin/rm.c
index 3d0967cdc11..b4132e5d8ee 100644
--- a/builtin/rm.c
+++ b/builtin/rm.c
@@ -399,12 +399,13 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
 	if (!index_only) {
 		int removed = 0, gitmodules_modified = 0;
 		struct strbuf buf = STRBUF_INIT;
+		int flag = force ? REMOVE_DIR_PURGE_ORIGINAL_CWD : 0;
 		for (i = 0; i < list.nr; i++) {
 			const char *path = list.entry[i].name;
 			if (list.entry[i].is_submodule) {
 				strbuf_reset(&buf);
 				strbuf_addstr(&buf, path);
-				if (remove_dir_recursively(&buf, 0))
+				if (remove_dir_recursively(&buf, flag))
 					die(_("could not remove '%s'"), path);
 
 				removed = 1;
diff --git a/dir.c b/dir.c
index 97d6b71c872..52064345a6b 100644
--- a/dir.c
+++ b/dir.c
@@ -3204,6 +3204,7 @@ static int remove_dir_recurse(struct strbuf *path, int flag, int *kept_up)
 	int ret = 0, original_len = path->len, len, kept_down = 0;
 	int only_empty = (flag & REMOVE_DIR_EMPTY_ONLY);
 	int keep_toplevel = (flag & REMOVE_DIR_KEEP_TOPLEVEL);
+	int purge_original_cwd = (flag & REMOVE_DIR_PURGE_ORIGINAL_CWD);
 	struct object_id submodule_head;
 
 	if ((flag & REMOVE_DIR_KEEP_NESTED_GIT) &&
@@ -3259,9 +3260,14 @@ static int remove_dir_recurse(struct strbuf *path, int flag, int *kept_up)
 	closedir(dir);
 
 	strbuf_setlen(path, original_len);
-	if (!ret && !keep_toplevel && !kept_down)
-		ret = (!rmdir(path->buf) || errno == ENOENT) ? 0 : -1;
-	else if (kept_up)
+	if (!ret && !keep_toplevel && !kept_down) {
+		if (!purge_original_cwd &&
+		    startup_info->original_cwd &&
+		    !strcmp(startup_info->original_cwd, path->buf))
+			ret = -1; /* Do not remove current working directory */
+		else
+			ret = (!rmdir(path->buf) || errno == ENOENT) ? 0 : -1;
+	} else if (kept_up)
 		/*
 		 * report the uplevel that it is not an error that we
 		 * did not rmdir() our directory.
diff --git a/dir.h b/dir.h
index d6a5d03bec2..8e02dfb505d 100644
--- a/dir.h
+++ b/dir.h
@@ -495,6 +495,9 @@ int get_sparse_checkout_patterns(struct pattern_list *pl);
 /* Remove the contents of path, but leave path itself. */
 #define REMOVE_DIR_KEEP_TOPLEVEL 04
 
+/* Remove the_original_cwd too */
+#define REMOVE_DIR_PURGE_ORIGINAL_CWD 0x08
+
 /*
  * Remove path and its contents, recursively. flags is a combination
  * of the above REMOVE_DIR_* constants. Return 0 on success.
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 30b8ffaa11b..6d8f68c08dd 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -291,11 +291,6 @@ test_submodule_removal () {
 	test_status=
 	test "$path_status" = dir && test_status=test_must_fail
 
-	# Actually, while path_status == dir && test_status=test_must_fail
-	# reflect our desired behavior, current behavior is:
-	path_status=missing
-	test_status=
-
 	test_when_finished "git reset --hard HEAD~1" &&
 	test_when_finished "rm -rf .git/modules/my_submodule" &&
 
-- 
gitgitgadget

[PATCH v4 11/11] t2501: simplify the tests since we can now assume desired behavior

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

We no longer are dealing with a mixture of previous and desired
behavior, so simplify the tests a bit.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh | 123 +++++++++++++------------------------------
 1 file changed, 36 insertions(+), 87 deletions(-)

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 6d8f68c08dd..f6d8d7d03d7 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -32,9 +32,6 @@ test_expect_success setup '
 '
 
 test_incidental_dir_removal () {
-	works=$1 &&
-	shift &&
-
 	test_when_finished "git reset --hard" &&
 
 	git checkout foo/bar/baz^{commit} &&
@@ -44,88 +41,57 @@ test_incidental_dir_removal () {
 		cd foo &&
 		"$@" &&
 
-		# Although we want pwd & git status to pass, test for existing
-		# rather than desired behavior.
-		if test "$works" == "success"
-		then
-			test-tool getcwd &&
-			git status --porcelain
-		else
-			! test-tool getcwd &&
-			test_might_fail git status --porcelain
-		fi
+		# Make sure foo still exists, and commands needing it work
+		test-tool getcwd &&
+		git status --porcelain
 	) &&
 	test_path_is_missing foo/bar/baz &&
 	test_path_is_missing foo/bar &&
 
-	# Although we want dir to be present, test for existing rather
-	# than desired behavior.
-	if test "$works" == "success"
-	then
-		test_path_is_dir foo
-	else
-		test_path_is_missing foo
-	fi
+	test_path_is_dir foo
 }
 
 test_required_dir_removal () {
-	works=$1 &&
-	shift &&
-
 	git checkout df_conflict^{commit} &&
 	test_when_finished "git clean -fdx" &&
 
 	(
 		cd dirORfile &&
 
-		# We'd like for the command to fail (much as it would if there
-		# was an untracked file there), and for the index and worktree
-		# to be left clean with pwd and git status working afterwards.
-		# But test for existing rather than desired behavior.
-		if test "$works" == "success"
-		then
-			test_must_fail "$@" 2>../error &&
-			grep "Refusing to remove.*current working directory" ../error &&
-
-			git diff --exit-code HEAD &&
-
-			test-tool getcwd &&
-			git status --porcelain
-		else
-			"$@" &&
-			! test-tool getcwd &&
-			test_might_fail git status --porcelain
-		fi
+		# Ensure command refuses to run
+		test_must_fail "$@" 2>../error &&
+		grep "Refusing to remove.*current working directory" ../error &&
+
+		# ...and that the index and working tree are left clean
+		git diff --exit-code HEAD &&
+
+		# Ensure that getcwd and git status do not error out (which
+		# they might if the current working directory had been removed)
+		test-tool getcwd &&
+		git status --porcelain
 	) &&
 
-	# Although we want dirORfile to be present, test for existing rather
-	# than desired behavior.
-	if test "$works" == "success"
-	then
-		test_path_is_dir dirORfile
-	else
-		test_path_is_file dirORfile
-	fi
+	test_path_is_dir dirORfile
 }
 
 test_expect_success 'checkout does not clean cwd incidentally' '
-	test_incidental_dir_removal success git checkout init
+	test_incidental_dir_removal git checkout init
 '
 
 test_expect_success 'checkout fails if cwd needs to be removed' '
-	test_required_dir_removal success git checkout fd_conflict
+	test_required_dir_removal git checkout fd_conflict
 '
 
 test_expect_success 'reset --hard does not clean cwd incidentally' '
-	test_incidental_dir_removal success git reset --hard init
+	test_incidental_dir_removal git reset --hard init
 '
 
 test_expect_success 'reset --hard fails if cwd needs to be removed' '
-	test_required_dir_removal success git reset --hard fd_conflict
+	test_required_dir_removal git reset --hard fd_conflict
 '
 
 test_expect_success 'merge does not clean cwd incidentally' '
-	test_incidental_dir_removal success git merge reverted
+	test_incidental_dir_removal git merge reverted
 '
 
 # This file uses some simple merges where
@@ -154,46 +120,43 @@ test_expect_success 'merge fails if cwd needs to be removed; recursive friendly'
 GIT_TEST_MERGE_ALGORITHM=ort
 
 test_expect_success 'merge fails if cwd needs to be removed' '
-	test_required_dir_removal success git merge fd_conflict
+	test_required_dir_removal git merge fd_conflict
 '
 
 test_expect_success 'cherry-pick does not clean cwd incidentally' '
-	test_incidental_dir_removal success git cherry-pick reverted
+	test_incidental_dir_removal git cherry-pick reverted
 '
 
 test_expect_success 'cherry-pick fails if cwd needs to be removed' '
-	test_required_dir_removal success git cherry-pick fd_conflict
+	test_required_dir_removal git cherry-pick fd_conflict
 '
 
 test_expect_success 'rebase does not clean cwd incidentally' '
-	test_incidental_dir_removal success git rebase reverted
+	test_incidental_dir_removal git rebase reverted
 '
 
 test_expect_success 'rebase fails if cwd needs to be removed' '
-	test_required_dir_removal success git rebase fd_conflict
+	test_required_dir_removal git rebase fd_conflict
 '
 
 test_expect_success 'revert does not clean cwd incidentally' '
-	test_incidental_dir_removal success git revert HEAD
+	test_incidental_dir_removal git revert HEAD
 '
 
 test_expect_success 'revert fails if cwd needs to be removed' '
-	test_required_dir_removal success git revert undo_fd_conflict
+	test_required_dir_removal git revert undo_fd_conflict
 '
 
 test_expect_success 'rm does not clean cwd incidentally' '
-	test_incidental_dir_removal success git rm bar/baz.t
+	test_incidental_dir_removal git rm bar/baz.t
 '
 
 test_expect_success 'apply does not remove cwd incidentally' '
 	git diff HEAD HEAD~1 >patch &&
-	test_incidental_dir_removal success git apply ../patch
+	test_incidental_dir_removal git apply ../patch
 '
 
 test_incidental_untracked_dir_removal () {
-	works=$1 &&
-	shift &&
-
 	test_when_finished "git reset --hard" &&
 
 	git checkout foo/bar/baz^{commit} &&
@@ -205,38 +168,24 @@ test_incidental_untracked_dir_removal () {
 		cd untracked &&
 		"$@" &&
 
-		# Although we want pwd & git status to pass, test for existing
-		# rather than desired behavior.
-		if test "$works" == "success"
-		then
-			test-tool getcwd &&
-			git status --porcelain
-		else
-			! test-tool getcwd &&
-			test_might_fail git status --porcelain
-		fi
+		# Make sure untracked still exists, and commands needing it work
+		test-tool getcwd &&
+		git status --porcelain
 	) &&
 	test_path_is_missing empty &&
 	test_path_is_missing untracked/random &&
 
-	# Although we want dir to be present, test for existing rather
-	# than desired behavior.
-	if test "$works" == "success"
-	then
-		test_path_is_dir untracked
-	else
-		test_path_is_missing untracked
-	fi
+	test_path_is_dir untracked
 }
 
 test_expect_success 'clean does not remove cwd incidentally' '
-	test_incidental_untracked_dir_removal success \
+	test_incidental_untracked_dir_removal \
 		git -C .. clean -fd -e warnings . >warnings &&
 	grep "Refusing to remove current working directory" warnings
 '
 
 test_expect_success 'stash does not remove cwd incidentally' '
-	test_incidental_untracked_dir_removal success \
+	test_incidental_untracked_dir_removal \
 		git stash --include-untracked
 '
 
-- 
gitgitgadget

Re: [PATCH v4 00/11] Avoid removing the current working directory, even if it becomes empty

[Eric Sunshine]

On Mon, Nov 29, 2021 at 6:10 PM Elijah Newren via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> Changes since v3:
>  * fixed bashism
>      -+         if [[ $works == "success" ]]; then
>      ++         if test "$works" == "success"

This and any other instances in the series should be using POSIX `=`,
not Bash `==`.

Re: [PATCH v4 00/11] Avoid removing the current working directory, even if it becomes empty

[Elijah Newren]

On Mon, Nov 29, 2021 at 3:38 PM Eric Sunshine <sunshine@sunshineco.com> wrote:
>
> On Mon, Nov 29, 2021 at 6:10 PM Elijah Newren via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
> > Changes since v3:
> >  * fixed bashism
> >      -+         if [[ $works == "success" ]]; then
> >      ++         if test "$works" == "success"
>
> This and any other instances in the series should be using POSIX `=`,
> not Bash `==`.

Ugh, I had them removed by the end of the series but somehow missed
checking the earlier patch.  I'll fix it up; thanks for spotting.

Re: [PATCH v4 01/11] t2501: add various tests for removing the current working directory

[Junio C Hamano]

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> +		# Although we want pwd & git status to pass, test for existing
> +		# rather than desired behavior.
> +		if test "$works" == "success"

Don't double "=" here.  We are not writing for bash.  There are a
few more instances of the same mistake in this patch.

Re: [PATCH v4 01/11] t2501: add various tests for removing the current working directory

[Elijah Newren]

On Mon, Nov 29, 2021 at 10:47 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > +             # Although we want pwd & git status to pass, test for existing
> > +             # rather than desired behavior.
> > +             if test "$works" == "success"
>
> Don't double "=" here.  We are not writing for bash.  There are a
> few more instances of the same mistake in this patch.

Yep, also pointed out by Eric.  I've got a v5 queued up whose change
is just fixing this, but I'm waiting a bit to see if other items are
spotted in review before sending.

Thanks for spotting.

Re: [PATCH v3 00/11] Avoid removing the current working directory, even if it becomes empty

[Phillip Wood]

Hi Elijah

On 26/11/2021 22:40, Elijah Newren via GitGitGadget wrote:
> Traditionally, if folks run git commands such as checkout or rebase from a
> subdirectory, that git command could remove their current working directory
> and result in subsequent git and non-git commands either getting confused or
> printing messages that confuse the user (e.g. "fatal: Unable to read current
> working directory: No such file or directory"). Many commands either
> silently avoid removing directories that are not empty (i.e. those that have
> untracked or modified files in them)[1], or show an error and abort,
> depending on which is more appropriate for the command in question. With
> this series, we augment the reasons to avoid removing directories to include
> not just has-untracked-or-modified-files, but also to avoid removing the
> original_cwd as well.
> 
> Peff and Junio provided some good pros/cons, if it helps:
> 
>   * Pros: Peff (original suggester of the idea)[2], and Junio[3]
>   * Cons: Peff [2, again -- see the "P.S."], and Junio[4]
> 
> [1] well, with a few exceptions; see
> https://lore.kernel.org/git/pull.1036.v3.git.1632760428.gitgitgadget@gmail.com/
> [2] https://lore.kernel.org/git/YS8eEtwQvF7TaLCb@coredump.intra.peff.net/
> [3] https://lore.kernel.org/git/xmqqo86elyht.fsf@gitster.g/ [4]
> https://lore.kernel.org/git/xmqqo8691gr8.fsf@gitster.g/


Thanks for working on this, I'm sorry I haven't had time to take a 
proper look at it but I think it is a good idea. I did notice that 
you're using ":/" in patch 8 and wondered what happens if some runs 'git 
--literal-pathspecs stash' but I haven't looked properly.

Best Wishes

Phillip

> Changes since v2:
> 
>   * the series is now only about the working tree. So if the original cwd is
>     outside the worktree (or we're in a bare repo), then the new code is a
>     no-op.
>   * fixed ugly early die() possibility (uses strbuf_getcwd() instead of
>     xgetcwd())
>   * modified the initial tests to show both expected and desired behavior.
>     subsequent patches fix the tests. One new patch added at the end which
>     simplifies the tests to only check for desired behavior.
>   * NULLify startup_info->original_cwd when it matches the toplevel worktree;
>     that is already protected and we don't need secondary protection for it.
>     This simplified some other codepaths so we don't have to check for
>     startup_info->original_cwd == "".
>   * clarified some commit messages
> 
> Changes since v1:
> 
>   * clarified multiple commit messages
>   * renamed the_cwd to startup_info->original_cwd to make it clearer that
>     it's our parent process'es cwd that really matters, which we inherited at
>     program startup. Also pulls it out of the global namespace.
>   * Normalize the path for startup_info->original_cwd, and ensure that it's
>     actually the original cwd even if -C is passed to git.
>   * small code cleanups suggested by René and Ævar
>   * split the final patch, which got the most comments into two, one for each
>     function being modified; significantly extending the first of the two
>     commit messages with a lot of history
>   * no longer has a content conflict with so/stash-staged
>   * add another value for the flags parameter that remove_dir_recursively()
>     takes so that it can opt into either the old or the new behavior. Use
>     that for the one special corner case I could find where it matters, and
>     add a few tests around it to highlight the utility of the flag.
> 
> Elijah Newren (11):
>    t2501: add various tests for removing the current working directory
>    setup: introduce startup_info->original_cwd
>    unpack-trees: refuse to remove startup_info->original_cwd
>    unpack-trees: add special cwd handling
>    symlinks: do not include startup_info->original_cwd in dir removal
>    clean: do not attempt to remove startup_info->original_cwd
>    rebase: do not attempt to remove startup_info->original_cwd
>    stash: do not attempt to remove startup_info->original_cwd
>    dir: avoid incidentally removing the original_cwd in remove_path()
>    dir: new flag to remove_dir_recurse() to spare the original_cwd
>    t2501: simplify the tests since we can now assume desired behavior
> 
>   builtin/clean.c      |  44 +++++--
>   builtin/rm.c         |   3 +-
>   builtin/stash.c      |   5 +-
>   cache.h              |   2 +
>   common-main.c        |   4 +
>   dir.c                |  15 ++-
>   dir.h                |   9 +-
>   sequencer.c          |   3 +
>   setup.c              |  65 +++++++++++
>   symlinks.c           |   8 +-
>   t/t2501-cwd-empty.sh | 268 +++++++++++++++++++++++++++++++++++++++++++
>   unpack-trees.c       |  30 ++++-
>   unpack-trees.h       |   1 +
>   13 files changed, 435 insertions(+), 22 deletions(-)
>   create mode 100755 t/t2501-cwd-empty.sh
> 
> 
> base-commit: 88d915a634b449147855041d44875322de2b286d
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1140%2Fnewren%2Fcwd_removal-v3
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1140/newren/cwd_removal-v3
> Pull-Request: https://github.com/git/git/pull/1140
> 
> Range-diff vs v2:
> 
>    1:  38a120f5c03 !  1:  4b0044656b0 t2501: add various tests for removing the current working directory
>       @@ Metadata
>         ## Commit message ##
>            t2501: add various tests for removing the current working directory
>        
>       -    Numerous commands will remove empty working directories, especially if
>       -    they are in the way of placing needed files.  That is normally fine, but
>       -    removing the current working directory can cause confusion for the user
>       -    when they run subsequent commands.  For example, after one git process
>       -    has removed the current working directory, git status/log/diff will all
>       -    abort with the message:
>       +    Numerous commands will remove directories left empty as a "convenience"
>       +    after removing files within them.  That is normally fine, but removing
>       +    the current working directory can be rather inconvenient since it can
>       +    cause confusion for the user when they run subsequent commands.  For
>       +    example, after one git process has removed the current working
>       +    directory, git status/log/diff will all abort with the message:
>        
>                fatal: Unable to read current working directory: No such file or directory
>        
>       +    We also have code paths that, when a file needs to be placed where a
>       +    directory is (due to e.g. checkout, merge, reset, whatever), will check
>       +    if this is okay and error out if not.  These rules include:
>       +      * all tracked files under that directory are intended to be removed by
>       +        the operation
>       +      * none of the tracked files under that directory have uncommitted
>       +        modification
>       +      * there are no untracked files under that directory
>       +    However, if we end up remove the current working directory, we can cause
>       +    user confusion when they run subsequent commands, so we would prefer if
>       +    there was a fourth rule added to this list: avoid removing the current
>       +    working directory.
>       +
>            Since there are several code paths that can result in the current
>            working directory being removed, add several tests of various different
>       -    codepaths that check for the behavior we would instead like to see.
>       -    This include a number of new error messages that we will be adding in
>       -    subsequent commits as we implement the desired checks.
>       +    codepaths.  To make it clearer what the difference between the current
>       +    behavior and the behavior at the end of the series, code both of them
>       +    into the tests and have the appropriate behavior be selected by a flag.
>       +    Subsequent commits will toggle the flag from current to desired
>       +    behavior.
>       +
>       +    Also add a few tests suggested during the review of earlier rounds of
>       +    this patch series.
>        
>            Signed-off-by: Elijah Newren <newren@gmail.com>
>        
>       @@ t/t2501-cwd-empty.sh (new)
>        +
>        +test_expect_success setup '
>        +	test_commit init &&
>       -+	mkdir subdir &&
>       -+	test_commit subdir/file &&
>        +
>        +	git branch fd_conflict &&
>        +
>       @@ t/t2501-cwd-empty.sh (new)
>        +	git tag reverted &&
>        +
>        +	git checkout fd_conflict &&
>       -+	git rm subdir/file.t &&
>       ++	mkdir dirORfile &&
>       ++	test_commit dirORfile/foo &&
>       ++
>       ++	git rm -r dirORfile &&
>        +	echo not-a-directory >dirORfile &&
>        +	git add dirORfile &&
>       -+	git commit -m dirORfile
>       ++	git commit -m dirORfile &&
>       ++
>       ++	git switch -c df_conflict HEAD~1 &&
>       ++	test_commit random_file &&
>       ++
>       ++	git switch -c undo_fd_conflict fd_conflict &&
>       ++	git revert HEAD
>        +'
>        +
>       -+test_expect_failure 'checkout does not clean cwd incidentally' '
>       -+	git checkout foo/bar/baz &&
>       ++test_incidental_dir_removal () {
>       ++	works=$1 &&
>       ++	shift &&
>       ++
>       ++	test_when_finished "git reset --hard" &&
>       ++
>       ++	git checkout foo/bar/baz^{commit} &&
>        +	test_path_is_dir foo/bar &&
>        +
>        +	(
>        +		cd foo &&
>       -+		git checkout init &&
>       -+		cd ..
>       ++		"$@" &&
>       ++
>       ++		# Although we want pwd & git status to pass, test for existing
>       ++		# rather than desired behavior.
>       ++		if [[ $works == "success" ]]; then
>       ++			pwd -P &&
>       ++			git status --porcelain
>       ++		else
>       ++			! pwd -P &&
>       ++			test_might_fail git status --porcelain
>       ++		fi
>        +	) &&
>        +	test_path_is_missing foo/bar/baz &&
>        +	test_path_is_missing foo/bar &&
>       -+	test_path_is_dir foo
>       -+'
>        +
>       -+test_expect_failure 'checkout fails if cwd needs to be removed' '
>       -+	git checkout foo/bar/baz &&
>       ++	# Although we want dir to be present, test for existing rather
>       ++	# than desired behavior.
>       ++	if [[ $works == "success" ]]; then
>       ++		test_path_is_dir foo
>       ++	else
>       ++		test_path_is_missing foo
>       ++	fi
>       ++}
>       ++
>       ++test_required_dir_removal () {
>       ++	works=$1 &&
>       ++	shift &&
>       ++
>       ++	git checkout df_conflict^{commit} &&
>        +	test_when_finished "git clean -fdx" &&
>        +
>       -+	mkdir dirORfile &&
>        +	(
>        +		cd dirORfile &&
>        +
>       -+		test_must_fail git checkout fd_conflict 2>../error &&
>       -+		grep "Refusing to remove the current working directory" ../error
>       ++		# We'd like for the command to fail (much as it would if there
>       ++		# was an untracked file there), and for pwd & git status to
>       ++		# succeed afterwards.  But test for existing rather than
>       ++		# desired behavior.
>       ++		if [[ $works == "success" ]]; then
>       ++			test_must_fail "$@" 2>../error &&
>       ++			grep "Refusing to remove.*current working directory" ../error &&
>       ++			pwd -P &&
>       ++			git status --porcelain
>       ++		else
>       ++			"$@" &&
>       ++			! pwd -P &&
>       ++			test_might_fail git status --porcelain
>       ++		fi
>        +	) &&
>        +
>       -+	test_path_is_dir dirORfile
>       ++	# Although we want dirORfile to be present, test for existing rather
>       ++	# than desired behavior.
>       ++	if [[ $works == "success" ]]; then
>       ++		test_path_is_dir dirORfile
>       ++	else
>       ++		test_path_is_file dirORfile
>       ++	fi
>       ++}
>       ++
>       ++test_expect_success 'checkout does not clean cwd incidentally' '
>       ++	test_incidental_dir_removal failure git checkout init
>        +'
>        +
>       -+test_expect_failure 'reset --hard does not clean cwd incidentally' '
>       -+	git checkout foo/bar/baz &&
>       -+	test_path_is_dir foo/bar &&
>       ++test_expect_success 'checkout fails if cwd needs to be removed' '
>       ++	test_required_dir_removal failure git checkout fd_conflict
>       ++'
>        +
>       -+	(
>       -+		cd foo &&
>       -+		git reset --hard init &&
>       -+		cd ..
>       -+	) &&
>       -+	test_path_is_missing foo/bar/baz &&
>       -+	test_path_is_missing foo/bar &&
>       -+	test_path_is_dir foo
>       ++test_expect_success 'reset --hard does not clean cwd incidentally' '
>       ++	test_incidental_dir_removal failure git reset --hard init
>       ++'
>       ++
>       ++test_expect_success 'reset --hard fails if cwd needs to be removed' '
>       ++	test_required_dir_removal failure git reset --hard fd_conflict
>       ++'
>       ++
>       ++test_expect_success 'merge does not clean cwd incidentally' '
>       ++	test_incidental_dir_removal failure git merge reverted
>        +'
>        +
>       -+test_expect_failure 'reset --hard fails if cwd needs to be removed' '
>       ++# This file uses some simple merges where
>       ++#   Base: 'dirORfile/' exists
>       ++#   Side1: random other file changed
>       ++#   Side2: 'dirORfile/' removed, 'dirORfile' added
>       ++# this should resolve cleanly, but merge-recursive throws merge conflicts
>       ++# because it's dumb.  Add a special test for checking merge-recursive (and
>       ++# merge-ort), then after this just hard require ort for all remaining tests.
>       ++#
>       ++test_expect_success 'merge fails if cwd needs to be removed; recursive friendly' '
>        +	git checkout foo/bar/baz &&
>        +	test_when_finished "git clean -fdx" &&
>        +
>       @@ t/t2501-cwd-empty.sh (new)
>        +	(
>        +		cd dirORfile &&
>        +
>       -+		test_must_fail git reset --hard fd_conflict 2>../error &&
>       -+		grep "Refusing to remove.*the current working directory" ../error
>       ++		# We would rather this failed, but we test for existing
>       ++		# rather than desired behavior
>       ++		git merge fd_conflict 2>../error
>        +	) &&
>        +
>       -+	test_path_is_dir dirORfile
>       ++	## Here is the behavior we would rather have:
>       ++	#test_path_is_dir dirORfile &&
>       ++	#grep "Refusing to remove the current working directory" error
>       ++	## But instead we test for existing behavior
>       ++	test_path_is_file dirORfile &&
>       ++	test_must_be_empty error
>        +'
>        +
>       -+test_expect_failure 'merge does not remove cwd incidentally' '
>       -+	git checkout foo/bar/baz &&
>       -+	test_when_finished "git clean -fdx" &&
>       ++GIT_TEST_MERGE_ALGORITHM=ort
>        +
>       -+	(
>       -+		cd subdir &&
>       -+		git merge fd_conflict
>       -+	) &&
>       ++test_expect_success 'merge fails if cwd needs to be removed' '
>       ++	test_required_dir_removal failure git merge fd_conflict
>       ++'
>        +
>       -+	test_path_is_missing subdir/file.t &&
>       -+	test_path_is_dir subdir
>       ++test_expect_success 'cherry-pick does not clean cwd incidentally' '
>       ++	test_incidental_dir_removal failure git cherry-pick reverted
>        +'
>        +
>       -+test_expect_failure 'merge fails if cwd needs to be removed' '
>       -+	git checkout foo/bar/baz &&
>       -+	test_when_finished "git clean -fdx" &&
>       ++test_expect_success 'cherry-pick fails if cwd needs to be removed' '
>       ++	test_required_dir_removal failure git cherry-pick fd_conflict
>       ++'
>        +
>       -+	mkdir dirORfile &&
>       -+	(
>       -+		cd dirORfile &&
>       -+		test_must_fail git merge fd_conflict 2>../error &&
>       -+		grep "Refusing to remove the current working directory" ../error
>       -+	) &&
>       ++test_expect_success 'rebase does not clean cwd incidentally' '
>       ++	test_incidental_dir_removal failure git rebase reverted
>       ++'
>        +
>       -+	test_path_is_dir dirORfile
>       ++test_expect_success 'rebase fails if cwd needs to be removed' '
>       ++	test_required_dir_removal failure git rebase fd_conflict
>        +'
>        +
>       -+test_expect_failure 'cherry-pick does not remove cwd incidentally' '
>       -+	git checkout foo/bar/baz &&
>       -+	test_when_finished "git clean -fdx" &&
>       ++test_expect_success 'revert does not clean cwd incidentally' '
>       ++	test_incidental_dir_removal failure git revert HEAD
>       ++'
>        +
>       -+	(
>       -+		cd subdir &&
>       -+		git cherry-pick fd_conflict
>       -+	) &&
>       ++test_expect_success 'revert fails if cwd needs to be removed' '
>       ++	test_required_dir_removal failure git revert undo_fd_conflict
>       ++'
>        +
>       -+	test_path_is_missing subdir/file.t &&
>       -+	test_path_is_dir subdir
>       ++test_expect_success 'rm does not clean cwd incidentally' '
>       ++	test_incidental_dir_removal failure git rm bar/baz.t
>        +'
>        +
>       -+test_expect_failure 'cherry-pick fails if cwd needs to be removed' '
>       -+	git checkout foo/bar/baz &&
>       -+	test_when_finished "git clean -fdx" &&
>       ++test_expect_success 'apply does not remove cwd incidentally' '
>       ++	git diff HEAD HEAD~1 >patch &&
>       ++	test_incidental_dir_removal failure git apply ../patch
>       ++'
>        +
>       -+	mkdir dirORfile &&
>       -+	(
>       -+		cd dirORfile &&
>       -+		test_must_fail git cherry-pick fd_conflict 2>../error &&
>       -+		grep "Refusing to remove the current working directory" ../error
>       -+	) &&
>       ++test_incidental_untracked_dir_removal () {
>       ++	works=$1 &&
>       ++	shift &&
>        +
>       -+	test_path_is_dir dirORfile
>       -+'
>       ++	test_when_finished "git reset --hard" &&
>        +
>       -+test_expect_failure 'rebase does not remove cwd incidentally' '
>       -+	git checkout foo/bar/baz &&
>       -+	test_when_finished "git clean -fdx" &&
>       ++	git checkout foo/bar/baz^{commit} &&
>       ++	mkdir -p untracked &&
>       ++	mkdir empty
>       ++	>untracked/random &&
>        +
>        +	(
>       -+		cd subdir &&
>       -+		git rebase foo/bar/baz fd_conflict
>       ++		cd untracked &&
>       ++		"$@" &&
>       ++
>       ++		# Although we want pwd & git status to pass, test for existing
>       ++		# rather than desired behavior.
>       ++		if [[ $works == "success" ]]; then
>       ++			pwd -P &&
>       ++			git status --porcelain
>       ++		else
>       ++			! pwd -P &&
>       ++			test_might_fail git status --porcelain
>       ++		fi
>        +	) &&
>       ++	test_path_is_missing empty &&
>       ++	test_path_is_missing untracked/random &&
>        +
>       -+	test_path_is_missing subdir/file.t &&
>       -+	test_path_is_dir subdir
>       ++	# Although we want dir to be present, test for existing rather
>       ++	# than desired behavior.
>       ++	if [[ $works == "success" ]]; then
>       ++		test_path_is_dir untracked
>       ++	else
>       ++		test_path_is_missing untracked
>       ++	fi
>       ++}
>       ++
>       ++test_expect_success 'clean does not remove cwd incidentally' '
>       ++	test_incidental_untracked_dir_removal failure \
>       ++		git -C .. clean -fd -e warnings . >warnings
>        +'
>        +
>       -+test_expect_failure 'rebase fails if cwd needs to be removed' '
>       -+	git checkout foo/bar/baz &&
>       -+	test_when_finished "git clean -fdx" &&
>       -+
>       -+	mkdir dirORfile &&
>       -+	(
>       -+		cd dirORfile &&
>       -+		test_must_fail git rebase foo/bar/baz fd_conflict 2>../error &&
>       -+		grep "Refusing to remove the current working directory" ../error
>       -+	) &&
>       -+
>       -+	test_path_is_dir dirORfile
>       ++test_expect_success 'stash does not remove cwd incidentally' '
>       ++	test_incidental_untracked_dir_removal failure \
>       ++		git stash --include-untracked
>        +'
>        +
>       -+test_expect_failure 'revert does not remove cwd incidentally' '
>       -+	git checkout foo/bar/baz &&
>       -+	test_when_finished "git clean -fdx" &&
>       ++test_expect_success '`rm -rf dir` only removes a subset of dir' '
>       ++	test_when_finished "rm -rf a/" &&
>       ++
>       ++	mkdir -p a/b/c &&
>       ++	>a/b/c/untracked &&
>       ++	>a/b/c/tracked &&
>       ++	git add a/b/c/tracked &&
>        +
>        +	(
>       -+		cd subdir &&
>       -+		git revert subdir/file
>       ++		cd a/b &&
>       ++		git rm -rf ../b
>        +	) &&
>        +
>       -+	test_path_is_missing subdir/file.t &&
>       -+	test_path_is_dir subdir
>       ++	test_path_is_dir a/b &&
>       ++	test_path_is_missing a/b/c/tracked &&
>       ++	test_path_is_file a/b/c/untracked
>        +'
>        +
>       -+test_expect_failure 'revert fails if cwd needs to be removed' '
>       -+	git checkout fd_conflict &&
>       -+	git revert HEAD &&
>       -+	test_when_finished "git clean -fdx" &&
>       ++test_expect_success '`rm -rf dir` even with only tracked files will remove something else' '
>       ++	test_when_finished "rm -rf a/" &&
>       ++
>       ++	mkdir -p a/b/c &&
>       ++	>a/b/c/tracked &&
>       ++	git add a/b/c/tracked &&
>        +
>       -+	mkdir dirORfile &&
>        +	(
>       -+		cd dirORfile &&
>       -+		test_must_fail git revert HEAD 2>../error &&
>       -+		grep "Refusing to remove the current working directory" ../error
>       ++		cd a/b &&
>       ++		git rm -rf ../b
>        +	) &&
>        +
>       -+	test_path_is_dir dirORfile
>       ++	test_path_is_missing a/b/c/tracked &&
>       ++	## We would prefer if a/b was still present, though empty, since it
>       ++	## was the current working directory
>       ++	#test_path_is_dir a/b
>       ++	## But the current behavior is that it not only deletes the directory
>       ++	## a/b as requested, but also goes and deletes a
>       ++	test_path_is_missing a
>        +'
>        +
>       -+test_expect_failure 'rm does not remove cwd incidentally' '
>       -+	test_when_finished "git reset --hard" &&
>       -+	git checkout foo/bar/baz &&
>       -+
>       ++test_expect_success 'git version continues working from a deleted dir' '
>       ++	mkdir tmp &&
>        +	(
>       -+		cd foo &&
>       -+		git rm bar/baz.t
>       -+	) &&
>       -+
>       -+	test_path_is_missing foo/bar/baz &&
>       -+	test_path_is_missing foo/bar &&
>       -+	test_path_is_dir foo
>       ++		cd tmp &&
>       ++		rm -rf ../tmp &&
>       ++		git version
>       ++	)
>        +'
>        +
>       -+test_expect_failure 'apply does not remove cwd incidentally' '
>       -+	test_when_finished "git reset --hard" &&
>       -+	git checkout foo/bar/baz &&
>       ++test_submodule_removal () {
>       ++	path_status=$1 &&
>       ++	shift &&
>        +
>       -+	(
>       -+		cd subdir &&
>       -+		git diff subdir/file init | git apply
>       -+	) &&
>       ++	test_status=
>       ++	test $path_status = dir && test_status=test_must_fail
>        +
>       -+	test_path_is_missing subdir/file.t &&
>       -+	test_path_is_dir subdir
>       -+'
>       ++	# Actually, while path_status == dir && test_status=test_must_fail
>       ++	# reflect our desired behavior, current behavior is:
>       ++	path_status=missing
>       ++	test_status=
>       ++
>       ++	test_when_finished "git reset --hard HEAD~1" &&
>       ++	test_when_finished "rm -rf .git/modules/my_submodule" &&
>        +
>       -+test_expect_failure 'clean does not remove cwd incidentally' '
>        +	git checkout foo/bar/baz &&
>       -+	test_when_finished "git clean -fdx" &&
>        +
>       -+	mkdir empty &&
>       -+	mkdir untracked &&
>       -+	>untracked/random &&
>       ++	git init my_submodule &&
>       ++	touch my_submodule/file &&
>       ++	git -C my_submodule add file &&
>       ++	git -C my_submodule commit -m "initial commit" &&
>       ++	git submodule add ./my_submodule &&
>       ++	git commit -m "Add the submodule" &&
>       ++
>        +	(
>       -+		cd untracked &&
>       -+		git clean -fd -e warnings :/ >../warnings &&
>       -+		grep "Refusing to remove current working directory" ../warnings
>       ++		cd my_submodule &&
>       ++		$test_status "$@"
>        +	) &&
>        +
>       -+	test_path_is_missing empty &&
>       -+	test_path_is_missing untracked/random &&
>       -+	test_path_is_dir untracked
>       -+'
>       ++	test_path_is_${path_status} my_submodule
>       ++}
>        +
>       -+test_expect_failure 'stash does not remove cwd incidentally' '
>       -+	git checkout foo/bar/baz &&
>       -+	test_when_finished "git clean -fdx" &&
>       ++test_expect_success 'rm -r with -C leaves submodule if cwd inside' '
>       ++	test_submodule_removal dir git -C .. rm -r my_submodule/
>       ++'
>        +
>       -+	mkdir untracked &&
>       -+	>untracked/random &&
>       -+	(
>       -+		cd untracked &&
>       -+		git stash --include-untracked &&
>       -+		git status
>       -+	) &&
>       ++test_expect_success 'rm -r leaves submodule if cwd inside' '
>       ++	test_submodule_removal dir \
>       ++		git --git-dir=../.git --work-tree=.. rm -r ../my_submodule/
>       ++'
>        +
>       -+	test_path_is_missing untracked/random &&
>       -+	test_path_is_dir untracked
>       ++test_expect_success 'rm -rf removes submodule even if cwd inside' '
>       ++	test_submodule_removal missing \
>       ++		git --git-dir=../.git --work-tree=.. rm -rf ../my_submodule/
>        +'
>        +
>        +test_done
>    2:  f6129a8ac9c !  2:  200ddece05d setup: introduce startup_info->original_cwd
>       @@ cache.h: void overlay_tree_on_index(struct index_state *istate,
>        +	const char *original_cwd;
>         };
>         extern struct startup_info *startup_info;
>       ++extern const char *tmp_original_cwd;
>         
>       + /* merge.c */
>       + struct commit_list;
>        
>       - ## git.c ##
>       -@@ git.c: int cmd_main(int argc, const char **argv)
>       + ## common-main.c ##
>       +@@ common-main.c: static void restore_sigpipe_to_default(void)
>       + int main(int argc, const char **argv)
>       + {
>       + 	int result;
>       ++	struct strbuf tmp = STRBUF_INIT;
>       +
>       + 	trace2_initialize_clock();
>         
>       - 	trace_command_performance(argv);
>       +@@ common-main.c: int main(int argc, const char **argv)
>       + 	trace2_cmd_start(argv);
>       + 	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
>         
>       -+	startup_info->original_cwd = xgetcwd();
>       ++	if (!strbuf_getcwd(&tmp))
>       ++		tmp_original_cwd = strbuf_detach(&tmp, NULL);
>        +
>       - 	/*
>       - 	 * "git-xxxx" is the same as "git xxxx", but we obviously:
>       - 	 *
>       + 	result = cmd_main(argc, argv);
>       +
>       + 	trace2_cmd_exit(result);
>        
>         ## setup.c ##
>       +@@ setup.c: static int work_tree_config_is_bogus;
>       +
>       + static struct startup_info the_startup_info;
>       + struct startup_info *startup_info = &the_startup_info;
>       ++const char *tmp_original_cwd;
>       +
>       + /*
>       +  * The input parameter must contain an absolute path, and it must already be
>        @@ setup.c: void setup_work_tree(void)
>         	initialized = 1;
>         }
>       @@ setup.c: void setup_work_tree(void)
>        +	const char *worktree = NULL;
>        +	int offset = -1;
>        +
>       -+	/*
>       -+	 * startup_info->original_cwd wass set early on in cmd_main(), unless
>       -+	 * we're an auxiliary tool like git-remote-http or test-tool.
>       -+	 */
>       -+	if (!startup_info->original_cwd)
>       ++	if (!tmp_original_cwd)
>        +		return;
>        +
>        +	/*
>        +	 * startup_info->original_cwd points to the current working
>        +	 * directory we inherited from our parent process, which is a
>       -+	 * directory we want to avoid incidentally removing.
>       ++	 * directory we want to avoid removing.
>        +	 *
>        +	 * For convience, we would like to have the path relative to the
>        +	 * worktree instead of an absolute path.
>       @@ setup.c: void setup_work_tree(void)
>        +	 */
>        +
>        +	/* Normalize the directory */
>       -+	strbuf_realpath(&tmp, startup_info->original_cwd, 1);
>       -+	free((char*)startup_info->original_cwd);
>       ++	strbuf_realpath(&tmp, tmp_original_cwd, 1);
>       ++	free((char*)tmp_original_cwd);
>       ++	tmp_original_cwd = NULL;
>        +	startup_info->original_cwd = strbuf_detach(&tmp, NULL);
>        +
>       -+	/* Find out if this is in the worktree */
>       ++	/*
>       ++	 * Get our worktree; we only protect the current working directory
>       ++	 * if it's in the worktree.
>       ++	 */
>        +	worktree = get_git_work_tree();
>       -+	if (worktree)
>       -+		offset = dir_inside_of(startup_info->original_cwd, worktree);
>       ++	if (!worktree)
>       ++		goto no_prevention_needed;
>       ++
>       ++	offset = dir_inside_of(startup_info->original_cwd, worktree);
>        +	if (offset >= 0) {
>        +		/*
>       ++		 * If startup_info->original_cwd == worktree, that is already
>       ++		 * protected and we don't need original_cwd as a secondary
>       ++		 * protection measure.
>       ++		 */
>       ++		if (!*(startup_info->original_cwd + offset))
>       ++			goto no_prevention_needed;
>       ++
>       ++		/*
>        +		 * original_cwd was inside worktree; precompose it just as
>        +		 * we do prefix so that built up paths will match
>        +		 */
>       @@ setup.c: void setup_work_tree(void)
>        +			precompose_string_if_needed(startup_info->original_cwd
>        +						    + offset);
>        +	}
>       ++	return;
>       ++
>       ++no_prevention_needed:
>       ++	free((char*)startup_info->original_cwd);
>       ++	startup_info->original_cwd = NULL;
>        +}
>        +
>         static int read_worktree_config(const char *var, const char *value, void *vdata)
>    3:  e74975e83cc !  3:  68ae90546fe unpack-trees: refuse to remove startup_info->original_cwd
>       @@ Commit message
>            Signed-off-by: Elijah Newren <newren@gmail.com>
>        
>         ## t/t2501-cwd-empty.sh ##
>       -@@ t/t2501-cwd-empty.sh: test_expect_failure 'checkout does not clean cwd incidentally' '
>       - 	test_path_is_dir foo
>       +@@ t/t2501-cwd-empty.sh: test_expect_success 'checkout does not clean cwd incidentally' '
>         '
>         
>       --test_expect_failure 'checkout fails if cwd needs to be removed' '
>       -+test_expect_success 'checkout fails if cwd needs to be removed' '
>       - 	git checkout foo/bar/baz &&
>       - 	test_when_finished "git clean -fdx" &&
>       + test_expect_success 'checkout fails if cwd needs to be removed' '
>       +-	test_required_dir_removal failure git checkout fd_conflict
>       ++	test_required_dir_removal success git checkout fd_conflict
>       + '
>       +
>       + test_expect_success 'reset --hard does not clean cwd incidentally' '
>       +@@ t/t2501-cwd-empty.sh: test_expect_success 'merge fails if cwd needs to be removed; recursive friendly'
>       + 	(
>       + 		cd dirORfile &&
>       +
>       +-		# We would rather this failed, but we test for existing
>       +-		# rather than desired behavior
>       +-		git merge fd_conflict 2>../error
>       ++		test_must_fail git merge fd_conflict 2>../error
>       + 	) &&
>         
>       -@@ t/t2501-cwd-empty.sh: test_expect_failure 'merge does not remove cwd incidentally' '
>       - 	test_path_is_dir subdir
>       +-	## Here is the behavior we would rather have:
>       +-	#test_path_is_dir dirORfile &&
>       +-	#grep "Refusing to remove the current working directory" error
>       +-	## But instead we test for existing behavior
>       +-	test_path_is_file dirORfile &&
>       +-	test_must_be_empty error
>       ++	test_path_is_dir dirORfile &&
>       ++	grep "Refusing to remove the current working directory" error
>         '
>         
>       --test_expect_failure 'merge fails if cwd needs to be removed' '
>       -+test_expect_success 'merge fails if cwd needs to be removed' '
>       - 	git checkout foo/bar/baz &&
>       - 	test_when_finished "git clean -fdx" &&
>       + GIT_TEST_MERGE_ALGORITHM=ort
>         
>       -@@ t/t2501-cwd-empty.sh: test_expect_failure 'cherry-pick does not remove cwd incidentally' '
>       - 	test_path_is_dir subdir
>       + test_expect_success 'merge fails if cwd needs to be removed' '
>       +-	test_required_dir_removal failure git merge fd_conflict
>       ++	test_required_dir_removal success git merge fd_conflict
>         '
>         
>       --test_expect_failure 'cherry-pick fails if cwd needs to be removed' '
>       -+test_expect_success 'cherry-pick fails if cwd needs to be removed' '
>       - 	git checkout foo/bar/baz &&
>       - 	test_when_finished "git clean -fdx" &&
>       + test_expect_success 'cherry-pick does not clean cwd incidentally' '
>       +@@ t/t2501-cwd-empty.sh: test_expect_success 'cherry-pick does not clean cwd incidentally' '
>       + '
>         
>       -@@ t/t2501-cwd-empty.sh: test_expect_failure 'rebase does not remove cwd incidentally' '
>       - 	test_path_is_dir subdir
>       + test_expect_success 'cherry-pick fails if cwd needs to be removed' '
>       +-	test_required_dir_removal failure git cherry-pick fd_conflict
>       ++	test_required_dir_removal success git cherry-pick fd_conflict
>         '
>         
>       --test_expect_failure 'rebase fails if cwd needs to be removed' '
>       -+test_expect_success 'rebase fails if cwd needs to be removed' '
>       - 	git checkout foo/bar/baz &&
>       - 	test_when_finished "git clean -fdx" &&
>       + test_expect_success 'rebase does not clean cwd incidentally' '
>       +@@ t/t2501-cwd-empty.sh: test_expect_success 'revert does not clean cwd incidentally' '
>       + '
>         
>       -@@ t/t2501-cwd-empty.sh: test_expect_failure 'revert does not remove cwd incidentally' '
>       - 	test_path_is_dir subdir
>       + test_expect_success 'revert fails if cwd needs to be removed' '
>       +-	test_required_dir_removal failure git revert undo_fd_conflict
>       ++	test_required_dir_removal success git revert undo_fd_conflict
>         '
>         
>       --test_expect_failure 'revert fails if cwd needs to be removed' '
>       -+test_expect_success 'revert fails if cwd needs to be removed' '
>       - 	git checkout fd_conflict &&
>       - 	git revert HEAD &&
>       - 	test_when_finished "git clean -fdx" &&
>       + test_expect_success 'rm does not clean cwd incidentally' '
>        
>         ## unpack-trees.c ##
>        @@ unpack-trees.c: static const char *unpack_plumbing_errors[NB_UNPACK_TREES_WARNING_TYPES] = {
>    4:  e06806e3a32 !  4:  1bb8905900c unpack-trees: add special cwd handling
>       @@ Commit message
>            Signed-off-by: Elijah Newren <newren@gmail.com>
>        
>         ## t/t2501-cwd-empty.sh ##
>       -@@ t/t2501-cwd-empty.sh: test_expect_failure 'reset --hard does not clean cwd incidentally' '
>       - 	test_path_is_dir foo
>       +@@ t/t2501-cwd-empty.sh: test_expect_success 'reset --hard does not clean cwd incidentally' '
>         '
>         
>       --test_expect_failure 'reset --hard fails if cwd needs to be removed' '
>       -+test_expect_success 'reset --hard fails if cwd needs to be removed' '
>       - 	git checkout foo/bar/baz &&
>       - 	test_when_finished "git clean -fdx" &&
>       + test_expect_success 'reset --hard fails if cwd needs to be removed' '
>       +-	test_required_dir_removal failure git reset --hard fd_conflict
>       ++	test_required_dir_removal success git reset --hard fd_conflict
>       + '
>         
>       + test_expect_success 'merge does not clean cwd incidentally' '
>        
>         ## unpack-trees.c ##
>        @@ unpack-trees.c: static int verify_absent_1(const struct cache_entry *ce,
>    5:  46728f74ea1 !  5:  8a33d74e7cf symlinks: do not include startup_info->original_cwd in dir removal
>       @@ symlinks.c: void schedule_dir_for_removal(const char *name, int len)
>         				   &previous_slash);
>        
>         ## t/t2501-cwd-empty.sh ##
>       -@@ t/t2501-cwd-empty.sh: test_expect_success setup '
>       - 	git commit -m dirORfile
>       - '
>       +@@ t/t2501-cwd-empty.sh: test_required_dir_removal () {
>       + }
>         
>       --test_expect_failure 'checkout does not clean cwd incidentally' '
>       -+test_expect_success 'checkout does not clean cwd incidentally' '
>       - 	git checkout foo/bar/baz &&
>       - 	test_path_is_dir foo/bar &&
>       + test_expect_success 'checkout does not clean cwd incidentally' '
>       +-	test_incidental_dir_removal failure git checkout init
>       ++	test_incidental_dir_removal success git checkout init
>       + '
>         
>       + test_expect_success 'checkout fails if cwd needs to be removed' '
>        @@ t/t2501-cwd-empty.sh: test_expect_success 'checkout fails if cwd needs to be removed' '
>       - 	test_path_is_dir dirORfile
>         '
>         
>       --test_expect_failure 'reset --hard does not clean cwd incidentally' '
>       -+test_expect_success 'reset --hard does not clean cwd incidentally' '
>       - 	git checkout foo/bar/baz &&
>       - 	test_path_is_dir foo/bar &&
>       + test_expect_success 'reset --hard does not clean cwd incidentally' '
>       +-	test_incidental_dir_removal failure git reset --hard init
>       ++	test_incidental_dir_removal success git reset --hard init
>       + '
>         
>       + test_expect_success 'reset --hard fails if cwd needs to be removed' '
>        @@ t/t2501-cwd-empty.sh: test_expect_success 'reset --hard fails if cwd needs to be removed' '
>       - 	test_path_is_dir dirORfile
>         '
>         
>       --test_expect_failure 'merge does not remove cwd incidentally' '
>       -+test_expect_success 'merge does not remove cwd incidentally' '
>       - 	git checkout foo/bar/baz &&
>       - 	test_when_finished "git clean -fdx" &&
>       + test_expect_success 'merge does not clean cwd incidentally' '
>       +-	test_incidental_dir_removal failure git merge reverted
>       ++	test_incidental_dir_removal success git merge reverted
>       + '
>         
>       + # This file uses some simple merges where
>        @@ t/t2501-cwd-empty.sh: test_expect_success 'merge fails if cwd needs to be removed' '
>       - 	test_path_is_dir dirORfile
>         '
>         
>       --test_expect_failure 'cherry-pick does not remove cwd incidentally' '
>       -+test_expect_success 'cherry-pick does not remove cwd incidentally' '
>       - 	git checkout foo/bar/baz &&
>       - 	test_when_finished "git clean -fdx" &&
>       -
>       -@@ t/t2501-cwd-empty.sh: test_expect_success 'cherry-pick fails if cwd needs to be removed' '
>       - 	test_path_is_dir dirORfile
>       + test_expect_success 'cherry-pick does not clean cwd incidentally' '
>       +-	test_incidental_dir_removal failure git cherry-pick reverted
>       ++	test_incidental_dir_removal success git cherry-pick reverted
>         '
>         
>       --test_expect_failure 'rebase does not remove cwd incidentally' '
>       -+test_expect_success 'rebase does not remove cwd incidentally' '
>       - 	git checkout foo/bar/baz &&
>       - 	test_when_finished "git clean -fdx" &&
>       -
>       + test_expect_success 'cherry-pick fails if cwd needs to be removed' '
>        @@ t/t2501-cwd-empty.sh: test_expect_success 'rebase fails if cwd needs to be removed' '
>       - 	test_path_is_dir dirORfile
>         '
>         
>       --test_expect_failure 'revert does not remove cwd incidentally' '
>       -+test_expect_success 'revert does not remove cwd incidentally' '
>       - 	git checkout foo/bar/baz &&
>       - 	test_when_finished "git clean -fdx" &&
>       + test_expect_success 'revert does not clean cwd incidentally' '
>       +-	test_incidental_dir_removal failure git revert HEAD
>       ++	test_incidental_dir_removal success git revert HEAD
>       + '
>         
>       + test_expect_success 'revert fails if cwd needs to be removed' '
>    6:  01ce9444dae !  6:  11e4ec881bb clean: do not attempt to remove startup_info->original_cwd
>       @@ builtin/clean.c: static int remove_dirs(struct strbuf *path, const char *prefix,
>        +		strbuf_realpath(&realpath, path->buf, 1);
>        +
>        +		/*
>       -+		 * path and realpath are absolute; for comparison, we want
>       -+		 * startup_info->original_cwd to be an absolute path too.  We
>       -+		 * can use strbuf_realpath for this.  Also, if original_cwd
>       -+		 * started out as the empty string, then it corresponded to
>       -+		 * the top of the worktree, which is protected by other means
>       -+		 * so we just leave it blank.
>       ++		 * path and realpath are absolute; for comparison, we would
>       ++		 * like to transform startup_info->original_cwd to an absolute
>       ++		 * path too.
>        +		 */
>       -+		 if (*startup_info->original_cwd)
>       ++		 if (startup_info->original_cwd)
>        +			 strbuf_realpath(&real_ocwd,
>        +					 startup_info->original_cwd, 1);
>        +
>       @@ builtin/clean.c: static int remove_dirs(struct strbuf *path, const char *prefix,
>         	return ret;
>        
>         ## t/t2501-cwd-empty.sh ##
>       -@@ t/t2501-cwd-empty.sh: test_expect_failure 'apply does not remove cwd incidentally' '
>       - 	test_path_is_dir subdir
>       - '
>       +@@ t/t2501-cwd-empty.sh: test_incidental_untracked_dir_removal () {
>       + }
>         
>       --test_expect_failure 'clean does not remove cwd incidentally' '
>       -+test_expect_success 'clean does not remove cwd incidentally' '
>       - 	git checkout foo/bar/baz &&
>       - 	test_when_finished "git clean -fdx" &&
>       + test_expect_success 'clean does not remove cwd incidentally' '
>       +-	test_incidental_untracked_dir_removal failure \
>       +-		git -C .. clean -fd -e warnings . >warnings
>       ++	test_incidental_untracked_dir_removal success \
>       ++		git -C .. clean -fd -e warnings . >warnings &&
>       ++	grep "Refusing to remove current working directory" warnings
>       + '
>         
>       + test_expect_success 'stash does not remove cwd incidentally' '
>    -:  ----------- >  7:  39b1f3a225e rebase: do not attempt to remove startup_info->original_cwd
>    7:  edec0894ca2 !  8:  0110462a19c stash: do not attempt to remove startup_info->original_cwd
>       @@ Metadata
>         ## Commit message ##
>            stash: do not attempt to remove startup_info->original_cwd
>        
>       +    Since stash spawns a `clean` subprocess, make sure we run that from the
>       +    startup_info->original_cwd directory, so that the `clean` processs knows
>       +    to protect that directory.  Also, since the `clean` command might no
>       +    longer run from the toplevel, pass the ':/' magic pathspec to ensure we
>       +    still clean from the toplevel.
>       +
>            Signed-off-by: Elijah Newren <newren@gmail.com>
>        
>         ## builtin/stash.c ##
>       @@ builtin/stash.c: static int do_push_stash(const struct pathspec *ps, const char
>         
>         			cp.git_cmd = 1;
>        +			if (startup_info->original_cwd &&
>       -+			    *startup_info->original_cwd &&
>        +			    !is_absolute_path(startup_info->original_cwd))
>        +				cp.dir = startup_info->original_cwd;
>         			strvec_pushl(&cp.args, "clean", "--force",
>       @@ builtin/stash.c: static int do_push_stash(const struct pathspec *ps, const char
>        
>         ## t/t2501-cwd-empty.sh ##
>        @@ t/t2501-cwd-empty.sh: test_expect_success 'clean does not remove cwd incidentally' '
>       - 	test_path_is_dir untracked
>         '
>         
>       --test_expect_failure 'stash does not remove cwd incidentally' '
>       -+test_expect_success 'stash does not remove cwd incidentally' '
>       - 	git checkout foo/bar/baz &&
>       - 	test_when_finished "git clean -fdx" &&
>       + test_expect_success 'stash does not remove cwd incidentally' '
>       +-	test_incidental_untracked_dir_removal failure \
>       ++	test_incidental_untracked_dir_removal success \
>       + 		git stash --include-untracked
>       + '
>         
>    8:  1815f18592b !  9:  2c73a09a2e8 dir: avoid incidentally removing the original_cwd in remove_path()
>       @@ dir.h: int get_sparse_checkout_patterns(struct pattern_list *pl);
>        
>         ## t/t2501-cwd-empty.sh ##
>        @@ t/t2501-cwd-empty.sh: test_expect_success 'revert fails if cwd needs to be removed' '
>       - 	test_path_is_dir dirORfile
>         '
>         
>       --test_expect_failure 'rm does not remove cwd incidentally' '
>       -+test_expect_success 'rm does not remove cwd incidentally' '
>       - 	test_when_finished "git reset --hard" &&
>       - 	git checkout foo/bar/baz &&
>       + test_expect_success 'rm does not clean cwd incidentally' '
>       +-	test_incidental_dir_removal failure git rm bar/baz.t
>       ++	test_incidental_dir_removal success git rm bar/baz.t
>       + '
>         
>       -@@ t/t2501-cwd-empty.sh: test_expect_failure 'rm does not remove cwd incidentally' '
>       - 	test_path_is_dir foo
>       + test_expect_success 'apply does not remove cwd incidentally' '
>       + 	git diff HEAD HEAD~1 >patch &&
>       +-	test_incidental_dir_removal failure git apply ../patch
>       ++	test_incidental_dir_removal success git apply ../patch
>         '
>         
>       --test_expect_failure 'apply does not remove cwd incidentally' '
>       -+test_expect_success 'apply does not remove cwd incidentally' '
>       - 	test_when_finished "git reset --hard" &&
>       - 	git checkout foo/bar/baz &&
>       + test_incidental_untracked_dir_removal () {
>       +@@ t/t2501-cwd-empty.sh: test_expect_success '`rm -rf dir` even with only tracked files will remove somet
>       + 	) &&
>       +
>       + 	test_path_is_missing a/b/c/tracked &&
>       +-	## We would prefer if a/b was still present, though empty, since it
>       +-	## was the current working directory
>       +-	#test_path_is_dir a/b
>       +-	## But the current behavior is that it not only deletes the directory
>       +-	## a/b as requested, but also goes and deletes a
>       +-	test_path_is_missing a
>       ++	test_path_is_missing a/b/c &&
>       ++	test_path_is_dir a/b
>       + '
>         
>       + test_expect_success 'git version continues working from a deleted dir' '
>    9:  adaad7aeaac ! 10:  d4e50b4053d dir: new flag to remove_dir_recurse() to spare the original_cwd
>       @@ dir.h: int get_sparse_checkout_patterns(struct pattern_list *pl);
>          * of the above REMOVE_DIR_* constants. Return 0 on success.
>        
>         ## t/t2501-cwd-empty.sh ##
>       -@@ t/t2501-cwd-empty.sh: test_expect_success 'stash does not remove cwd incidentally' '
>       - 	test_path_is_dir untracked
>       - '
>       +@@ t/t2501-cwd-empty.sh: test_submodule_removal () {
>       + 	test_status=
>       + 	test $path_status = dir && test_status=test_must_fail
>       +
>       +-	# Actually, while path_status == dir && test_status=test_must_fail
>       +-	# reflect our desired behavior, current behavior is:
>       +-	path_status=missing
>       +-	test_status=
>       +-
>       + 	test_when_finished "git reset --hard HEAD~1" &&
>       + 	test_when_finished "rm -rf .git/modules/my_submodule" &&
>         
>       -+test_expect_success 'rm -r leaves submodule if cwd inside' '
>       -+	test_when_finished "git reset --hard HEAD~1" &&
>       -+	test_when_finished "rm -rf .git/modules/my_submodule" &&
>       -+
>       -+	git checkout foo/bar/baz &&
>       -+
>       -+	git init my_submodule &&
>       -+	touch my_submodule/file &&
>       -+	git -C my_submodule add file &&
>       -+	git -C my_submodule commit -m "initial commit" &&
>       -+	git submodule add ./my_submodule &&
>       -+	git commit -m "Add the submodule" &&
>       -+
>       -+	(
>       -+		cd my_submodule &&
>       -+		test_must_fail git --git-dir=../.git --work-tree=.. rm -r ../my_submodule/
>       -+	) &&
>       -+
>       -+	test_path_is_dir my_submodule
>       -+'
>       -+
>       -+test_expect_success 'rm -rf removes submodule even if cwd inside' '
>       -+	test_when_finished "git reset --hard HEAD~1" &&
>       -+	test_when_finished "rm -rf .git/modules/my_submodule" &&
>       -+
>       -+	git checkout foo/bar/baz &&
>       -+
>       -+	git init my_submodule &&
>       -+	touch my_submodule/file &&
>       -+	git -C my_submodule add file &&
>       -+	git -C my_submodule commit -m "initial commit" &&
>       -+	git submodule add ./my_submodule &&
>       -+	git commit -m "Add the submodule" &&
>       -+
>       -+	(
>       -+		cd my_submodule &&
>       -+		git --git-dir=../.git --work-tree=.. rm -rf ../my_submodule/
>       -+	) &&
>       -+
>       -+	test_path_is_missing my_submodule
>       -+'
>       -+
>       - test_done
>    -:  ----------- > 11:  7eb6281be4b t2501: simplify the tests since we can now assume desired behavior
> 

Re: [PATCH v3 00/11] Avoid removing the current working directory, even if it becomes empty

[Elijah Newren]

Hi Phillip,

On Tue, Nov 30, 2021 at 3:04 AM Phillip Wood <phillip.wood123@gmail.com> wrote:
>
> Hi Elijah
>
> On 26/11/2021 22:40, Elijah Newren via GitGitGadget wrote:
> > Traditionally, if folks run git commands such as checkout or rebase from a
> > subdirectory, that git command could remove their current working directory
> > and result in subsequent git and non-git commands either getting confused or
> > printing messages that confuse the user (e.g. "fatal: Unable to read current
> > working directory: No such file or directory"). Many commands either
> > silently avoid removing directories that are not empty (i.e. those that have
> > untracked or modified files in them)[1], or show an error and abort,
> > depending on which is more appropriate for the command in question. With
> > this series, we augment the reasons to avoid removing directories to include
> > not just has-untracked-or-modified-files, but also to avoid removing the
> > original_cwd as well.
> >
> > Peff and Junio provided some good pros/cons, if it helps:
> >
> >   * Pros: Peff (original suggester of the idea)[2], and Junio[3]
> >   * Cons: Peff [2, again -- see the "P.S."], and Junio[4]
> >
> > [1] well, with a few exceptions; see
> > https://lore.kernel.org/git/pull.1036.v3.git.1632760428.gitgitgadget@gmail.com/
> > [2] https://lore.kernel.org/git/YS8eEtwQvF7TaLCb@coredump.intra.peff.net/
> > [3] https://lore.kernel.org/git/xmqqo86elyht.fsf@gitster.g/ [4]
> > https://lore.kernel.org/git/xmqqo8691gr8.fsf@gitster.g/
>
>
> Thanks for working on this, I'm sorry I haven't had time to take a
> proper look at it but I think it is a good idea. I did notice that
> you're using ":/" in patch 8 and wondered what happens if some runs 'git
> --literal-pathspecs stash' but I haven't looked properly.

Interesting point.  The --literal-pathspecs passed to stash is not
passed along to the git clean subprocess, and we only use the ":/"
pathspec with this subprocess.  This means whereas before we'd always
run
    git clean --force --quiet -d [-x]
from the toplevel, now we run
    git clean --force --quiet -d :/ [-x]
from the relevant subdirectory.  So, I don't think I've introduced any bug here.

(Now, the fact that we use a git clean subprocess instead of just
calling the appropriate library functions is IMO a
design/implementation bug, but a longstanding one.)

[PATCH v5 00/11] Avoid removing the current working directory, even if it becomes empty

[Elijah Newren via GitGitGadget]

Traditionally, if folks run git commands such as checkout or rebase from a
subdirectory, that git command could remove their current working directory
and result in subsequent git and non-git commands either getting confused or
printing messages that confuse the user (e.g. "fatal: Unable to read current
working directory: No such file or directory"). Many commands either
silently avoid removing directories that are not empty (i.e. those that have
untracked or modified files in them)[1], or show an error and abort,
depending on which is more appropriate for the command in question. With
this series, we augment the reasons to avoid removing directories to include
not just has-untracked-or-modified-files, but also to avoid removing the
original_cwd as well.

Peff and Junio provided some good pros/cons, if it helps:

 * Pros: Peff (original suggester of the idea)[2], and Junio[3]
 * Cons: Peff [2, again -- see the "P.S."], and Junio[4]

[1] well, with a few exceptions; see
https://lore.kernel.org/git/pull.1036.v3.git.1632760428.gitgitgadget@gmail.com/
[2] https://lore.kernel.org/git/YS8eEtwQvF7TaLCb@coredump.intra.peff.net/
[3] https://lore.kernel.org/git/xmqqo86elyht.fsf@gitster.g/ [4]
https://lore.kernel.org/git/xmqqo8691gr8.fsf@gitster.g/

Changes since v4:

 * actually fix bashism

Changes since v3:

 * fixed one codepath from v2 so that the series really is only about the
   working tree
 * used test-tool getcwd instead of pwd -P as suggested by Ævar for some
   less common platforms
 * fixed bashism
 * check for clean index/worktree after verifying that expected-to-abort
   codepaths do abort, to make it clearer that we expect an early abort
 * remove a leftover (and confusing) is_absolute_dir() check in sequencer
   and stash from an earlier round of the series

Changes since v2:

 * the series is now only about the working tree. So if the original cwd is
   outside the worktree (or we're in a bare repo), then the new code is a
   no-op.
 * fixed ugly early die() possibility (uses strbuf_getcwd() instead of
   xgetcwd())
 * modified the initial tests to show both expected and desired behavior.
   subsequent patches fix the tests. One new patch added at the end which
   simplifies the tests to only check for desired behavior.
 * NULLify startup_info->original_cwd when it matches the toplevel worktree;
   that is already protected and we don't need secondary protection for it.
   This simplified some other codepaths so we don't have to check for
   startup_info->original_cwd == "".
 * clarified some commit messages

Changes since v1:

 * clarified multiple commit messages
 * renamed the_cwd to startup_info->original_cwd to make it clearer that
   it's our parent process'es cwd that really matters, which we inherited at
   program startup. Also pulls it out of the global namespace.
 * Normalize the path for startup_info->original_cwd, and ensure that it's
   actually the original cwd even if -C is passed to git.
 * small code cleanups suggested by René and Ævar
 * split the final patch (which got the most comments) into two -- one for
   each function being modified. Also, add a bunch more history to the first
   of the two resulting commit messages
 * no longer has a content conflict with so/stash-staged
 * add another value for the flags parameter that remove_dir_recursively()
   takes so that it can opt into either the old or the new behavior. Use
   that for the one special corner case I could find where it matters, and
   add a few tests around it to highlight the utility of the flag.

Elijah Newren (11):
  t2501: add various tests for removing the current working directory
  setup: introduce startup_info->original_cwd
  unpack-trees: refuse to remove startup_info->original_cwd
  unpack-trees: add special cwd handling
  symlinks: do not include startup_info->original_cwd in dir removal
  clean: do not attempt to remove startup_info->original_cwd
  rebase: do not attempt to remove startup_info->original_cwd
  stash: do not attempt to remove startup_info->original_cwd
  dir: avoid incidentally removing the original_cwd in remove_path()
  dir: new flag to remove_dir_recurse() to spare the original_cwd
  t2501: simplify the tests since we can now assume desired behavior

 builtin/clean.c      |  44 +++++--
 builtin/rm.c         |   3 +-
 builtin/stash.c      |   4 +-
 cache.h              |   2 +
 common-main.c        |   4 +
 dir.c                |  15 ++-
 dir.h                |   9 +-
 sequencer.c          |   2 +
 setup.c              |  65 ++++++++++
 symlinks.c           |   8 +-
 t/t2501-cwd-empty.sh | 277 +++++++++++++++++++++++++++++++++++++++++++
 unpack-trees.c       |  30 ++++-
 unpack-trees.h       |   1 +
 13 files changed, 442 insertions(+), 22 deletions(-)
 create mode 100755 t/t2501-cwd-empty.sh


base-commit: 88d915a634b449147855041d44875322de2b286d
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1140%2Fnewren%2Fcwd_removal-v5
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1140/newren/cwd_removal-v5
Pull-Request: https://github.com/git/git/pull/1140

Range-diff vs v4:

  1:  a45b3f08802 !  1:  7c72e888d2e t2501: add various tests for removing the current working directory
     @@ t/t2501-cwd-empty.sh (new)
      +
      +		# Although we want pwd & git status to pass, test for existing
      +		# rather than desired behavior.
     -+		if test "$works" == "success"
     ++		if test "$works" = "success"
      +		then
      +			test-tool getcwd &&
      +			git status --porcelain
     @@ t/t2501-cwd-empty.sh (new)
      +
      +	# Although we want dir to be present, test for existing rather
      +	# than desired behavior.
     -+	if test "$works" == "success"
     ++	if test "$works" = "success"
      +	then
      +		test_path_is_dir foo
      +	else
     @@ t/t2501-cwd-empty.sh (new)
      +		# was an untracked file there), and for the index and worktree
      +		# to be left clean with pwd and git status working afterwards.
      +		# But test for existing rather than desired behavior.
     -+		if test "$works" == "success"
     ++		if test "$works" = "success"
      +		then
      +			test_must_fail "$@" 2>../error &&
      +			grep "Refusing to remove.*current working directory" ../error &&
     @@ t/t2501-cwd-empty.sh (new)
      +
      +	# Although we want dirORfile to be present, test for existing rather
      +	# than desired behavior.
     -+	if test "$works" == "success"
     ++	if test "$works" = "success"
      +	then
      +		test_path_is_dir dirORfile
      +	else
     @@ t/t2501-cwd-empty.sh (new)
      +
      +		# Although we want pwd & git status to pass, test for existing
      +		# rather than desired behavior.
     -+		if test "$works" == "success"
     ++		if test "$works" = "success"
      +		then
      +			test-tool getcwd &&
      +			git status --porcelain
     @@ t/t2501-cwd-empty.sh (new)
      +
      +	# Although we want dir to be present, test for existing rather
      +	# than desired behavior.
     -+	if test "$works" == "success"
     ++	if test "$works" = "success"
      +	then
      +		test_path_is_dir untracked
      +	else
     @@ t/t2501-cwd-empty.sh (new)
      +	test_status=
      +	test "$path_status" = dir && test_status=test_must_fail
      +
     -+	# Actually, while path_status == dir && test_status=test_must_fail
     ++	# Actually, while path_status=dir && test_status=test_must_fail
      +	# reflect our desired behavior, current behavior is:
      +	path_status=missing
      +	test_status=
  2:  ca9f632bd11 =  2:  37f333b2024 setup: introduce startup_info->original_cwd
  3:  41a82eff41e =  3:  b611c73bd15 unpack-trees: refuse to remove startup_info->original_cwd
  4:  2e2ea02f97b =  4:  706415a4547 unpack-trees: add special cwd handling
  5:  f444a541da4 =  5:  66ef6b4d943 symlinks: do not include startup_info->original_cwd in dir removal
  6:  1990e36bb41 =  6:  54d059c683e clean: do not attempt to remove startup_info->original_cwd
  7:  1035ee7f9ce =  7:  f8efb7446c3 rebase: do not attempt to remove startup_info->original_cwd
  8:  a2be40a22d1 =  8:  2855ed45112 stash: do not attempt to remove startup_info->original_cwd
  9:  834031be9e0 =  9:  23dfc3e399d dir: avoid incidentally removing the original_cwd in remove_path()
 10:  d5750fcb6d5 ! 10:  fe47c0f0c17 dir: new flag to remove_dir_recurse() to spare the original_cwd
     @@ t/t2501-cwd-empty.sh: test_submodule_removal () {
       	test_status=
       	test "$path_status" = dir && test_status=test_must_fail
       
     --	# Actually, while path_status == dir && test_status=test_must_fail
     +-	# Actually, while path_status=dir && test_status=test_must_fail
      -	# reflect our desired behavior, current behavior is:
      -	path_status=missing
      -	test_status=
 11:  21ff99a767c ! 11:  431dd651a7e t2501: simplify the tests since we can now assume desired behavior
     @@ t/t2501-cwd-empty.sh: test_incidental_dir_removal () {
       
      -		# Although we want pwd & git status to pass, test for existing
      -		# rather than desired behavior.
     --		if test "$works" == "success"
     +-		if test "$works" = "success"
      -		then
      -			test-tool getcwd &&
      -			git status --porcelain
     @@ t/t2501-cwd-empty.sh: test_incidental_dir_removal () {
       
      -	# Although we want dir to be present, test for existing rather
      -	# than desired behavior.
     --	if test "$works" == "success"
     +-	if test "$works" = "success"
      -	then
      -		test_path_is_dir foo
      -	else
     @@ t/t2501-cwd-empty.sh: test_incidental_dir_removal () {
      -		# was an untracked file there), and for the index and worktree
      -		# to be left clean with pwd and git status working afterwards.
      -		# But test for existing rather than desired behavior.
     --		if test "$works" == "success"
     +-		if test "$works" = "success"
      -		then
      -			test_must_fail "$@" 2>../error &&
      -			grep "Refusing to remove.*current working directory" ../error &&
     @@ t/t2501-cwd-empty.sh: test_incidental_dir_removal () {
       
      -	# Although we want dirORfile to be present, test for existing rather
      -	# than desired behavior.
     --	if test "$works" == "success"
     +-	if test "$works" = "success"
      -	then
      -		test_path_is_dir dirORfile
      -	else
     @@ t/t2501-cwd-empty.sh: test_incidental_untracked_dir_removal () {
       
      -		# Although we want pwd & git status to pass, test for existing
      -		# rather than desired behavior.
     --		if test "$works" == "success"
     +-		if test "$works" = "success"
      -		then
      -			test-tool getcwd &&
      -			git status --porcelain
     @@ t/t2501-cwd-empty.sh: test_incidental_untracked_dir_removal () {
       
      -	# Although we want dir to be present, test for existing rather
      -	# than desired behavior.
     --	if test "$works" == "success"
     +-	if test "$works" = "success"
      -	then
      -		test_path_is_dir untracked
      -	else

-- 
gitgitgadget

[PATCH v5 01/11] t2501: add various tests for removing the current working directory

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Numerous commands will remove directories left empty as a "convenience"
after removing files within them.  That is normally fine, but removing
the current working directory can be rather inconvenient since it can
cause confusion for the user when they run subsequent commands.  For
example, after one git process has removed the current working
directory, git status/log/diff will all abort with the message:

    fatal: Unable to read current working directory: No such file or directory

We also have code paths that, when a file needs to be placed where a
directory is (due to e.g. checkout, merge, reset, whatever), will check
if this is okay and error out if not.  These rules include:
  * all tracked files under that directory are intended to be removed by
    the operation
  * none of the tracked files under that directory have uncommitted
    modification
  * there are no untracked files under that directory
However, if we end up remove the current working directory, we can cause
user confusion when they run subsequent commands, so we would prefer if
there was a fourth rule added to this list: avoid removing the current
working directory.

Since there are several code paths that can result in the current
working directory being removed, add several tests of various different
codepaths.  To make it clearer what the difference between the current
behavior and the behavior at the end of the series, code both of them
into the tests and have the appropriate behavior be selected by a flag.
Subsequent commits will toggle the flag from current to desired
behavior.

Also add a few tests suggested during the review of earlier rounds of
this patch series.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh | 342 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 342 insertions(+)
 create mode 100755 t/t2501-cwd-empty.sh

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
new file mode 100755
index 00000000000..a05abd18187
--- /dev/null
+++ b/t/t2501-cwd-empty.sh
@@ -0,0 +1,342 @@
+#!/bin/sh
+
+test_description='Test handling of the current working directory becoming empty'
+
+. ./test-lib.sh
+
+test_expect_success setup '
+	test_commit init &&
+
+	git branch fd_conflict &&
+
+	mkdir -p foo/bar &&
+	test_commit foo/bar/baz &&
+
+	git revert HEAD &&
+	git tag reverted &&
+
+	git checkout fd_conflict &&
+	mkdir dirORfile &&
+	test_commit dirORfile/foo &&
+
+	git rm -r dirORfile &&
+	echo not-a-directory >dirORfile &&
+	git add dirORfile &&
+	git commit -m dirORfile &&
+
+	git switch -c df_conflict HEAD~1 &&
+	test_commit random_file &&
+
+	git switch -c undo_fd_conflict fd_conflict &&
+	git revert HEAD
+'
+
+test_incidental_dir_removal () {
+	works=$1 &&
+	shift &&
+
+	test_when_finished "git reset --hard" &&
+
+	git checkout foo/bar/baz^{commit} &&
+	test_path_is_dir foo/bar &&
+
+	(
+		cd foo &&
+		"$@" &&
+
+		# Although we want pwd & git status to pass, test for existing
+		# rather than desired behavior.
+		if test "$works" = "success"
+		then
+			test-tool getcwd &&
+			git status --porcelain
+		else
+			! test-tool getcwd &&
+			test_might_fail git status --porcelain
+		fi
+	) &&
+	test_path_is_missing foo/bar/baz &&
+	test_path_is_missing foo/bar &&
+
+	# Although we want dir to be present, test for existing rather
+	# than desired behavior.
+	if test "$works" = "success"
+	then
+		test_path_is_dir foo
+	else
+		test_path_is_missing foo
+	fi
+}
+
+test_required_dir_removal () {
+	works=$1 &&
+	shift &&
+
+	git checkout df_conflict^{commit} &&
+	test_when_finished "git clean -fdx" &&
+
+	(
+		cd dirORfile &&
+
+		# We'd like for the command to fail (much as it would if there
+		# was an untracked file there), and for the index and worktree
+		# to be left clean with pwd and git status working afterwards.
+		# But test for existing rather than desired behavior.
+		if test "$works" = "success"
+		then
+			test_must_fail "$@" 2>../error &&
+			grep "Refusing to remove.*current working directory" ../error &&
+
+			git diff --exit-code HEAD &&
+
+			test-tool getcwd &&
+			git status --porcelain
+		else
+			"$@" &&
+			! test-tool getcwd &&
+			test_might_fail git status --porcelain
+		fi
+	) &&
+
+	# Although we want dirORfile to be present, test for existing rather
+	# than desired behavior.
+	if test "$works" = "success"
+	then
+		test_path_is_dir dirORfile
+	else
+		test_path_is_file dirORfile
+	fi
+}
+
+test_expect_success 'checkout does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git checkout init
+'
+
+test_expect_success 'checkout fails if cwd needs to be removed' '
+	test_required_dir_removal failure git checkout fd_conflict
+'
+
+test_expect_success 'reset --hard does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git reset --hard init
+'
+
+test_expect_success 'reset --hard fails if cwd needs to be removed' '
+	test_required_dir_removal failure git reset --hard fd_conflict
+'
+
+test_expect_success 'merge does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git merge reverted
+'
+
+# This file uses some simple merges where
+#   Base: 'dirORfile/' exists
+#   Side1: random other file changed
+#   Side2: 'dirORfile/' removed, 'dirORfile' added
+# this should resolve cleanly, but merge-recursive throws merge conflicts
+# because it's dumb.  Add a special test for checking merge-recursive (and
+# merge-ort), then after this just hard require ort for all remaining tests.
+#
+test_expect_success 'merge fails if cwd needs to be removed; recursive friendly' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir dirORfile &&
+	(
+		cd dirORfile &&
+
+		# We would rather this failed, but we test for existing
+		# rather than desired behavior
+		git merge fd_conflict 2>../error
+	) &&
+
+	## Here is the behavior we would rather have:
+	#test_path_is_dir dirORfile &&
+	#grep "Refusing to remove the current working directory" error
+	## But instead we test for existing behavior
+	test_path_is_file dirORfile &&
+	test_must_be_empty error
+'
+
+GIT_TEST_MERGE_ALGORITHM=ort
+
+test_expect_success 'merge fails if cwd needs to be removed' '
+	test_required_dir_removal failure git merge fd_conflict
+'
+
+test_expect_success 'cherry-pick does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git cherry-pick reverted
+'
+
+test_expect_success 'cherry-pick fails if cwd needs to be removed' '
+	test_required_dir_removal failure git cherry-pick fd_conflict
+'
+
+test_expect_success 'rebase does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git rebase reverted
+'
+
+test_expect_success 'rebase fails if cwd needs to be removed' '
+	test_required_dir_removal failure git rebase fd_conflict
+'
+
+test_expect_success 'revert does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git revert HEAD
+'
+
+test_expect_success 'revert fails if cwd needs to be removed' '
+	test_required_dir_removal failure git revert undo_fd_conflict
+'
+
+test_expect_success 'rm does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git rm bar/baz.t
+'
+
+test_expect_success 'apply does not remove cwd incidentally' '
+	git diff HEAD HEAD~1 >patch &&
+	test_incidental_dir_removal failure git apply ../patch
+'
+
+test_incidental_untracked_dir_removal () {
+	works=$1 &&
+	shift &&
+
+	test_when_finished "git reset --hard" &&
+
+	git checkout foo/bar/baz^{commit} &&
+	mkdir -p untracked &&
+	mkdir empty
+	>untracked/random &&
+
+	(
+		cd untracked &&
+		"$@" &&
+
+		# Although we want pwd & git status to pass, test for existing
+		# rather than desired behavior.
+		if test "$works" = "success"
+		then
+			test-tool getcwd &&
+			git status --porcelain
+		else
+			! test-tool getcwd &&
+			test_might_fail git status --porcelain
+		fi
+	) &&
+	test_path_is_missing empty &&
+	test_path_is_missing untracked/random &&
+
+	# Although we want dir to be present, test for existing rather
+	# than desired behavior.
+	if test "$works" = "success"
+	then
+		test_path_is_dir untracked
+	else
+		test_path_is_missing untracked
+	fi
+}
+
+test_expect_success 'clean does not remove cwd incidentally' '
+	test_incidental_untracked_dir_removal failure \
+		git -C .. clean -fd -e warnings . >warnings
+'
+
+test_expect_success 'stash does not remove cwd incidentally' '
+	test_incidental_untracked_dir_removal failure \
+		git stash --include-untracked
+'
+
+test_expect_success '`rm -rf dir` only removes a subset of dir' '
+	test_when_finished "rm -rf a/" &&
+
+	mkdir -p a/b/c &&
+	>a/b/c/untracked &&
+	>a/b/c/tracked &&
+	git add a/b/c/tracked &&
+
+	(
+		cd a/b &&
+		git rm -rf ../b
+	) &&
+
+	test_path_is_dir a/b &&
+	test_path_is_missing a/b/c/tracked &&
+	test_path_is_file a/b/c/untracked
+'
+
+test_expect_success '`rm -rf dir` even with only tracked files will remove something else' '
+	test_when_finished "rm -rf a/" &&
+
+	mkdir -p a/b/c &&
+	>a/b/c/tracked &&
+	git add a/b/c/tracked &&
+
+	(
+		cd a/b &&
+		git rm -rf ../b
+	) &&
+
+	test_path_is_missing a/b/c/tracked &&
+	## We would prefer if a/b was still present, though empty, since it
+	## was the current working directory
+	#test_path_is_dir a/b
+	## But the current behavior is that it not only deletes the directory
+	## a/b as requested, but also goes and deletes a
+	test_path_is_missing a
+'
+
+test_expect_success 'git version continues working from a deleted dir' '
+	mkdir tmp &&
+	(
+		cd tmp &&
+		rm -rf ../tmp &&
+		git version
+	)
+'
+
+test_submodule_removal () {
+	path_status=$1 &&
+	shift &&
+
+	test_status=
+	test "$path_status" = dir && test_status=test_must_fail
+
+	# Actually, while path_status=dir && test_status=test_must_fail
+	# reflect our desired behavior, current behavior is:
+	path_status=missing
+	test_status=
+
+	test_when_finished "git reset --hard HEAD~1" &&
+	test_when_finished "rm -rf .git/modules/my_submodule" &&
+
+	git checkout foo/bar/baz &&
+
+	git init my_submodule &&
+	touch my_submodule/file &&
+	git -C my_submodule add file &&
+	git -C my_submodule commit -m "initial commit" &&
+	git submodule add ./my_submodule &&
+	git commit -m "Add the submodule" &&
+
+	(
+		cd my_submodule &&
+		$test_status "$@"
+	) &&
+
+	test_path_is_${path_status} my_submodule
+}
+
+test_expect_success 'rm -r with -C leaves submodule if cwd inside' '
+	test_submodule_removal dir git -C .. rm -r my_submodule/
+'
+
+test_expect_success 'rm -r leaves submodule if cwd inside' '
+	test_submodule_removal dir \
+		git --git-dir=../.git --work-tree=.. rm -r ../my_submodule/
+'
+
+test_expect_success 'rm -rf removes submodule even if cwd inside' '
+	test_submodule_removal missing \
+		git --git-dir=../.git --work-tree=.. rm -rf ../my_submodule/
+'
+
+test_done
-- 
gitgitgadget

[PATCH v5 02/11] setup: introduce startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Removing the current working directory causes all subsequent git
commands run from that directory to get confused and fail with a message
about being unable to read the current working directory:

    $ git status
    fatal: Unable to read current working directory: No such file or directory

Non-git commands likely have similar warnings or even errors, e.g.

    $ bash -c 'echo hello'
    shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    hello

This confuses end users, particularly since the command they get the
error from is not the one that caused the problem; the problem came from
the side-effect of some previous command.

We would like to avoid removing the current working directory of our
parent process; towards this end, introduce a new variable,
startup_info->original_cwd, that tracks the current working directory
that we inherited from our parent process.  For convenience of later
comparisons, we prefer that this new variable store a path relative to
the toplevel working directory (thus much like 'prefix'), except without
the trailing slash.

Subsequent commits will make use of this new variable.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 cache.h       |  2 ++
 common-main.c |  4 ++++
 setup.c       | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 71 insertions(+)

diff --git a/cache.h b/cache.h
index eba12487b99..92e181ea759 100644
--- a/cache.h
+++ b/cache.h
@@ -1834,8 +1834,10 @@ void overlay_tree_on_index(struct index_state *istate,
 struct startup_info {
 	int have_repository;
 	const char *prefix;
+	const char *original_cwd;
 };
 extern struct startup_info *startup_info;
+extern const char *tmp_original_cwd;
 
 /* merge.c */
 struct commit_list;
diff --git a/common-main.c b/common-main.c
index 71e21dd20a3..aa8d5aba5bb 100644
--- a/common-main.c
+++ b/common-main.c
@@ -26,6 +26,7 @@ static void restore_sigpipe_to_default(void)
 int main(int argc, const char **argv)
 {
 	int result;
+	struct strbuf tmp = STRBUF_INIT;
 
 	trace2_initialize_clock();
 
@@ -49,6 +50,9 @@ int main(int argc, const char **argv)
 	trace2_cmd_start(argv);
 	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
 
+	if (!strbuf_getcwd(&tmp))
+		tmp_original_cwd = strbuf_detach(&tmp, NULL);
+
 	result = cmd_main(argc, argv);
 
 	trace2_cmd_exit(result);
diff --git a/setup.c b/setup.c
index 347d7181ae9..af3b8c09abe 100644
--- a/setup.c
+++ b/setup.c
@@ -12,6 +12,7 @@ static int work_tree_config_is_bogus;
 
 static struct startup_info the_startup_info;
 struct startup_info *startup_info = &the_startup_info;
+const char *tmp_original_cwd;
 
 /*
  * The input parameter must contain an absolute path, and it must already be
@@ -432,6 +433,69 @@ void setup_work_tree(void)
 	initialized = 1;
 }
 
+static void setup_original_cwd(void)
+{
+	struct strbuf tmp = STRBUF_INIT;
+	const char *worktree = NULL;
+	int offset = -1;
+
+	if (!tmp_original_cwd)
+		return;
+
+	/*
+	 * startup_info->original_cwd points to the current working
+	 * directory we inherited from our parent process, which is a
+	 * directory we want to avoid removing.
+	 *
+	 * For convience, we would like to have the path relative to the
+	 * worktree instead of an absolute path.
+	 *
+	 * Yes, startup_info->original_cwd is usually the same as 'prefix',
+	 * but differs in two ways:
+	 *   - prefix has a trailing '/'
+	 *   - if the user passes '-C' to git, that modifies the prefix but
+	 *     not startup_info->original_cwd.
+	 */
+
+	/* Normalize the directory */
+	strbuf_realpath(&tmp, tmp_original_cwd, 1);
+	free((char*)tmp_original_cwd);
+	tmp_original_cwd = NULL;
+	startup_info->original_cwd = strbuf_detach(&tmp, NULL);
+
+	/*
+	 * Get our worktree; we only protect the current working directory
+	 * if it's in the worktree.
+	 */
+	worktree = get_git_work_tree();
+	if (!worktree)
+		goto no_prevention_needed;
+
+	offset = dir_inside_of(startup_info->original_cwd, worktree);
+	if (offset >= 0) {
+		/*
+		 * If startup_info->original_cwd == worktree, that is already
+		 * protected and we don't need original_cwd as a secondary
+		 * protection measure.
+		 */
+		if (!*(startup_info->original_cwd + offset))
+			goto no_prevention_needed;
+
+		/*
+		 * original_cwd was inside worktree; precompose it just as
+		 * we do prefix so that built up paths will match
+		 */
+		startup_info->original_cwd = \
+			precompose_string_if_needed(startup_info->original_cwd
+						    + offset);
+		return;
+	}
+
+no_prevention_needed:
+	free((char*)startup_info->original_cwd);
+	startup_info->original_cwd = NULL;
+}
+
 static int read_worktree_config(const char *var, const char *value, void *vdata)
 {
 	struct repository_format *data = vdata;
@@ -1330,6 +1394,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
 		setenv(GIT_PREFIX_ENVIRONMENT, "", 1);
 	}
 
+	setup_original_cwd();
 
 	strbuf_release(&dir);
 	strbuf_release(&gitdir);
-- 
gitgitgadget

[PATCH v5 03/11] unpack-trees: refuse to remove startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

In the past, when a directory needs to be removed to make room for a
file, we have always errored out when that directory contains any
untracked (but not ignored) files.  Add an extra condition on that: also
error out if the directory is the current working directory we inherited
from our parent process.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh | 20 +++++++-------------
 unpack-trees.c       | 17 +++++++++++++----
 unpack-trees.h       |  1 +
 3 files changed, 21 insertions(+), 17 deletions(-)

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index a05abd18187..398908dfc93 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -113,7 +113,7 @@ test_expect_success 'checkout does not clean cwd incidentally' '
 '
 
 test_expect_success 'checkout fails if cwd needs to be removed' '
-	test_required_dir_removal failure git checkout fd_conflict
+	test_required_dir_removal success git checkout fd_conflict
 '
 
 test_expect_success 'reset --hard does not clean cwd incidentally' '
@@ -144,23 +144,17 @@ test_expect_success 'merge fails if cwd needs to be removed; recursive friendly'
 	(
 		cd dirORfile &&
 
-		# We would rather this failed, but we test for existing
-		# rather than desired behavior
-		git merge fd_conflict 2>../error
+		test_must_fail git merge fd_conflict 2>../error
 	) &&
 
-	## Here is the behavior we would rather have:
-	#test_path_is_dir dirORfile &&
-	#grep "Refusing to remove the current working directory" error
-	## But instead we test for existing behavior
-	test_path_is_file dirORfile &&
-	test_must_be_empty error
+	test_path_is_dir dirORfile &&
+	grep "Refusing to remove the current working directory" error
 '
 
 GIT_TEST_MERGE_ALGORITHM=ort
 
 test_expect_success 'merge fails if cwd needs to be removed' '
-	test_required_dir_removal failure git merge fd_conflict
+	test_required_dir_removal success git merge fd_conflict
 '
 
 test_expect_success 'cherry-pick does not clean cwd incidentally' '
@@ -168,7 +162,7 @@ test_expect_success 'cherry-pick does not clean cwd incidentally' '
 '
 
 test_expect_success 'cherry-pick fails if cwd needs to be removed' '
-	test_required_dir_removal failure git cherry-pick fd_conflict
+	test_required_dir_removal success git cherry-pick fd_conflict
 '
 
 test_expect_success 'rebase does not clean cwd incidentally' '
@@ -184,7 +178,7 @@ test_expect_success 'revert does not clean cwd incidentally' '
 '
 
 test_expect_success 'revert fails if cwd needs to be removed' '
-	test_required_dir_removal failure git revert undo_fd_conflict
+	test_required_dir_removal success git revert undo_fd_conflict
 '
 
 test_expect_success 'rm does not clean cwd incidentally' '
diff --git a/unpack-trees.c b/unpack-trees.c
index 89ca95ce90b..6bc16f3a714 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -36,6 +36,9 @@ static const char *unpack_plumbing_errors[NB_UNPACK_TREES_WARNING_TYPES] = {
 	/* ERROR_NOT_UPTODATE_DIR */
 	"Updating '%s' would lose untracked files in it",
 
+	/* ERROR_CWD_IN_THE_WAY */
+	"Refusing to remove '%s' since it is the current working directory.",
+
 	/* ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN */
 	"Untracked working tree file '%s' would be overwritten by merge.",
 
@@ -131,6 +134,9 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 	msgs[ERROR_NOT_UPTODATE_DIR] =
 		_("Updating the following directories would lose untracked files in them:\n%s");
 
+	msgs[ERROR_CWD_IN_THE_WAY] =
+		_("Refusing to remove the current working directory:\n%s");
+
 	if (!strcmp(cmd, "checkout"))
 		msg = advice_enabled(ADVICE_COMMIT_BEFORE_MERGE)
 		      ? _("The following untracked working tree files would be removed by checkout:\n%%s"
@@ -2146,10 +2152,7 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,
 		cnt++;
 	}
 
-	/*
-	 * Then we need to make sure that we do not lose a locally
-	 * present file that is not ignored.
-	 */
+	/* Do not lose a locally present file that is not ignored. */
 	pathbuf = xstrfmt("%.*s/", namelen, ce->name);
 
 	memset(&d, 0, sizeof(d));
@@ -2160,6 +2163,12 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,
 	free(pathbuf);
 	if (i)
 		return add_rejected_path(o, ERROR_NOT_UPTODATE_DIR, ce->name);
+
+	/* Do not lose startup_info->original_cwd */
+	if (startup_info->original_cwd &&
+	    !strcmp(startup_info->original_cwd, ce->name))
+		return add_rejected_path(o, ERROR_CWD_IN_THE_WAY, ce->name);
+
 	return cnt;
 }
 
diff --git a/unpack-trees.h b/unpack-trees.h
index 71ffb7eeb0c..efb9edfbb27 100644
--- a/unpack-trees.h
+++ b/unpack-trees.h
@@ -19,6 +19,7 @@ enum unpack_trees_error_types {
 	ERROR_WOULD_OVERWRITE = 0,
 	ERROR_NOT_UPTODATE_FILE,
 	ERROR_NOT_UPTODATE_DIR,
+	ERROR_CWD_IN_THE_WAY,
 	ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN,
 	ERROR_WOULD_LOSE_UNTRACKED_REMOVED,
 	ERROR_BIND_OVERLAP,
-- 
gitgitgadget

[PATCH v5 04/11] unpack-trees: add special cwd handling

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

When running commands such as `git reset --hard` from a subdirectory, if
that subdirectory is in the way of adding needed files, bail with an
error message.

Note that this change looks kind of like it duplicates the new lines of
code from the previous commit in verify_clean_subdirectory().  However,
when we are preserving untracked files, we would rather any error
messages about untracked files being in the way take precedence over
error messages about a subdirectory that happens to be the_original_cwd
being in the way.  But in the UNPACK_RESET_OVERWRITE_UNTRACKED case,
there is no untracked checking to be done, so we simply add a special
case near the top of verify_absent_1.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh |  2 +-
 unpack-trees.c       | 13 +++++++++++--
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 398908dfc93..5af1fec6fec 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -121,7 +121,7 @@ test_expect_success 'reset --hard does not clean cwd incidentally' '
 '
 
 test_expect_success 'reset --hard fails if cwd needs to be removed' '
-	test_required_dir_removal failure git reset --hard fd_conflict
+	test_required_dir_removal success git reset --hard fd_conflict
 '
 
 test_expect_success 'merge does not clean cwd incidentally' '
diff --git a/unpack-trees.c b/unpack-trees.c
index 6bc16f3a714..5852807d2fb 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -2261,10 +2261,19 @@ static int verify_absent_1(const struct cache_entry *ce,
 	int len;
 	struct stat st;
 
-	if (o->index_only || !o->update ||
-	    o->reset == UNPACK_RESET_OVERWRITE_UNTRACKED)
+	if (o->index_only || !o->update)
 		return 0;
 
+	if (o->reset == UNPACK_RESET_OVERWRITE_UNTRACKED) {
+		/* Avoid nuking startup_info->original_cwd... */
+		if (startup_info->original_cwd &&
+		    !strcmp(startup_info->original_cwd, ce->name))
+			return add_rejected_path(o, ERROR_CWD_IN_THE_WAY,
+						 ce->name);
+		/* ...but nuke anything else. */
+		return 0;
+	}
+
 	len = check_leading_path(ce->name, ce_namelen(ce), 0);
 	if (!len)
 		return 0;
-- 
gitgitgadget

[PATCH v5 05/11] symlinks: do not include startup_info->original_cwd in dir removal

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

symlinks has a pair of schedule_dir_for_removal() and
remove_scheduled_dirs() functions that ensure that directories made
empty by removing other files also themselves get removed.  However, we
want to exclude startup_info->original_cwd and leave it around.  This
avoids the user getting confused by subsequent git commands (and non-git
commands) that would otherwise report confusing messages about being
unable to read the current working directory.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 symlinks.c           |  8 +++++++-
 t/t2501-cwd-empty.sh | 10 +++++-----
 2 files changed, 12 insertions(+), 6 deletions(-)

diff --git a/symlinks.c b/symlinks.c
index 5232d02020c..c667baa949b 100644
--- a/symlinks.c
+++ b/symlinks.c
@@ -279,7 +279,9 @@ static void do_remove_scheduled_dirs(int new_len)
 {
 	while (removal.len > new_len) {
 		removal.buf[removal.len] = '\0';
-		if (rmdir(removal.buf))
+		if ((startup_info->original_cwd &&
+		     !strcmp(removal.buf, startup_info->original_cwd)) ||
+		    rmdir(removal.buf))
 			break;
 		do {
 			removal.len--;
@@ -293,6 +295,10 @@ void schedule_dir_for_removal(const char *name, int len)
 {
 	int match_len, last_slash, i, previous_slash;
 
+	if (startup_info->original_cwd &&
+	    !strcmp(name, startup_info->original_cwd))
+		return;	/* Do not remove the current working directory */
+
 	match_len = last_slash = i =
 		longest_path_match(name, len, removal.buf, removal.len,
 				   &previous_slash);
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 5af1fec6fec..e4502d24d57 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -109,7 +109,7 @@ test_required_dir_removal () {
 }
 
 test_expect_success 'checkout does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git checkout init
+	test_incidental_dir_removal success git checkout init
 '
 
 test_expect_success 'checkout fails if cwd needs to be removed' '
@@ -117,7 +117,7 @@ test_expect_success 'checkout fails if cwd needs to be removed' '
 '
 
 test_expect_success 'reset --hard does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git reset --hard init
+	test_incidental_dir_removal success git reset --hard init
 '
 
 test_expect_success 'reset --hard fails if cwd needs to be removed' '
@@ -125,7 +125,7 @@ test_expect_success 'reset --hard fails if cwd needs to be removed' '
 '
 
 test_expect_success 'merge does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git merge reverted
+	test_incidental_dir_removal success git merge reverted
 '
 
 # This file uses some simple merges where
@@ -158,7 +158,7 @@ test_expect_success 'merge fails if cwd needs to be removed' '
 '
 
 test_expect_success 'cherry-pick does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git cherry-pick reverted
+	test_incidental_dir_removal success git cherry-pick reverted
 '
 
 test_expect_success 'cherry-pick fails if cwd needs to be removed' '
@@ -174,7 +174,7 @@ test_expect_success 'rebase fails if cwd needs to be removed' '
 '
 
 test_expect_success 'revert does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git revert HEAD
+	test_incidental_dir_removal success git revert HEAD
 '
 
 test_expect_success 'revert fails if cwd needs to be removed' '
-- 
gitgitgadget

[PATCH v5 06/11] clean: do not attempt to remove startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/clean.c      | 44 +++++++++++++++++++++++++++++++++++---------
 t/t2501-cwd-empty.sh |  5 +++--
 2 files changed, 38 insertions(+), 11 deletions(-)

diff --git a/builtin/clean.c b/builtin/clean.c
index 98a2860409b..3ff02bbbffe 100644
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@ -36,6 +36,8 @@ static const char *msg_skip_git_dir = N_("Skipping repository %s\n");
 static const char *msg_would_skip_git_dir = N_("Would skip repository %s\n");
 static const char *msg_warn_remove_failed = N_("failed to remove %s");
 static const char *msg_warn_lstat_failed = N_("could not lstat %s\n");
+static const char *msg_skip_cwd = N_("Refusing to remove current working directory\n");
+static const char *msg_would_skip_cwd = N_("Would refuse to remove current working directory\n");
 
 enum color_clean {
 	CLEAN_COLOR_RESET = 0,
@@ -153,6 +155,8 @@ static int remove_dirs(struct strbuf *path, const char *prefix, int force_flag,
 {
 	DIR *dir;
 	struct strbuf quoted = STRBUF_INIT;
+	struct strbuf realpath = STRBUF_INIT;
+	struct strbuf real_ocwd = STRBUF_INIT;
 	struct dirent *e;
 	int res = 0, ret = 0, gone = 1, original_len = path->len, len;
 	struct string_list dels = STRING_LIST_INIT_DUP;
@@ -231,16 +235,36 @@ static int remove_dirs(struct strbuf *path, const char *prefix, int force_flag,
 	strbuf_setlen(path, original_len);
 
 	if (*dir_gone) {
-		res = dry_run ? 0 : rmdir(path->buf);
-		if (!res)
-			*dir_gone = 1;
-		else {
-			int saved_errno = errno;
-			quote_path(path->buf, prefix, &quoted, 0);
-			errno = saved_errno;
-			warning_errno(_(msg_warn_remove_failed), quoted.buf);
+		/*
+		 * Normalize path components in path->buf, e.g. change '\' to
+		 * '/' on Windows.
+		 */
+		strbuf_realpath(&realpath, path->buf, 1);
+
+		/*
+		 * path and realpath are absolute; for comparison, we would
+		 * like to transform startup_info->original_cwd to an absolute
+		 * path too.
+		 */
+		 if (startup_info->original_cwd)
+			 strbuf_realpath(&real_ocwd,
+					 startup_info->original_cwd, 1);
+
+		if (!strbuf_cmp(&realpath, &real_ocwd)) {
+			printf("%s", dry_run ? _(msg_would_skip_cwd) : _(msg_skip_cwd));
 			*dir_gone = 0;
-			ret = 1;
+		} else {
+			res = dry_run ? 0 : rmdir(path->buf);
+			if (!res)
+				*dir_gone = 1;
+			else {
+				int saved_errno = errno;
+				quote_path(path->buf, prefix, &quoted, 0);
+				errno = saved_errno;
+				warning_errno(_(msg_warn_remove_failed), quoted.buf);
+				*dir_gone = 0;
+				ret = 1;
+			}
 		}
 	}
 
@@ -250,6 +274,8 @@ static int remove_dirs(struct strbuf *path, const char *prefix, int force_flag,
 			printf(dry_run ?  _(msg_would_remove) : _(msg_remove), dels.items[i].string);
 	}
 out:
+	strbuf_release(&realpath);
+	strbuf_release(&real_ocwd);
 	strbuf_release(&quoted);
 	string_list_clear(&dels, 0);
 	return ret;
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index e4502d24d57..b1182390ba3 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -230,8 +230,9 @@ test_incidental_untracked_dir_removal () {
 }
 
 test_expect_success 'clean does not remove cwd incidentally' '
-	test_incidental_untracked_dir_removal failure \
-		git -C .. clean -fd -e warnings . >warnings
+	test_incidental_untracked_dir_removal success \
+		git -C .. clean -fd -e warnings . >warnings &&
+	grep "Refusing to remove current working directory" warnings
 '
 
 test_expect_success 'stash does not remove cwd incidentally' '
-- 
gitgitgadget

[PATCH v5 07/11] rebase: do not attempt to remove startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Since rebase spawns a `checkout` subprocess, make sure we run that from
the startup_info->original_cwd directory, so that the checkout process
knows to protect that directory.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 sequencer.c          | 2 ++
 t/t2501-cwd-empty.sh | 4 ++--
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index ea96837cde3..83f257e7fa4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -4228,6 +4228,8 @@ static int run_git_checkout(struct repository *r, struct replay_opts *opts,
 
 	cmd.git_cmd = 1;
 
+	if (startup_info->original_cwd)
+		cmd.dir = startup_info->original_cwd;
 	strvec_push(&cmd.args, "checkout");
 	strvec_push(&cmd.args, commit);
 	strvec_pushf(&cmd.env_array, GIT_REFLOG_ACTION "=%s", action);
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index b1182390ba3..52335a8afe9 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -166,11 +166,11 @@ test_expect_success 'cherry-pick fails if cwd needs to be removed' '
 '
 
 test_expect_success 'rebase does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git rebase reverted
+	test_incidental_dir_removal success git rebase reverted
 '
 
 test_expect_success 'rebase fails if cwd needs to be removed' '
-	test_required_dir_removal failure git rebase fd_conflict
+	test_required_dir_removal success git rebase fd_conflict
 '
 
 test_expect_success 'revert does not clean cwd incidentally' '
-- 
gitgitgadget

[PATCH v5 08/11] stash: do not attempt to remove startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Since stash spawns a `clean` subprocess, make sure we run that from the
startup_info->original_cwd directory, so that the `clean` processs knows
to protect that directory.  Also, since the `clean` command might no
longer run from the toplevel, pass the ':/' magic pathspec to ensure we
still clean from the toplevel.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/stash.c      | 4 +++-
 t/t2501-cwd-empty.sh | 2 +-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/builtin/stash.c b/builtin/stash.c
index a0ccc8654df..de0e432a4ff 100644
--- a/builtin/stash.c
+++ b/builtin/stash.c
@@ -1485,8 +1485,10 @@ static int do_push_stash(const struct pathspec *ps, const char *stash_msg, int q
 			struct child_process cp = CHILD_PROCESS_INIT;
 
 			cp.git_cmd = 1;
+			if (startup_info->original_cwd)
+				cp.dir = startup_info->original_cwd;
 			strvec_pushl(&cp.args, "clean", "--force",
-				     "--quiet", "-d", NULL);
+				     "--quiet", "-d", ":/", NULL);
 			if (include_untracked == INCLUDE_ALL_FILES)
 				strvec_push(&cp.args, "-x");
 			if (run_command(&cp)) {
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 52335a8afe9..be9ef903bd4 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -236,7 +236,7 @@ test_expect_success 'clean does not remove cwd incidentally' '
 '
 
 test_expect_success 'stash does not remove cwd incidentally' '
-	test_incidental_untracked_dir_removal failure \
+	test_incidental_untracked_dir_removal success \
 		git stash --include-untracked
 '
 
-- 
gitgitgadget

[PATCH v5 09/11] dir: avoid incidentally removing the original_cwd in remove_path()

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Modern git often tries to avoid leaving empty directories around when
removing files.  Originally, it did not bother.  This behavior started
with commit 80e21a9ed809 (merge-recursive::removeFile: remove empty
directories, 2005-11-19), stating the reason simply as:

    When the last file in a directory is removed as the result of a
    merge, try to rmdir the now-empty directory.

This was reimplemented in C and renamed to remove_path() in commit
e1b3a2cad7 ("Build-in merge-recursive", 2008-02-07), but was still
internal to merge-recursive.

This trend towards removing leading empty directories continued with
commit d9b814cc97f1 (Add builtin "git rm" command, 2006-05-19), which
stated the reasoning as:

    The other question is what to do with leading directories. The old
    "git rm" script didn't do anything, which is somewhat inconsistent.
    This one will actually clean up directories that have become empty
    as a result of removing the last file, but maybe we want to have a
    flag to decide the behaviour?

remove_path() in dir.c was added in 4a92d1bfb784 (Add remove_path: a
function to remove as much as possible of a path, 2008-09-27), because
it was noted that we had two separate implementations of the same idea
AND both were buggy.  It described the purpose of the function as

    a function to remove as much as possible of a path

Why remove as much as possible?  Well, at the time we probably would
have said something like:

  * removing leading directories makes things feel tidy
  * removing leading directories doesn't hurt anything so long as they
    had no files in them.

But I don't believe those reasons hold when the empty directory happens
to be the current working directory we inherited from our parent
process.  Leaving the parent process in a deleted directory can cause
user confusion when subsequent processes fail: any git command, for
example, will immediately fail with

    fatal: Unable to read current working directory: No such file or directory

Other commands may similarly get confused.  Modify remove_path() so that
the empty leading directories it also deletes does not include the
current working directory we inherited from our parent process.  I have
looked through every caller of remove_path() in the current codebase to
make sure that all should take this change.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 dir.c                |  3 +++
 dir.h                |  6 +++++-
 t/t2501-cwd-empty.sh | 12 ++++--------
 3 files changed, 12 insertions(+), 9 deletions(-)

diff --git a/dir.c b/dir.c
index 94489298f4c..97d6b71c872 100644
--- a/dir.c
+++ b/dir.c
@@ -3327,6 +3327,9 @@ int remove_path(const char *name)
 		slash = dirs + (slash - name);
 		do {
 			*slash = '\0';
+			if (startup_info->original_cwd &&
+			    !strcmp(startup_info->original_cwd, dirs))
+				break;
 		} while (rmdir(dirs) == 0 && (slash = strrchr(dirs, '/')));
 		free(dirs);
 	}
diff --git a/dir.h b/dir.h
index 83f46c0fb4c..d6a5d03bec2 100644
--- a/dir.h
+++ b/dir.h
@@ -504,7 +504,11 @@ int get_sparse_checkout_patterns(struct pattern_list *pl);
  */
 int remove_dir_recursively(struct strbuf *path, int flag);
 
-/* tries to remove the path with empty directories along it, ignores ENOENT */
+/*
+ * Tries to remove the path, along with leading empty directories so long as
+ * those empty directories are not startup_info->original_cwd.  Ignores
+ * ENOENT.
+ */
 int remove_path(const char *path);
 
 int fspathcmp(const char *a, const char *b);
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index be9ef903bd4..ce2efb9d30a 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -182,12 +182,12 @@ test_expect_success 'revert fails if cwd needs to be removed' '
 '
 
 test_expect_success 'rm does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git rm bar/baz.t
+	test_incidental_dir_removal success git rm bar/baz.t
 '
 
 test_expect_success 'apply does not remove cwd incidentally' '
 	git diff HEAD HEAD~1 >patch &&
-	test_incidental_dir_removal failure git apply ../patch
+	test_incidental_dir_removal success git apply ../patch
 '
 
 test_incidental_untracked_dir_removal () {
@@ -271,12 +271,8 @@ test_expect_success '`rm -rf dir` even with only tracked files will remove somet
 	) &&
 
 	test_path_is_missing a/b/c/tracked &&
-	## We would prefer if a/b was still present, though empty, since it
-	## was the current working directory
-	#test_path_is_dir a/b
-	## But the current behavior is that it not only deletes the directory
-	## a/b as requested, but also goes and deletes a
-	test_path_is_missing a
+	test_path_is_missing a/b/c &&
+	test_path_is_dir a/b
 '
 
 test_expect_success 'git version continues working from a deleted dir' '
-- 
gitgitgadget

[PATCH v5 10/11] dir: new flag to remove_dir_recurse() to spare the original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

remove_dir_recurse(), and its non-static wrapper called
remove_dir_recursively(), both take flags for modifying its behavior.
As with the previous commits, we would generally like to protect
the original_cwd, but we want to forced user commands (e.g. 'git rm -rf
...') or other special cases to remove it.  Add a flag for this purpose.
After reading through every caller of remove_dir_recursively() in the
current codebase, there was only one that should be adjusted and that
one only in a very unusual circumstance.  Add a pair of new testcases to
highlight that very specific case involving submodules && --git-dir &&
--work-tree.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/rm.c         |  3 ++-
 dir.c                | 12 +++++++++---
 dir.h                |  3 +++
 t/t2501-cwd-empty.sh |  5 -----
 4 files changed, 14 insertions(+), 9 deletions(-)

diff --git a/builtin/rm.c b/builtin/rm.c
index 3d0967cdc11..b4132e5d8ee 100644
--- a/builtin/rm.c
+++ b/builtin/rm.c
@@ -399,12 +399,13 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
 	if (!index_only) {
 		int removed = 0, gitmodules_modified = 0;
 		struct strbuf buf = STRBUF_INIT;
+		int flag = force ? REMOVE_DIR_PURGE_ORIGINAL_CWD : 0;
 		for (i = 0; i < list.nr; i++) {
 			const char *path = list.entry[i].name;
 			if (list.entry[i].is_submodule) {
 				strbuf_reset(&buf);
 				strbuf_addstr(&buf, path);
-				if (remove_dir_recursively(&buf, 0))
+				if (remove_dir_recursively(&buf, flag))
 					die(_("could not remove '%s'"), path);
 
 				removed = 1;
diff --git a/dir.c b/dir.c
index 97d6b71c872..52064345a6b 100644
--- a/dir.c
+++ b/dir.c
@@ -3204,6 +3204,7 @@ static int remove_dir_recurse(struct strbuf *path, int flag, int *kept_up)
 	int ret = 0, original_len = path->len, len, kept_down = 0;
 	int only_empty = (flag & REMOVE_DIR_EMPTY_ONLY);
 	int keep_toplevel = (flag & REMOVE_DIR_KEEP_TOPLEVEL);
+	int purge_original_cwd = (flag & REMOVE_DIR_PURGE_ORIGINAL_CWD);
 	struct object_id submodule_head;
 
 	if ((flag & REMOVE_DIR_KEEP_NESTED_GIT) &&
@@ -3259,9 +3260,14 @@ static int remove_dir_recurse(struct strbuf *path, int flag, int *kept_up)
 	closedir(dir);
 
 	strbuf_setlen(path, original_len);
-	if (!ret && !keep_toplevel && !kept_down)
-		ret = (!rmdir(path->buf) || errno == ENOENT) ? 0 : -1;
-	else if (kept_up)
+	if (!ret && !keep_toplevel && !kept_down) {
+		if (!purge_original_cwd &&
+		    startup_info->original_cwd &&
+		    !strcmp(startup_info->original_cwd, path->buf))
+			ret = -1; /* Do not remove current working directory */
+		else
+			ret = (!rmdir(path->buf) || errno == ENOENT) ? 0 : -1;
+	} else if (kept_up)
 		/*
 		 * report the uplevel that it is not an error that we
 		 * did not rmdir() our directory.
diff --git a/dir.h b/dir.h
index d6a5d03bec2..8e02dfb505d 100644
--- a/dir.h
+++ b/dir.h
@@ -495,6 +495,9 @@ int get_sparse_checkout_patterns(struct pattern_list *pl);
 /* Remove the contents of path, but leave path itself. */
 #define REMOVE_DIR_KEEP_TOPLEVEL 04
 
+/* Remove the_original_cwd too */
+#define REMOVE_DIR_PURGE_ORIGINAL_CWD 0x08
+
 /*
  * Remove path and its contents, recursively. flags is a combination
  * of the above REMOVE_DIR_* constants. Return 0 on success.
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index ce2efb9d30a..bc92230f2f2 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -291,11 +291,6 @@ test_submodule_removal () {
 	test_status=
 	test "$path_status" = dir && test_status=test_must_fail
 
-	# Actually, while path_status=dir && test_status=test_must_fail
-	# reflect our desired behavior, current behavior is:
-	path_status=missing
-	test_status=
-
 	test_when_finished "git reset --hard HEAD~1" &&
 	test_when_finished "rm -rf .git/modules/my_submodule" &&
 
-- 
gitgitgadget

[PATCH v5 11/11] t2501: simplify the tests since we can now assume desired behavior

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

We no longer are dealing with a mixture of previous and desired
behavior, so simplify the tests a bit.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh | 123 +++++++++++++------------------------------
 1 file changed, 36 insertions(+), 87 deletions(-)

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index bc92230f2f2..f6d8d7d03d7 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -32,9 +32,6 @@ test_expect_success setup '
 '
 
 test_incidental_dir_removal () {
-	works=$1 &&
-	shift &&
-
 	test_when_finished "git reset --hard" &&
 
 	git checkout foo/bar/baz^{commit} &&
@@ -44,88 +41,57 @@ test_incidental_dir_removal () {
 		cd foo &&
 		"$@" &&
 
-		# Although we want pwd & git status to pass, test for existing
-		# rather than desired behavior.
-		if test "$works" = "success"
-		then
-			test-tool getcwd &&
-			git status --porcelain
-		else
-			! test-tool getcwd &&
-			test_might_fail git status --porcelain
-		fi
+		# Make sure foo still exists, and commands needing it work
+		test-tool getcwd &&
+		git status --porcelain
 	) &&
 	test_path_is_missing foo/bar/baz &&
 	test_path_is_missing foo/bar &&
 
-	# Although we want dir to be present, test for existing rather
-	# than desired behavior.
-	if test "$works" = "success"
-	then
-		test_path_is_dir foo
-	else
-		test_path_is_missing foo
-	fi
+	test_path_is_dir foo
 }
 
 test_required_dir_removal () {
-	works=$1 &&
-	shift &&
-
 	git checkout df_conflict^{commit} &&
 	test_when_finished "git clean -fdx" &&
 
 	(
 		cd dirORfile &&
 
-		# We'd like for the command to fail (much as it would if there
-		# was an untracked file there), and for the index and worktree
-		# to be left clean with pwd and git status working afterwards.
-		# But test for existing rather than desired behavior.
-		if test "$works" = "success"
-		then
-			test_must_fail "$@" 2>../error &&
-			grep "Refusing to remove.*current working directory" ../error &&
-
-			git diff --exit-code HEAD &&
-
-			test-tool getcwd &&
-			git status --porcelain
-		else
-			"$@" &&
-			! test-tool getcwd &&
-			test_might_fail git status --porcelain
-		fi
+		# Ensure command refuses to run
+		test_must_fail "$@" 2>../error &&
+		grep "Refusing to remove.*current working directory" ../error &&
+
+		# ...and that the index and working tree are left clean
+		git diff --exit-code HEAD &&
+
+		# Ensure that getcwd and git status do not error out (which
+		# they might if the current working directory had been removed)
+		test-tool getcwd &&
+		git status --porcelain
 	) &&
 
-	# Although we want dirORfile to be present, test for existing rather
-	# than desired behavior.
-	if test "$works" = "success"
-	then
-		test_path_is_dir dirORfile
-	else
-		test_path_is_file dirORfile
-	fi
+	test_path_is_dir dirORfile
 }
 
 test_expect_success 'checkout does not clean cwd incidentally' '
-	test_incidental_dir_removal success git checkout init
+	test_incidental_dir_removal git checkout init
 '
 
 test_expect_success 'checkout fails if cwd needs to be removed' '
-	test_required_dir_removal success git checkout fd_conflict
+	test_required_dir_removal git checkout fd_conflict
 '
 
 test_expect_success 'reset --hard does not clean cwd incidentally' '
-	test_incidental_dir_removal success git reset --hard init
+	test_incidental_dir_removal git reset --hard init
 '
 
 test_expect_success 'reset --hard fails if cwd needs to be removed' '
-	test_required_dir_removal success git reset --hard fd_conflict
+	test_required_dir_removal git reset --hard fd_conflict
 '
 
 test_expect_success 'merge does not clean cwd incidentally' '
-	test_incidental_dir_removal success git merge reverted
+	test_incidental_dir_removal git merge reverted
 '
 
 # This file uses some simple merges where
@@ -154,46 +120,43 @@ test_expect_success 'merge fails if cwd needs to be removed; recursive friendly'
 GIT_TEST_MERGE_ALGORITHM=ort
 
 test_expect_success 'merge fails if cwd needs to be removed' '
-	test_required_dir_removal success git merge fd_conflict
+	test_required_dir_removal git merge fd_conflict
 '
 
 test_expect_success 'cherry-pick does not clean cwd incidentally' '
-	test_incidental_dir_removal success git cherry-pick reverted
+	test_incidental_dir_removal git cherry-pick reverted
 '
 
 test_expect_success 'cherry-pick fails if cwd needs to be removed' '
-	test_required_dir_removal success git cherry-pick fd_conflict
+	test_required_dir_removal git cherry-pick fd_conflict
 '
 
 test_expect_success 'rebase does not clean cwd incidentally' '
-	test_incidental_dir_removal success git rebase reverted
+	test_incidental_dir_removal git rebase reverted
 '
 
 test_expect_success 'rebase fails if cwd needs to be removed' '
-	test_required_dir_removal success git rebase fd_conflict
+	test_required_dir_removal git rebase fd_conflict
 '
 
 test_expect_success 'revert does not clean cwd incidentally' '
-	test_incidental_dir_removal success git revert HEAD
+	test_incidental_dir_removal git revert HEAD
 '
 
 test_expect_success 'revert fails if cwd needs to be removed' '
-	test_required_dir_removal success git revert undo_fd_conflict
+	test_required_dir_removal git revert undo_fd_conflict
 '
 
 test_expect_success 'rm does not clean cwd incidentally' '
-	test_incidental_dir_removal success git rm bar/baz.t
+	test_incidental_dir_removal git rm bar/baz.t
 '
 
 test_expect_success 'apply does not remove cwd incidentally' '
 	git diff HEAD HEAD~1 >patch &&
-	test_incidental_dir_removal success git apply ../patch
+	test_incidental_dir_removal git apply ../patch
 '
 
 test_incidental_untracked_dir_removal () {
-	works=$1 &&
-	shift &&
-
 	test_when_finished "git reset --hard" &&
 
 	git checkout foo/bar/baz^{commit} &&
@@ -205,38 +168,24 @@ test_incidental_untracked_dir_removal () {
 		cd untracked &&
 		"$@" &&
 
-		# Although we want pwd & git status to pass, test for existing
-		# rather than desired behavior.
-		if test "$works" = "success"
-		then
-			test-tool getcwd &&
-			git status --porcelain
-		else
-			! test-tool getcwd &&
-			test_might_fail git status --porcelain
-		fi
+		# Make sure untracked still exists, and commands needing it work
+		test-tool getcwd &&
+		git status --porcelain
 	) &&
 	test_path_is_missing empty &&
 	test_path_is_missing untracked/random &&
 
-	# Although we want dir to be present, test for existing rather
-	# than desired behavior.
-	if test "$works" = "success"
-	then
-		test_path_is_dir untracked
-	else
-		test_path_is_missing untracked
-	fi
+	test_path_is_dir untracked
 }
 
 test_expect_success 'clean does not remove cwd incidentally' '
-	test_incidental_untracked_dir_removal success \
+	test_incidental_untracked_dir_removal \
 		git -C .. clean -fd -e warnings . >warnings &&
 	grep "Refusing to remove current working directory" warnings
 '
 
 test_expect_success 'stash does not remove cwd incidentally' '
-	test_incidental_untracked_dir_removal success \
+	test_incidental_untracked_dir_removal \
 		git stash --include-untracked
 '
 
-- 
gitgitgadget

Re: [PATCH v5 00/11] Avoid removing the current working directory, even if it becomes empty

[Derrick Stolee]

On 12/1/2021 1:40 AM, Elijah Newren via GitGitGadget wrote:
> Traditionally, if folks run git commands such as checkout or rebase from a
> subdirectory, that git command could remove their current working directory
> and result in subsequent git and non-git commands either getting confused or
> printing messages that confuse the user (e.g. "fatal: Unable to read current
> working directory: No such file or directory"). Many commands either
> silently avoid removing directories that are not empty (i.e. those that have
> untracked or modified files in them)[1], or show an error and abort,
> depending on which is more appropriate for the command in question. With
> this series, we augment the reasons to avoid removing directories to include
> not just has-untracked-or-modified-files, but also to avoid removing the
> original_cwd as well.

I did not clearly voice my approval of the core idea here, but I do like it.

I think this fits squarely into a category of "help the user not get stuck"
which Git has enough of those situations that we don't need this one. Even
expert users won't know for sure if a 'git checkout' will cause their current
directory to be removed, however unlikely.

In the Git project, we spend a lot of time in the root of our workdir, but
this is not the typical case for large projects. I remember spending most of
my time in a previous role working four levels deep in the directory hierarchy.


I read the previous two range-diffs and took another pass at this v5 and
didn't see anything worth commenting on. This version is good to go.

There is _also_ more work to do, as follow-ups. In particular, the thing
that I thought about was sparse-checkout and created this test which still
fails at the end of your series (as an addition to t1092)

test_expect_success 'remove cwd' '
	init_repos &&

	test_sparse_match git sparse-checkout set deep/deeper1/deepest &&
	for repo in sparse-checkout sparse-index
	do
		(
			cd $repo/deep/deeper1 &&
			test-tool getcwd >"$TRASH_DIRECTORY/expect" &&
			git sparse-checkout set &&

			test-tool getcwd >"$TRASH_DIRECTORY/actual" &&
			test_sparse_match git status --porcelain &&
			cd "$TRASH_DIRECTORY" &&
			test_cmp expect actual
		)
	done
'

Please do not let this test delay the advancement of this series. As we
find these kinds of issues, we can fix them one-by-one as needed.

Thanks,
-Stolee

Re: [PATCH v5 00/11] Avoid removing the current working directory, even if it becomes empty

[Ævar Arnfjörð Bjarmason]

On Tue, Dec 07 2021, Derrick Stolee wrote:

> On 12/1/2021 1:40 AM, Elijah Newren via GitGitGadget wrote:
>> Traditionally, if folks run git commands such as checkout or rebase from a
>> subdirectory, that git command could remove their current working directory
>> and result in subsequent git and non-git commands either getting confused or
>> printing messages that confuse the user (e.g. "fatal: Unable to read current
>> working directory: No such file or directory"). Many commands either
>> silently avoid removing directories that are not empty (i.e. those that have
>> untracked or modified files in them)[1], or show an error and abort,
>> depending on which is more appropriate for the command in question. With
>> this series, we augment the reasons to avoid removing directories to include
>> not just has-untracked-or-modified-files, but also to avoid removing the
>> original_cwd as well.
>
> I did not clearly voice my approval of the core idea here, but I do like it.
>
> I think this fits squarely into a category of "help the user not get stuck"
> which Git has enough of those situations that we don't need this one. Even
> expert users won't know for sure if a 'git checkout' will cause their current
> directory to be removed, however unlikely.
>
> In the Git project, we spend a lot of time in the root of our workdir, but
> this is not the typical case for large projects. I remember spending most of
> my time in a previous role working four levels deep in the directory hierarchy.
>
>
> I read the previous two range-diffs and took another pass at this v5 and
> didn't see anything worth commenting on. This version is good to go.
>
> There is _also_ more work to do, as follow-ups. In particular, the thing
> that I thought about was sparse-checkout and created this test which still
> fails at the end of your series (as an addition to t1092)
>
> test_expect_success 'remove cwd' '
> 	init_repos &&
>
> 	test_sparse_match git sparse-checkout set deep/deeper1/deepest &&
> 	for repo in sparse-checkout sparse-index
> 	do
> 		(
> 			cd $repo/deep/deeper1 &&
> 			test-tool getcwd >"$TRASH_DIRECTORY/expect" &&
> 			git sparse-checkout set &&
>
> 			test-tool getcwd >"$TRASH_DIRECTORY/actual" &&
> 			test_sparse_match git status --porcelain &&
> 			cd "$TRASH_DIRECTORY" &&
> 			test_cmp expect actual
> 		)
> 	done
> '
>
> Please do not let this test delay the advancement of this series. As we
> find these kinds of issues, we can fix them one-by-one as needed.

Not to pile on about "the core idea", just a question while this is
fresh in your mind:

I think that those cases would per [1] be ones where a more isolated
change of reading the $PWD from the environment would make all those
commands work as expected. Or would the "$TRASH_DIRECTORY" also
otherwise go away in this examples?

Anyway, just per [1] and the potential future follow-ups is this (I
don't think so, but maybe I'm wrong) or other examples you have things
that specifically need the "retain the getcwd()" part of this series?

Or just (as I think would be the case with that "git status") to not
have setup.c die quite as eagerly as it does now when getcwd() fails,
but it can find its way back to the .git via the environment's $PWD?

Also note that writing any tests to check the current behavior is a pain
due to the bin-wrappers as I discussed in some previous threads, but
that's a test-only oddity, and won't impact how git would behave once
installed (although now it doesn't behave well at all). So probing the
current behavior via tests is hard, unless you use GIT_TEST_INSTALLED to
get around the bin-wrappers.

There *are* definitely cases where not-just-that-setup.c change but also
the "don't remove the CWD" is an inherently better & more complete
solution. But I think that's mainly to do with 3rd party shellscripts &
other programs outside of our control.

I'm assuming that you were working with this on Windows, where
presumably there's fewer/none such shellscripts you rely on, but that's
now two presumes in a row, so... :)

1. https://lore.kernel.org/git/211207.86ee6opy0f.gmgdl@evledraar.gmail.com/

Re: [PATCH v5 00/11] Avoid removing the current working directory, even if it becomes empty

[Elijah Newren]

On Tue, Dec 7, 2021 at 8:09 AM Derrick Stolee <stolee@gmail.com> wrote:
>
> On 12/1/2021 1:40 AM, Elijah Newren via GitGitGadget wrote:
> > Traditionally, if folks run git commands such as checkout or rebase from a
> > subdirectory, that git command could remove their current working directory
> > and result in subsequent git and non-git commands either getting confused or
> > printing messages that confuse the user (e.g. "fatal: Unable to read current
> > working directory: No such file or directory"). Many commands either
> > silently avoid removing directories that are not empty (i.e. those that have
> > untracked or modified files in them)[1], or show an error and abort,
> > depending on which is more appropriate for the command in question. With
> > this series, we augment the reasons to avoid removing directories to include
> > not just has-untracked-or-modified-files, but also to avoid removing the
> > original_cwd as well.
>
> I did not clearly voice my approval of the core idea here, but I do like it.
>
> I think this fits squarely into a category of "help the user not get stuck"
> which Git has enough of those situations that we don't need this one. Even
> expert users won't know for sure if a 'git checkout' will cause their current
> directory to be removed, however unlikely.
>
> In the Git project, we spend a lot of time in the root of our workdir, but
> this is not the typical case for large projects. I remember spending most of
> my time in a previous role working four levels deep in the directory hierarchy.
>
>
> I read the previous two range-diffs and took another pass at this v5 and
> didn't see anything worth commenting on. This version is good to go.
>
> There is _also_ more work to do, as follow-ups. In particular, the thing
> that I thought about was sparse-checkout and created this test which still
> fails at the end of your series (as an addition to t1092)

Interesting testcase...

> test_expect_success 'remove cwd' '
>         init_repos &&
>
>         test_sparse_match git sparse-checkout set deep/deeper1/deepest &&
>         for repo in sparse-checkout sparse-index
>         do
>                 (
>                         cd $repo/deep/deeper1 &&
>                         test-tool getcwd >"$TRASH_DIRECTORY/expect" &&
>                         git sparse-checkout set &&
>
>                         test-tool getcwd >"$TRASH_DIRECTORY/actual" &&
>                         test_sparse_match git status --porcelain &&

However, this line is broken even if the directory weren't removed.
Not because of the "git status --porcelain" part, but because of two
other reasons:

1) test_sparse_match presumes it is run from the directory above the
repos while you still have it in $repo/deep/deeper1
2) The point of test_sparse_match is to compare the results in two
different repositories, but it'll do this twice and the first time
without the changes having been made to the sparse-index repo.
Perhaps this belonged outside the surrounding for loop?

I think I'd either drop the "test_sparse_match" or else just drop the
whole line; the real comparison is the expect/actual files.  Dropping
this line makes it a good test.

>                         cd "$TRASH_DIRECTORY" &&
>                         test_cmp expect actual
>                 )
>         done
> '
>
> Please do not let this test delay the advancement of this series. As we
> find these kinds of issues, we can fix them one-by-one as needed.

Yeah, sounds good.  Since you piqued my interest, though, the problem
is that we're passing an absolute path to remove_dir_recursively()
inside clean_tracked_sparse_directories() when we should be passing a
relative path.  (We always chdir(r->worktree) in setup.c, so there's
no need to prepend the path with r->worktree+'/'.)

Still, the current series is long enough and unless there are issues
others have spotted with it, I'd rather just let it proceed as-is and
then send this fix and a correction of your testcase in separately.

Re: [PATCH v5 00/11] Avoid removing the current working directory, even if it becomes empty

[Derrick Stolee]

On 12/7/2021 1:30 PM, Ævar Arnfjörð Bjarmason wrote:
> 
> On Tue, Dec 07 2021, Derrick Stolee wrote:
>> test_expect_success 'remove cwd' '
>> 	init_repos &&
>>
>> 	test_sparse_match git sparse-checkout set deep/deeper1/deepest &&
>> 	for repo in sparse-checkout sparse-index
>> 	do
>> 		(
>> 			cd $repo/deep/deeper1 &&
>> 			test-tool getcwd >"$TRASH_DIRECTORY/expect" &&
>> 			git sparse-checkout set &&
>>
>> 			test-tool getcwd >"$TRASH_DIRECTORY/actual" &&
>> 			test_sparse_match git status --porcelain &&
>> 			cd "$TRASH_DIRECTORY" &&
>> 			test_cmp expect actual
>> 		)
>> 	done
>> '
>>
>> Please do not let this test delay the advancement of this series. As we
>> find these kinds of issues, we can fix them one-by-one as needed.
> 
> Not to pile on about "the core idea", just a question while this is
> fresh in your mind:
> 
> I think that those cases would per [1] be ones where a more isolated
> change of reading the $PWD from the environment would make all those
> commands work as expected. Or would the "$TRASH_DIRECTORY" also
> otherwise go away in this examples?

I have read this message and the one you are referring two twice and
I cannot understand what you are trying to say here.

> Anyway, just per [1] and the potential future follow-ups is this (I
> don't think so, but maybe I'm wrong) or other examples you have things
> that specifically need the "retain the getcwd()" part of this series?
> 
> Or just (as I think would be the case with that "git status") to not
> have setup.c die quite as eagerly as it does now when getcwd() fails,
> but it can find its way back to the .git via the environment's $PWD?

Are you implying that Git will be the only thing broken by a missing
directory after we leave in this state? I doubt that is true, and we
should be good citizens here by leaving the directory around.

> There *are* definitely cases where not-just-that-setup.c change but also
> the "don't remove the CWD" is an inherently better & more complete
> solution. But I think that's mainly to do with 3rd party shellscripts &
> other programs outside of our control.

Exactly. We should take this change because it is valuable to not cause
a confusing error in other tools.
 
> I'm assuming that you were working with this on Windows, where
> presumably there's fewer/none such shellscripts you rely on, but that's
> now two presumes in a row, so... :)

I'm working on Ubuntu, where I do all of my Git development unless there
is a platform specific reason to do so.

Thanks,
-Stolee

Re: [PATCH v5 00/11] Avoid removing the current working directory, even if it becomes empty

[Derrick Stolee]

On 12/7/2021 3:43 PM, Elijah Newren wrote:
> On Tue, Dec 7, 2021 at 8:09 AM Derrick Stolee <stolee@gmail.com> wrote:
>> There is _also_ more work to do, as follow-ups. In particular, the thing
>> that I thought about was sparse-checkout and created this test which still
>> fails at the end of your series (as an addition to t1092)
> 
> Interesting testcase...
> 
>> test_expect_success 'remove cwd' '
>>         init_repos &&
>>
>>         test_sparse_match git sparse-checkout set deep/deeper1/deepest &&
>>         for repo in sparse-checkout sparse-index
>>         do
>>                 (
>>                         cd $repo/deep/deeper1 &&
>>                         test-tool getcwd >"$TRASH_DIRECTORY/expect" &&
>>                         git sparse-checkout set &&
>>
>>                         test-tool getcwd >"$TRASH_DIRECTORY/actual" &&
>>                         test_sparse_match git status --porcelain &&
> 
> However, this line is broken even if the directory weren't removed.
> Not because of the "git status --porcelain" part, but because of two
> other reasons:
> 
> 1) test_sparse_match presumes it is run from the directory above the
> repos while you still have it in $repo/deep/deeper1
> 2) The point of test_sparse_match is to compare the results in two
> different repositories, but it'll do this twice and the first time
> without the changes having been made to the sparse-index repo.
> Perhaps this belonged outside the surrounding for loop?

You're right! I had first written this test with test_sparse_match
everywhere, but test_sparse_match uses subshells to 'cd' into different
places, so it doesn't let us get a failure in the 'test-tool getcwd'.

I missed this one because the test fails with 'test-tool getcwd' so I
never get to the broken line.

> I think I'd either drop the "test_sparse_match" or else just drop the
> whole line; the real comparison is the expect/actual files.  Dropping
> this line makes it a good test.

I agree.

>>                         cd "$TRASH_DIRECTORY" &&
>>                         test_cmp expect actual
>>                 )
>>         done
>> '
>>
>> Please do not let this test delay the advancement of this series. As we
>> find these kinds of issues, we can fix them one-by-one as needed.
> 
> Yeah, sounds good.  Since you piqued my interest, though, the problem
> is that we're passing an absolute path to remove_dir_recursively()
> inside clean_tracked_sparse_directories() when we should be passing a
> relative path.  (We always chdir(r->worktree) in setup.c, so there's
> no need to prepend the path with r->worktree+'/'.)
> 
> Still, the current series is long enough and unless there are issues
> others have spotted with it, I'd rather just let it proceed as-is and
> then send this fix and a correction of your testcase in separately.

Absolutely. Let's pick up this fix another time.

Thanks,
-Stolee

Re: [PATCH v5 00/11] Avoid removing the current working directory, even if it becomes empty

[Ævar Arnfjörð Bjarmason]

On Tue, Dec 07 2021, Derrick Stolee wrote:

> On 12/7/2021 1:30 PM, Ævar Arnfjörð Bjarmason wrote:
>> 
>> On Tue, Dec 07 2021, Derrick Stolee wrote:
>>> test_expect_success 'remove cwd' '
>>> 	init_repos &&
>>>
>>> 	test_sparse_match git sparse-checkout set deep/deeper1/deepest &&
>>> 	for repo in sparse-checkout sparse-index
>>> 	do
>>> 		(
>>> 			cd $repo/deep/deeper1 &&
>>> 			test-tool getcwd >"$TRASH_DIRECTORY/expect" &&
>>> 			git sparse-checkout set &&
>>>
>>> 			test-tool getcwd >"$TRASH_DIRECTORY/actual" &&
>>> 			test_sparse_match git status --porcelain &&
>>> 			cd "$TRASH_DIRECTORY" &&
>>> 			test_cmp expect actual
>>> 		)
>>> 	done
>>> '
>>>
>>> Please do not let this test delay the advancement of this series. As we
>>> find these kinds of issues, we can fix them one-by-one as needed.
>> 
>> Not to pile on about "the core idea", just a question while this is
>> fresh in your mind:
>> 
>> I think that those cases would per [1] be ones where a more isolated
>> change of reading the $PWD from the environment would make all those
>> commands work as expected. Or would the "$TRASH_DIRECTORY" also
>> otherwise go away in this examples?
>
> I have read this message and the one you are referring two twice and
> I cannot understand what you are trying to say here.

I'm asking whether the WIP patch I posted at
https://lore.kernel.org/git/211124.86sfvld4cl.gmgdl@evledraar.gmail.com/
would partially/entirely solve those issues you mentiened (but I see
Elijah has a side-reply saying it might not in either approach).

>> Anyway, just per [1] and the potential future follow-ups is this (I
>> don't think so, but maybe I'm wrong) or other examples you have things
>> that specifically need the "retain the getcwd()" part of this series?
>> 
>> Or just (as I think would be the case with that "git status") to not
>> have setup.c die quite as eagerly as it does now when getcwd() fails,
>> but it can find its way back to the .git via the environment's $PWD?
>
> Are you implying that Git will be the only thing broken by a missing
> directory after we leave in this state? I doubt that is true, and we
> should be good citizens here by leaving the directory around.

No, I'm just trying to clarify which specific thing we'd need in
this/other cases to fix things *in git*. Because...

>> There *are* definitely cases where not-just-that-setup.c change but also
>> the "don't remove the CWD" is an inherently better & more complete
>> solution. But I think that's mainly to do with 3rd party shellscripts &
>> other programs outside of our control.
>
> Exactly. We should take this change because it is valuable to not cause
> a confusing error in other tools.

...maybe we should fix things for other tools too, but for any future
development it helps to know what was needed to fix what issues.

>> I'm assuming that you were working with this on Windows, where
>> presumably there's fewer/none such shellscripts you rely on, but that's
>> now two presumes in a row, so... :)
>
> I'm working on Ubuntu, where I do all of my Git development unless there
> is a platform specific reason to do so.

*nod*, badly assumed then :)

[PATCH v6 00/11] Avoid removing the current working directory, even if it becomes empty

[Elijah Newren via GitGitGadget]

Traditionally, if folks run git commands such as checkout or rebase from a
subdirectory, that git command could remove their current working directory
and result in subsequent git and non-git commands either getting confused or
printing messages that confuse the user (e.g. "fatal: Unable to read current
working directory: No such file or directory"). Many commands either
silently avoid removing directories that are not empty (i.e. those that have
untracked or modified files in them)[1], or show an error and abort,
depending on which is more appropriate for the command in question. With
this series, we augment the reasons to avoid removing directories to include
not just has-untracked-or-modified-files, but also to avoid removing the
original_cwd as well.

Peff and Junio provided some good pros/cons, if it helps:

 * Pros: Peff (original suggester of the idea)[2], and Junio[3]
 * Cons: Peff [2, again -- see the "P.S."], and Junio[4]

[1] well, with a few exceptions; see
https://lore.kernel.org/git/pull.1036.v3.git.1632760428.gitgitgadget@gmail.com/
[2] https://lore.kernel.org/git/YS8eEtwQvF7TaLCb@coredump.intra.peff.net/
[3] https://lore.kernel.org/git/xmqqo86elyht.fsf@gitster.g/ [4]
https://lore.kernel.org/git/xmqqo8691gr8.fsf@gitster.g/

Changes since v5:

 * It's been about a week without changes, so including Acks that have come
   in (let me know if I was wrong in my interpretation of these statements
   as Acks):
   * from Stolee ("This version is good to go",
     https://lore.kernel.org/git/aa85e35d-143e-93e4-f54b-146b38dd4b88@gmail.com/)
   * from Ævar ("I've got no objections to these changes going in as they
     stand.",
     https://lore.kernel.org/git/211207.86ee6opy0f.gmgdl@evledraar.gmail.com/)

Changes since v4:

 * actually fix bashism

Changes since v3:

 * fixed one codepath from v2 so that the series really is only about the
   working tree
 * used test-tool getcwd instead of pwd -P as suggested by Ævar for some
   less common platforms
 * fixed bashism
 * check for clean index/worktree after verifying that expected-to-abort
   codepaths do abort, to make it clearer that we expect an early abort
 * remove a leftover (and confusing) is_absolute_dir() check in sequencer
   and stash from an earlier round of the series

Changes since v2:

 * the series is now only about the working tree. So if the original cwd is
   outside the worktree (or we're in a bare repo), then the new code is a
   no-op.
 * fixed ugly early die() possibility (uses strbuf_getcwd() instead of
   xgetcwd())
 * modified the initial tests to show both expected and desired behavior.
   subsequent patches fix the tests. One new patch added at the end which
   simplifies the tests to only check for desired behavior.
 * NULLify startup_info->original_cwd when it matches the toplevel worktree;
   that is already protected and we don't need secondary protection for it.
   This simplified some other codepaths so we don't have to check for
   startup_info->original_cwd == "".
 * clarified some commit messages

Changes since v1:

 * clarified multiple commit messages
 * renamed the_cwd to startup_info->original_cwd to make it clearer that
   it's our parent process'es cwd that really matters, which we inherited at
   program startup. Also pulls it out of the global namespace.
 * Normalize the path for startup_info->original_cwd, and ensure that it's
   actually the original cwd even if -C is passed to git.
 * small code cleanups suggested by René and Ævar
 * split the final patch (which got the most comments) into two -- one for
   each function being modified. Also, add a bunch more history to the first
   of the two resulting commit messages
 * no longer has a content conflict with so/stash-staged
 * add another value for the flags parameter that remove_dir_recursively()
   takes so that it can opt into either the old or the new behavior. Use
   that for the one special corner case I could find where it matters, and
   add a few tests around it to highlight the utility of the flag.

Elijah Newren (11):
  t2501: add various tests for removing the current working directory
  setup: introduce startup_info->original_cwd
  unpack-trees: refuse to remove startup_info->original_cwd
  unpack-trees: add special cwd handling
  symlinks: do not include startup_info->original_cwd in dir removal
  clean: do not attempt to remove startup_info->original_cwd
  rebase: do not attempt to remove startup_info->original_cwd
  stash: do not attempt to remove startup_info->original_cwd
  dir: avoid incidentally removing the original_cwd in remove_path()
  dir: new flag to remove_dir_recurse() to spare the original_cwd
  t2501: simplify the tests since we can now assume desired behavior

 builtin/clean.c      |  44 +++++--
 builtin/rm.c         |   3 +-
 builtin/stash.c      |   4 +-
 cache.h              |   2 +
 common-main.c        |   4 +
 dir.c                |  15 ++-
 dir.h                |   9 +-
 sequencer.c          |   2 +
 setup.c              |  65 ++++++++++
 symlinks.c           |   8 +-
 t/t2501-cwd-empty.sh | 277 +++++++++++++++++++++++++++++++++++++++++++
 unpack-trees.c       |  30 ++++-
 unpack-trees.h       |   1 +
 13 files changed, 442 insertions(+), 22 deletions(-)
 create mode 100755 t/t2501-cwd-empty.sh


base-commit: 88d915a634b449147855041d44875322de2b286d
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1140%2Fnewren%2Fcwd_removal-v6
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1140/newren/cwd_removal-v6
Pull-Request: https://github.com/git/git/pull/1140

Range-diff vs v5:

  1:  7c72e888d2e !  1:  1fd76d4f8a8 t2501: add various tests for removing the current working directory
     @@ Commit message
          Also add a few tests suggested during the review of earlier rounds of
          this patch series.
      
     +    Acked-by: Derrick Stolee <stolee@gmail.com>
     +    Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## t/t2501-cwd-empty.sh (new) ##
  2:  37f333b2024 !  2:  cd1f564c4c0 setup: introduce startup_info->original_cwd
     @@ Commit message
      
          Subsequent commits will make use of this new variable.
      
     +    Acked-by: Derrick Stolee <stolee@gmail.com>
     +    Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## cache.h ##
  3:  b611c73bd15 !  3:  4a1f62b662c unpack-trees: refuse to remove startup_info->original_cwd
     @@ Commit message
          error out if the directory is the current working directory we inherited
          from our parent process.
      
     +    Acked-by: Derrick Stolee <stolee@gmail.com>
     +    Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## t/t2501-cwd-empty.sh ##
  4:  706415a4547 !  4:  03f2b0b8fb7 unpack-trees: add special cwd handling
     @@ Commit message
          there is no untracked checking to be done, so we simply add a special
          case near the top of verify_absent_1.
      
     +    Acked-by: Derrick Stolee <stolee@gmail.com>
     +    Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## t/t2501-cwd-empty.sh ##
  5:  66ef6b4d943 !  5:  e75f6c3f9ad symlinks: do not include startup_info->original_cwd in dir removal
     @@ Commit message
          commands) that would otherwise report confusing messages about being
          unable to read the current working directory.
      
     +    Acked-by: Derrick Stolee <stolee@gmail.com>
     +    Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## symlinks.c ##
  6:  54d059c683e !  6:  d4f8784a6e4 clean: do not attempt to remove startup_info->original_cwd
     @@ Metadata
       ## Commit message ##
          clean: do not attempt to remove startup_info->original_cwd
      
     +    Acked-by: Derrick Stolee <stolee@gmail.com>
     +    Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## builtin/clean.c ##
  7:  f8efb7446c3 !  7:  fd500cc1843 rebase: do not attempt to remove startup_info->original_cwd
     @@ Commit message
          the startup_info->original_cwd directory, so that the checkout process
          knows to protect that directory.
      
     +    Acked-by: Derrick Stolee <stolee@gmail.com>
     +    Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## sequencer.c ##
  8:  2855ed45112 !  8:  6f3c4fd49b9 stash: do not attempt to remove startup_info->original_cwd
     @@ Commit message
          longer run from the toplevel, pass the ':/' magic pathspec to ensure we
          still clean from the toplevel.
      
     +    Acked-by: Derrick Stolee <stolee@gmail.com>
     +    Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## builtin/stash.c ##
  9:  23dfc3e399d !  9:  1a66b1a2386 dir: avoid incidentally removing the original_cwd in remove_path()
     @@ Commit message
          looked through every caller of remove_path() in the current codebase to
          make sure that all should take this change.
      
     +    Acked-by: Derrick Stolee <stolee@gmail.com>
     +    Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## dir.c ##
 10:  fe47c0f0c17 ! 10:  4a18efd51ef dir: new flag to remove_dir_recurse() to spare the original_cwd
     @@ Commit message
          highlight that very specific case involving submodules && --git-dir &&
          --work-tree.
      
     +    Acked-by: Derrick Stolee <stolee@gmail.com>
     +    Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## builtin/rm.c ##
 11:  431dd651a7e ! 11:  63781ed8b8e t2501: simplify the tests since we can now assume desired behavior
     @@ Commit message
          We no longer are dealing with a mixture of previous and desired
          behavior, so simplify the tests a bit.
      
     +    Acked-by: Derrick Stolee <stolee@gmail.com>
     +    Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## t/t2501-cwd-empty.sh ##

-- 
gitgitgadget

[PATCH v6 01/11] t2501: add various tests for removing the current working directory

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Numerous commands will remove directories left empty as a "convenience"
after removing files within them.  That is normally fine, but removing
the current working directory can be rather inconvenient since it can
cause confusion for the user when they run subsequent commands.  For
example, after one git process has removed the current working
directory, git status/log/diff will all abort with the message:

    fatal: Unable to read current working directory: No such file or directory

We also have code paths that, when a file needs to be placed where a
directory is (due to e.g. checkout, merge, reset, whatever), will check
if this is okay and error out if not.  These rules include:
  * all tracked files under that directory are intended to be removed by
    the operation
  * none of the tracked files under that directory have uncommitted
    modification
  * there are no untracked files under that directory
However, if we end up remove the current working directory, we can cause
user confusion when they run subsequent commands, so we would prefer if
there was a fourth rule added to this list: avoid removing the current
working directory.

Since there are several code paths that can result in the current
working directory being removed, add several tests of various different
codepaths.  To make it clearer what the difference between the current
behavior and the behavior at the end of the series, code both of them
into the tests and have the appropriate behavior be selected by a flag.
Subsequent commits will toggle the flag from current to desired
behavior.

Also add a few tests suggested during the review of earlier rounds of
this patch series.

Acked-by: Derrick Stolee <stolee@gmail.com>
Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh | 342 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 342 insertions(+)
 create mode 100755 t/t2501-cwd-empty.sh

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
new file mode 100755
index 00000000000..a05abd18187
--- /dev/null
+++ b/t/t2501-cwd-empty.sh
@@ -0,0 +1,342 @@
+#!/bin/sh
+
+test_description='Test handling of the current working directory becoming empty'
+
+. ./test-lib.sh
+
+test_expect_success setup '
+	test_commit init &&
+
+	git branch fd_conflict &&
+
+	mkdir -p foo/bar &&
+	test_commit foo/bar/baz &&
+
+	git revert HEAD &&
+	git tag reverted &&
+
+	git checkout fd_conflict &&
+	mkdir dirORfile &&
+	test_commit dirORfile/foo &&
+
+	git rm -r dirORfile &&
+	echo not-a-directory >dirORfile &&
+	git add dirORfile &&
+	git commit -m dirORfile &&
+
+	git switch -c df_conflict HEAD~1 &&
+	test_commit random_file &&
+
+	git switch -c undo_fd_conflict fd_conflict &&
+	git revert HEAD
+'
+
+test_incidental_dir_removal () {
+	works=$1 &&
+	shift &&
+
+	test_when_finished "git reset --hard" &&
+
+	git checkout foo/bar/baz^{commit} &&
+	test_path_is_dir foo/bar &&
+
+	(
+		cd foo &&
+		"$@" &&
+
+		# Although we want pwd & git status to pass, test for existing
+		# rather than desired behavior.
+		if test "$works" = "success"
+		then
+			test-tool getcwd &&
+			git status --porcelain
+		else
+			! test-tool getcwd &&
+			test_might_fail git status --porcelain
+		fi
+	) &&
+	test_path_is_missing foo/bar/baz &&
+	test_path_is_missing foo/bar &&
+
+	# Although we want dir to be present, test for existing rather
+	# than desired behavior.
+	if test "$works" = "success"
+	then
+		test_path_is_dir foo
+	else
+		test_path_is_missing foo
+	fi
+}
+
+test_required_dir_removal () {
+	works=$1 &&
+	shift &&
+
+	git checkout df_conflict^{commit} &&
+	test_when_finished "git clean -fdx" &&
+
+	(
+		cd dirORfile &&
+
+		# We'd like for the command to fail (much as it would if there
+		# was an untracked file there), and for the index and worktree
+		# to be left clean with pwd and git status working afterwards.
+		# But test for existing rather than desired behavior.
+		if test "$works" = "success"
+		then
+			test_must_fail "$@" 2>../error &&
+			grep "Refusing to remove.*current working directory" ../error &&
+
+			git diff --exit-code HEAD &&
+
+			test-tool getcwd &&
+			git status --porcelain
+		else
+			"$@" &&
+			! test-tool getcwd &&
+			test_might_fail git status --porcelain
+		fi
+	) &&
+
+	# Although we want dirORfile to be present, test for existing rather
+	# than desired behavior.
+	if test "$works" = "success"
+	then
+		test_path_is_dir dirORfile
+	else
+		test_path_is_file dirORfile
+	fi
+}
+
+test_expect_success 'checkout does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git checkout init
+'
+
+test_expect_success 'checkout fails if cwd needs to be removed' '
+	test_required_dir_removal failure git checkout fd_conflict
+'
+
+test_expect_success 'reset --hard does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git reset --hard init
+'
+
+test_expect_success 'reset --hard fails if cwd needs to be removed' '
+	test_required_dir_removal failure git reset --hard fd_conflict
+'
+
+test_expect_success 'merge does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git merge reverted
+'
+
+# This file uses some simple merges where
+#   Base: 'dirORfile/' exists
+#   Side1: random other file changed
+#   Side2: 'dirORfile/' removed, 'dirORfile' added
+# this should resolve cleanly, but merge-recursive throws merge conflicts
+# because it's dumb.  Add a special test for checking merge-recursive (and
+# merge-ort), then after this just hard require ort for all remaining tests.
+#
+test_expect_success 'merge fails if cwd needs to be removed; recursive friendly' '
+	git checkout foo/bar/baz &&
+	test_when_finished "git clean -fdx" &&
+
+	mkdir dirORfile &&
+	(
+		cd dirORfile &&
+
+		# We would rather this failed, but we test for existing
+		# rather than desired behavior
+		git merge fd_conflict 2>../error
+	) &&
+
+	## Here is the behavior we would rather have:
+	#test_path_is_dir dirORfile &&
+	#grep "Refusing to remove the current working directory" error
+	## But instead we test for existing behavior
+	test_path_is_file dirORfile &&
+	test_must_be_empty error
+'
+
+GIT_TEST_MERGE_ALGORITHM=ort
+
+test_expect_success 'merge fails if cwd needs to be removed' '
+	test_required_dir_removal failure git merge fd_conflict
+'
+
+test_expect_success 'cherry-pick does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git cherry-pick reverted
+'
+
+test_expect_success 'cherry-pick fails if cwd needs to be removed' '
+	test_required_dir_removal failure git cherry-pick fd_conflict
+'
+
+test_expect_success 'rebase does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git rebase reverted
+'
+
+test_expect_success 'rebase fails if cwd needs to be removed' '
+	test_required_dir_removal failure git rebase fd_conflict
+'
+
+test_expect_success 'revert does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git revert HEAD
+'
+
+test_expect_success 'revert fails if cwd needs to be removed' '
+	test_required_dir_removal failure git revert undo_fd_conflict
+'
+
+test_expect_success 'rm does not clean cwd incidentally' '
+	test_incidental_dir_removal failure git rm bar/baz.t
+'
+
+test_expect_success 'apply does not remove cwd incidentally' '
+	git diff HEAD HEAD~1 >patch &&
+	test_incidental_dir_removal failure git apply ../patch
+'
+
+test_incidental_untracked_dir_removal () {
+	works=$1 &&
+	shift &&
+
+	test_when_finished "git reset --hard" &&
+
+	git checkout foo/bar/baz^{commit} &&
+	mkdir -p untracked &&
+	mkdir empty
+	>untracked/random &&
+
+	(
+		cd untracked &&
+		"$@" &&
+
+		# Although we want pwd & git status to pass, test for existing
+		# rather than desired behavior.
+		if test "$works" = "success"
+		then
+			test-tool getcwd &&
+			git status --porcelain
+		else
+			! test-tool getcwd &&
+			test_might_fail git status --porcelain
+		fi
+	) &&
+	test_path_is_missing empty &&
+	test_path_is_missing untracked/random &&
+
+	# Although we want dir to be present, test for existing rather
+	# than desired behavior.
+	if test "$works" = "success"
+	then
+		test_path_is_dir untracked
+	else
+		test_path_is_missing untracked
+	fi
+}
+
+test_expect_success 'clean does not remove cwd incidentally' '
+	test_incidental_untracked_dir_removal failure \
+		git -C .. clean -fd -e warnings . >warnings
+'
+
+test_expect_success 'stash does not remove cwd incidentally' '
+	test_incidental_untracked_dir_removal failure \
+		git stash --include-untracked
+'
+
+test_expect_success '`rm -rf dir` only removes a subset of dir' '
+	test_when_finished "rm -rf a/" &&
+
+	mkdir -p a/b/c &&
+	>a/b/c/untracked &&
+	>a/b/c/tracked &&
+	git add a/b/c/tracked &&
+
+	(
+		cd a/b &&
+		git rm -rf ../b
+	) &&
+
+	test_path_is_dir a/b &&
+	test_path_is_missing a/b/c/tracked &&
+	test_path_is_file a/b/c/untracked
+'
+
+test_expect_success '`rm -rf dir` even with only tracked files will remove something else' '
+	test_when_finished "rm -rf a/" &&
+
+	mkdir -p a/b/c &&
+	>a/b/c/tracked &&
+	git add a/b/c/tracked &&
+
+	(
+		cd a/b &&
+		git rm -rf ../b
+	) &&
+
+	test_path_is_missing a/b/c/tracked &&
+	## We would prefer if a/b was still present, though empty, since it
+	## was the current working directory
+	#test_path_is_dir a/b
+	## But the current behavior is that it not only deletes the directory
+	## a/b as requested, but also goes and deletes a
+	test_path_is_missing a
+'
+
+test_expect_success 'git version continues working from a deleted dir' '
+	mkdir tmp &&
+	(
+		cd tmp &&
+		rm -rf ../tmp &&
+		git version
+	)
+'
+
+test_submodule_removal () {
+	path_status=$1 &&
+	shift &&
+
+	test_status=
+	test "$path_status" = dir && test_status=test_must_fail
+
+	# Actually, while path_status=dir && test_status=test_must_fail
+	# reflect our desired behavior, current behavior is:
+	path_status=missing
+	test_status=
+
+	test_when_finished "git reset --hard HEAD~1" &&
+	test_when_finished "rm -rf .git/modules/my_submodule" &&
+
+	git checkout foo/bar/baz &&
+
+	git init my_submodule &&
+	touch my_submodule/file &&
+	git -C my_submodule add file &&
+	git -C my_submodule commit -m "initial commit" &&
+	git submodule add ./my_submodule &&
+	git commit -m "Add the submodule" &&
+
+	(
+		cd my_submodule &&
+		$test_status "$@"
+	) &&
+
+	test_path_is_${path_status} my_submodule
+}
+
+test_expect_success 'rm -r with -C leaves submodule if cwd inside' '
+	test_submodule_removal dir git -C .. rm -r my_submodule/
+'
+
+test_expect_success 'rm -r leaves submodule if cwd inside' '
+	test_submodule_removal dir \
+		git --git-dir=../.git --work-tree=.. rm -r ../my_submodule/
+'
+
+test_expect_success 'rm -rf removes submodule even if cwd inside' '
+	test_submodule_removal missing \
+		git --git-dir=../.git --work-tree=.. rm -rf ../my_submodule/
+'
+
+test_done
-- 
gitgitgadget

[PATCH v6 02/11] setup: introduce startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Removing the current working directory causes all subsequent git
commands run from that directory to get confused and fail with a message
about being unable to read the current working directory:

    $ git status
    fatal: Unable to read current working directory: No such file or directory

Non-git commands likely have similar warnings or even errors, e.g.

    $ bash -c 'echo hello'
    shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    hello

This confuses end users, particularly since the command they get the
error from is not the one that caused the problem; the problem came from
the side-effect of some previous command.

We would like to avoid removing the current working directory of our
parent process; towards this end, introduce a new variable,
startup_info->original_cwd, that tracks the current working directory
that we inherited from our parent process.  For convenience of later
comparisons, we prefer that this new variable store a path relative to
the toplevel working directory (thus much like 'prefix'), except without
the trailing slash.

Subsequent commits will make use of this new variable.

Acked-by: Derrick Stolee <stolee@gmail.com>
Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 cache.h       |  2 ++
 common-main.c |  4 ++++
 setup.c       | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 71 insertions(+)

diff --git a/cache.h b/cache.h
index eba12487b99..92e181ea759 100644
--- a/cache.h
+++ b/cache.h
@@ -1834,8 +1834,10 @@ void overlay_tree_on_index(struct index_state *istate,
 struct startup_info {
 	int have_repository;
 	const char *prefix;
+	const char *original_cwd;
 };
 extern struct startup_info *startup_info;
+extern const char *tmp_original_cwd;
 
 /* merge.c */
 struct commit_list;
diff --git a/common-main.c b/common-main.c
index 71e21dd20a3..aa8d5aba5bb 100644
--- a/common-main.c
+++ b/common-main.c
@@ -26,6 +26,7 @@ static void restore_sigpipe_to_default(void)
 int main(int argc, const char **argv)
 {
 	int result;
+	struct strbuf tmp = STRBUF_INIT;
 
 	trace2_initialize_clock();
 
@@ -49,6 +50,9 @@ int main(int argc, const char **argv)
 	trace2_cmd_start(argv);
 	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
 
+	if (!strbuf_getcwd(&tmp))
+		tmp_original_cwd = strbuf_detach(&tmp, NULL);
+
 	result = cmd_main(argc, argv);
 
 	trace2_cmd_exit(result);
diff --git a/setup.c b/setup.c
index 347d7181ae9..af3b8c09abe 100644
--- a/setup.c
+++ b/setup.c
@@ -12,6 +12,7 @@ static int work_tree_config_is_bogus;
 
 static struct startup_info the_startup_info;
 struct startup_info *startup_info = &the_startup_info;
+const char *tmp_original_cwd;
 
 /*
  * The input parameter must contain an absolute path, and it must already be
@@ -432,6 +433,69 @@ void setup_work_tree(void)
 	initialized = 1;
 }
 
+static void setup_original_cwd(void)
+{
+	struct strbuf tmp = STRBUF_INIT;
+	const char *worktree = NULL;
+	int offset = -1;
+
+	if (!tmp_original_cwd)
+		return;
+
+	/*
+	 * startup_info->original_cwd points to the current working
+	 * directory we inherited from our parent process, which is a
+	 * directory we want to avoid removing.
+	 *
+	 * For convience, we would like to have the path relative to the
+	 * worktree instead of an absolute path.
+	 *
+	 * Yes, startup_info->original_cwd is usually the same as 'prefix',
+	 * but differs in two ways:
+	 *   - prefix has a trailing '/'
+	 *   - if the user passes '-C' to git, that modifies the prefix but
+	 *     not startup_info->original_cwd.
+	 */
+
+	/* Normalize the directory */
+	strbuf_realpath(&tmp, tmp_original_cwd, 1);
+	free((char*)tmp_original_cwd);
+	tmp_original_cwd = NULL;
+	startup_info->original_cwd = strbuf_detach(&tmp, NULL);
+
+	/*
+	 * Get our worktree; we only protect the current working directory
+	 * if it's in the worktree.
+	 */
+	worktree = get_git_work_tree();
+	if (!worktree)
+		goto no_prevention_needed;
+
+	offset = dir_inside_of(startup_info->original_cwd, worktree);
+	if (offset >= 0) {
+		/*
+		 * If startup_info->original_cwd == worktree, that is already
+		 * protected and we don't need original_cwd as a secondary
+		 * protection measure.
+		 */
+		if (!*(startup_info->original_cwd + offset))
+			goto no_prevention_needed;
+
+		/*
+		 * original_cwd was inside worktree; precompose it just as
+		 * we do prefix so that built up paths will match
+		 */
+		startup_info->original_cwd = \
+			precompose_string_if_needed(startup_info->original_cwd
+						    + offset);
+		return;
+	}
+
+no_prevention_needed:
+	free((char*)startup_info->original_cwd);
+	startup_info->original_cwd = NULL;
+}
+
 static int read_worktree_config(const char *var, const char *value, void *vdata)
 {
 	struct repository_format *data = vdata;
@@ -1330,6 +1394,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
 		setenv(GIT_PREFIX_ENVIRONMENT, "", 1);
 	}
 
+	setup_original_cwd();
 
 	strbuf_release(&dir);
 	strbuf_release(&gitdir);
-- 
gitgitgadget

[PATCH v6 03/11] unpack-trees: refuse to remove startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

In the past, when a directory needs to be removed to make room for a
file, we have always errored out when that directory contains any
untracked (but not ignored) files.  Add an extra condition on that: also
error out if the directory is the current working directory we inherited
from our parent process.

Acked-by: Derrick Stolee <stolee@gmail.com>
Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh | 20 +++++++-------------
 unpack-trees.c       | 17 +++++++++++++----
 unpack-trees.h       |  1 +
 3 files changed, 21 insertions(+), 17 deletions(-)

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index a05abd18187..398908dfc93 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -113,7 +113,7 @@ test_expect_success 'checkout does not clean cwd incidentally' '
 '
 
 test_expect_success 'checkout fails if cwd needs to be removed' '
-	test_required_dir_removal failure git checkout fd_conflict
+	test_required_dir_removal success git checkout fd_conflict
 '
 
 test_expect_success 'reset --hard does not clean cwd incidentally' '
@@ -144,23 +144,17 @@ test_expect_success 'merge fails if cwd needs to be removed; recursive friendly'
 	(
 		cd dirORfile &&
 
-		# We would rather this failed, but we test for existing
-		# rather than desired behavior
-		git merge fd_conflict 2>../error
+		test_must_fail git merge fd_conflict 2>../error
 	) &&
 
-	## Here is the behavior we would rather have:
-	#test_path_is_dir dirORfile &&
-	#grep "Refusing to remove the current working directory" error
-	## But instead we test for existing behavior
-	test_path_is_file dirORfile &&
-	test_must_be_empty error
+	test_path_is_dir dirORfile &&
+	grep "Refusing to remove the current working directory" error
 '
 
 GIT_TEST_MERGE_ALGORITHM=ort
 
 test_expect_success 'merge fails if cwd needs to be removed' '
-	test_required_dir_removal failure git merge fd_conflict
+	test_required_dir_removal success git merge fd_conflict
 '
 
 test_expect_success 'cherry-pick does not clean cwd incidentally' '
@@ -168,7 +162,7 @@ test_expect_success 'cherry-pick does not clean cwd incidentally' '
 '
 
 test_expect_success 'cherry-pick fails if cwd needs to be removed' '
-	test_required_dir_removal failure git cherry-pick fd_conflict
+	test_required_dir_removal success git cherry-pick fd_conflict
 '
 
 test_expect_success 'rebase does not clean cwd incidentally' '
@@ -184,7 +178,7 @@ test_expect_success 'revert does not clean cwd incidentally' '
 '
 
 test_expect_success 'revert fails if cwd needs to be removed' '
-	test_required_dir_removal failure git revert undo_fd_conflict
+	test_required_dir_removal success git revert undo_fd_conflict
 '
 
 test_expect_success 'rm does not clean cwd incidentally' '
diff --git a/unpack-trees.c b/unpack-trees.c
index 89ca95ce90b..6bc16f3a714 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -36,6 +36,9 @@ static const char *unpack_plumbing_errors[NB_UNPACK_TREES_WARNING_TYPES] = {
 	/* ERROR_NOT_UPTODATE_DIR */
 	"Updating '%s' would lose untracked files in it",
 
+	/* ERROR_CWD_IN_THE_WAY */
+	"Refusing to remove '%s' since it is the current working directory.",
+
 	/* ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN */
 	"Untracked working tree file '%s' would be overwritten by merge.",
 
@@ -131,6 +134,9 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 	msgs[ERROR_NOT_UPTODATE_DIR] =
 		_("Updating the following directories would lose untracked files in them:\n%s");
 
+	msgs[ERROR_CWD_IN_THE_WAY] =
+		_("Refusing to remove the current working directory:\n%s");
+
 	if (!strcmp(cmd, "checkout"))
 		msg = advice_enabled(ADVICE_COMMIT_BEFORE_MERGE)
 		      ? _("The following untracked working tree files would be removed by checkout:\n%%s"
@@ -2146,10 +2152,7 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,
 		cnt++;
 	}
 
-	/*
-	 * Then we need to make sure that we do not lose a locally
-	 * present file that is not ignored.
-	 */
+	/* Do not lose a locally present file that is not ignored. */
 	pathbuf = xstrfmt("%.*s/", namelen, ce->name);
 
 	memset(&d, 0, sizeof(d));
@@ -2160,6 +2163,12 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,
 	free(pathbuf);
 	if (i)
 		return add_rejected_path(o, ERROR_NOT_UPTODATE_DIR, ce->name);
+
+	/* Do not lose startup_info->original_cwd */
+	if (startup_info->original_cwd &&
+	    !strcmp(startup_info->original_cwd, ce->name))
+		return add_rejected_path(o, ERROR_CWD_IN_THE_WAY, ce->name);
+
 	return cnt;
 }
 
diff --git a/unpack-trees.h b/unpack-trees.h
index 71ffb7eeb0c..efb9edfbb27 100644
--- a/unpack-trees.h
+++ b/unpack-trees.h
@@ -19,6 +19,7 @@ enum unpack_trees_error_types {
 	ERROR_WOULD_OVERWRITE = 0,
 	ERROR_NOT_UPTODATE_FILE,
 	ERROR_NOT_UPTODATE_DIR,
+	ERROR_CWD_IN_THE_WAY,
 	ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN,
 	ERROR_WOULD_LOSE_UNTRACKED_REMOVED,
 	ERROR_BIND_OVERLAP,
-- 
gitgitgadget

[PATCH v6 04/11] unpack-trees: add special cwd handling

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

When running commands such as `git reset --hard` from a subdirectory, if
that subdirectory is in the way of adding needed files, bail with an
error message.

Note that this change looks kind of like it duplicates the new lines of
code from the previous commit in verify_clean_subdirectory().  However,
when we are preserving untracked files, we would rather any error
messages about untracked files being in the way take precedence over
error messages about a subdirectory that happens to be the_original_cwd
being in the way.  But in the UNPACK_RESET_OVERWRITE_UNTRACKED case,
there is no untracked checking to be done, so we simply add a special
case near the top of verify_absent_1.

Acked-by: Derrick Stolee <stolee@gmail.com>
Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh |  2 +-
 unpack-trees.c       | 13 +++++++++++--
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 398908dfc93..5af1fec6fec 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -121,7 +121,7 @@ test_expect_success 'reset --hard does not clean cwd incidentally' '
 '
 
 test_expect_success 'reset --hard fails if cwd needs to be removed' '
-	test_required_dir_removal failure git reset --hard fd_conflict
+	test_required_dir_removal success git reset --hard fd_conflict
 '
 
 test_expect_success 'merge does not clean cwd incidentally' '
diff --git a/unpack-trees.c b/unpack-trees.c
index 6bc16f3a714..5852807d2fb 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -2261,10 +2261,19 @@ static int verify_absent_1(const struct cache_entry *ce,
 	int len;
 	struct stat st;
 
-	if (o->index_only || !o->update ||
-	    o->reset == UNPACK_RESET_OVERWRITE_UNTRACKED)
+	if (o->index_only || !o->update)
 		return 0;
 
+	if (o->reset == UNPACK_RESET_OVERWRITE_UNTRACKED) {
+		/* Avoid nuking startup_info->original_cwd... */
+		if (startup_info->original_cwd &&
+		    !strcmp(startup_info->original_cwd, ce->name))
+			return add_rejected_path(o, ERROR_CWD_IN_THE_WAY,
+						 ce->name);
+		/* ...but nuke anything else. */
+		return 0;
+	}
+
 	len = check_leading_path(ce->name, ce_namelen(ce), 0);
 	if (!len)
 		return 0;
-- 
gitgitgadget

[PATCH v6 05/11] symlinks: do not include startup_info->original_cwd in dir removal

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

symlinks has a pair of schedule_dir_for_removal() and
remove_scheduled_dirs() functions that ensure that directories made
empty by removing other files also themselves get removed.  However, we
want to exclude startup_info->original_cwd and leave it around.  This
avoids the user getting confused by subsequent git commands (and non-git
commands) that would otherwise report confusing messages about being
unable to read the current working directory.

Acked-by: Derrick Stolee <stolee@gmail.com>
Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 symlinks.c           |  8 +++++++-
 t/t2501-cwd-empty.sh | 10 +++++-----
 2 files changed, 12 insertions(+), 6 deletions(-)

diff --git a/symlinks.c b/symlinks.c
index 5232d02020c..c667baa949b 100644
--- a/symlinks.c
+++ b/symlinks.c
@@ -279,7 +279,9 @@ static void do_remove_scheduled_dirs(int new_len)
 {
 	while (removal.len > new_len) {
 		removal.buf[removal.len] = '\0';
-		if (rmdir(removal.buf))
+		if ((startup_info->original_cwd &&
+		     !strcmp(removal.buf, startup_info->original_cwd)) ||
+		    rmdir(removal.buf))
 			break;
 		do {
 			removal.len--;
@@ -293,6 +295,10 @@ void schedule_dir_for_removal(const char *name, int len)
 {
 	int match_len, last_slash, i, previous_slash;
 
+	if (startup_info->original_cwd &&
+	    !strcmp(name, startup_info->original_cwd))
+		return;	/* Do not remove the current working directory */
+
 	match_len = last_slash = i =
 		longest_path_match(name, len, removal.buf, removal.len,
 				   &previous_slash);
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 5af1fec6fec..e4502d24d57 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -109,7 +109,7 @@ test_required_dir_removal () {
 }
 
 test_expect_success 'checkout does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git checkout init
+	test_incidental_dir_removal success git checkout init
 '
 
 test_expect_success 'checkout fails if cwd needs to be removed' '
@@ -117,7 +117,7 @@ test_expect_success 'checkout fails if cwd needs to be removed' '
 '
 
 test_expect_success 'reset --hard does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git reset --hard init
+	test_incidental_dir_removal success git reset --hard init
 '
 
 test_expect_success 'reset --hard fails if cwd needs to be removed' '
@@ -125,7 +125,7 @@ test_expect_success 'reset --hard fails if cwd needs to be removed' '
 '
 
 test_expect_success 'merge does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git merge reverted
+	test_incidental_dir_removal success git merge reverted
 '
 
 # This file uses some simple merges where
@@ -158,7 +158,7 @@ test_expect_success 'merge fails if cwd needs to be removed' '
 '
 
 test_expect_success 'cherry-pick does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git cherry-pick reverted
+	test_incidental_dir_removal success git cherry-pick reverted
 '
 
 test_expect_success 'cherry-pick fails if cwd needs to be removed' '
@@ -174,7 +174,7 @@ test_expect_success 'rebase fails if cwd needs to be removed' '
 '
 
 test_expect_success 'revert does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git revert HEAD
+	test_incidental_dir_removal success git revert HEAD
 '
 
 test_expect_success 'revert fails if cwd needs to be removed' '
-- 
gitgitgadget

[PATCH v6 06/11] clean: do not attempt to remove startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Acked-by: Derrick Stolee <stolee@gmail.com>
Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/clean.c      | 44 +++++++++++++++++++++++++++++++++++---------
 t/t2501-cwd-empty.sh |  5 +++--
 2 files changed, 38 insertions(+), 11 deletions(-)

diff --git a/builtin/clean.c b/builtin/clean.c
index 98a2860409b..3ff02bbbffe 100644
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@ -36,6 +36,8 @@ static const char *msg_skip_git_dir = N_("Skipping repository %s\n");
 static const char *msg_would_skip_git_dir = N_("Would skip repository %s\n");
 static const char *msg_warn_remove_failed = N_("failed to remove %s");
 static const char *msg_warn_lstat_failed = N_("could not lstat %s\n");
+static const char *msg_skip_cwd = N_("Refusing to remove current working directory\n");
+static const char *msg_would_skip_cwd = N_("Would refuse to remove current working directory\n");
 
 enum color_clean {
 	CLEAN_COLOR_RESET = 0,
@@ -153,6 +155,8 @@ static int remove_dirs(struct strbuf *path, const char *prefix, int force_flag,
 {
 	DIR *dir;
 	struct strbuf quoted = STRBUF_INIT;
+	struct strbuf realpath = STRBUF_INIT;
+	struct strbuf real_ocwd = STRBUF_INIT;
 	struct dirent *e;
 	int res = 0, ret = 0, gone = 1, original_len = path->len, len;
 	struct string_list dels = STRING_LIST_INIT_DUP;
@@ -231,16 +235,36 @@ static int remove_dirs(struct strbuf *path, const char *prefix, int force_flag,
 	strbuf_setlen(path, original_len);
 
 	if (*dir_gone) {
-		res = dry_run ? 0 : rmdir(path->buf);
-		if (!res)
-			*dir_gone = 1;
-		else {
-			int saved_errno = errno;
-			quote_path(path->buf, prefix, &quoted, 0);
-			errno = saved_errno;
-			warning_errno(_(msg_warn_remove_failed), quoted.buf);
+		/*
+		 * Normalize path components in path->buf, e.g. change '\' to
+		 * '/' on Windows.
+		 */
+		strbuf_realpath(&realpath, path->buf, 1);
+
+		/*
+		 * path and realpath are absolute; for comparison, we would
+		 * like to transform startup_info->original_cwd to an absolute
+		 * path too.
+		 */
+		 if (startup_info->original_cwd)
+			 strbuf_realpath(&real_ocwd,
+					 startup_info->original_cwd, 1);
+
+		if (!strbuf_cmp(&realpath, &real_ocwd)) {
+			printf("%s", dry_run ? _(msg_would_skip_cwd) : _(msg_skip_cwd));
 			*dir_gone = 0;
-			ret = 1;
+		} else {
+			res = dry_run ? 0 : rmdir(path->buf);
+			if (!res)
+				*dir_gone = 1;
+			else {
+				int saved_errno = errno;
+				quote_path(path->buf, prefix, &quoted, 0);
+				errno = saved_errno;
+				warning_errno(_(msg_warn_remove_failed), quoted.buf);
+				*dir_gone = 0;
+				ret = 1;
+			}
 		}
 	}
 
@@ -250,6 +274,8 @@ static int remove_dirs(struct strbuf *path, const char *prefix, int force_flag,
 			printf(dry_run ?  _(msg_would_remove) : _(msg_remove), dels.items[i].string);
 	}
 out:
+	strbuf_release(&realpath);
+	strbuf_release(&real_ocwd);
 	strbuf_release(&quoted);
 	string_list_clear(&dels, 0);
 	return ret;
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index e4502d24d57..b1182390ba3 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -230,8 +230,9 @@ test_incidental_untracked_dir_removal () {
 }
 
 test_expect_success 'clean does not remove cwd incidentally' '
-	test_incidental_untracked_dir_removal failure \
-		git -C .. clean -fd -e warnings . >warnings
+	test_incidental_untracked_dir_removal success \
+		git -C .. clean -fd -e warnings . >warnings &&
+	grep "Refusing to remove current working directory" warnings
 '
 
 test_expect_success 'stash does not remove cwd incidentally' '
-- 
gitgitgadget

[PATCH v6 07/11] rebase: do not attempt to remove startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Since rebase spawns a `checkout` subprocess, make sure we run that from
the startup_info->original_cwd directory, so that the checkout process
knows to protect that directory.

Acked-by: Derrick Stolee <stolee@gmail.com>
Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 sequencer.c          | 2 ++
 t/t2501-cwd-empty.sh | 4 ++--
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index ea96837cde3..83f257e7fa4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -4228,6 +4228,8 @@ static int run_git_checkout(struct repository *r, struct replay_opts *opts,
 
 	cmd.git_cmd = 1;
 
+	if (startup_info->original_cwd)
+		cmd.dir = startup_info->original_cwd;
 	strvec_push(&cmd.args, "checkout");
 	strvec_push(&cmd.args, commit);
 	strvec_pushf(&cmd.env_array, GIT_REFLOG_ACTION "=%s", action);
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index b1182390ba3..52335a8afe9 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -166,11 +166,11 @@ test_expect_success 'cherry-pick fails if cwd needs to be removed' '
 '
 
 test_expect_success 'rebase does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git rebase reverted
+	test_incidental_dir_removal success git rebase reverted
 '
 
 test_expect_success 'rebase fails if cwd needs to be removed' '
-	test_required_dir_removal failure git rebase fd_conflict
+	test_required_dir_removal success git rebase fd_conflict
 '
 
 test_expect_success 'revert does not clean cwd incidentally' '
-- 
gitgitgadget

[PATCH v6 08/11] stash: do not attempt to remove startup_info->original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Since stash spawns a `clean` subprocess, make sure we run that from the
startup_info->original_cwd directory, so that the `clean` processs knows
to protect that directory.  Also, since the `clean` command might no
longer run from the toplevel, pass the ':/' magic pathspec to ensure we
still clean from the toplevel.

Acked-by: Derrick Stolee <stolee@gmail.com>
Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/stash.c      | 4 +++-
 t/t2501-cwd-empty.sh | 2 +-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/builtin/stash.c b/builtin/stash.c
index a0ccc8654df..de0e432a4ff 100644
--- a/builtin/stash.c
+++ b/builtin/stash.c
@@ -1485,8 +1485,10 @@ static int do_push_stash(const struct pathspec *ps, const char *stash_msg, int q
 			struct child_process cp = CHILD_PROCESS_INIT;
 
 			cp.git_cmd = 1;
+			if (startup_info->original_cwd)
+				cp.dir = startup_info->original_cwd;
 			strvec_pushl(&cp.args, "clean", "--force",
-				     "--quiet", "-d", NULL);
+				     "--quiet", "-d", ":/", NULL);
 			if (include_untracked == INCLUDE_ALL_FILES)
 				strvec_push(&cp.args, "-x");
 			if (run_command(&cp)) {
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index 52335a8afe9..be9ef903bd4 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -236,7 +236,7 @@ test_expect_success 'clean does not remove cwd incidentally' '
 '
 
 test_expect_success 'stash does not remove cwd incidentally' '
-	test_incidental_untracked_dir_removal failure \
+	test_incidental_untracked_dir_removal success \
 		git stash --include-untracked
 '
 
-- 
gitgitgadget

[PATCH v6 09/11] dir: avoid incidentally removing the original_cwd in remove_path()

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Modern git often tries to avoid leaving empty directories around when
removing files.  Originally, it did not bother.  This behavior started
with commit 80e21a9ed809 (merge-recursive::removeFile: remove empty
directories, 2005-11-19), stating the reason simply as:

    When the last file in a directory is removed as the result of a
    merge, try to rmdir the now-empty directory.

This was reimplemented in C and renamed to remove_path() in commit
e1b3a2cad7 ("Build-in merge-recursive", 2008-02-07), but was still
internal to merge-recursive.

This trend towards removing leading empty directories continued with
commit d9b814cc97f1 (Add builtin "git rm" command, 2006-05-19), which
stated the reasoning as:

    The other question is what to do with leading directories. The old
    "git rm" script didn't do anything, which is somewhat inconsistent.
    This one will actually clean up directories that have become empty
    as a result of removing the last file, but maybe we want to have a
    flag to decide the behaviour?

remove_path() in dir.c was added in 4a92d1bfb784 (Add remove_path: a
function to remove as much as possible of a path, 2008-09-27), because
it was noted that we had two separate implementations of the same idea
AND both were buggy.  It described the purpose of the function as

    a function to remove as much as possible of a path

Why remove as much as possible?  Well, at the time we probably would
have said something like:

  * removing leading directories makes things feel tidy
  * removing leading directories doesn't hurt anything so long as they
    had no files in them.

But I don't believe those reasons hold when the empty directory happens
to be the current working directory we inherited from our parent
process.  Leaving the parent process in a deleted directory can cause
user confusion when subsequent processes fail: any git command, for
example, will immediately fail with

    fatal: Unable to read current working directory: No such file or directory

Other commands may similarly get confused.  Modify remove_path() so that
the empty leading directories it also deletes does not include the
current working directory we inherited from our parent process.  I have
looked through every caller of remove_path() in the current codebase to
make sure that all should take this change.

Acked-by: Derrick Stolee <stolee@gmail.com>
Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 dir.c                |  3 +++
 dir.h                |  6 +++++-
 t/t2501-cwd-empty.sh | 12 ++++--------
 3 files changed, 12 insertions(+), 9 deletions(-)

diff --git a/dir.c b/dir.c
index 94489298f4c..97d6b71c872 100644
--- a/dir.c
+++ b/dir.c
@@ -3327,6 +3327,9 @@ int remove_path(const char *name)
 		slash = dirs + (slash - name);
 		do {
 			*slash = '\0';
+			if (startup_info->original_cwd &&
+			    !strcmp(startup_info->original_cwd, dirs))
+				break;
 		} while (rmdir(dirs) == 0 && (slash = strrchr(dirs, '/')));
 		free(dirs);
 	}
diff --git a/dir.h b/dir.h
index 83f46c0fb4c..d6a5d03bec2 100644
--- a/dir.h
+++ b/dir.h
@@ -504,7 +504,11 @@ int get_sparse_checkout_patterns(struct pattern_list *pl);
  */
 int remove_dir_recursively(struct strbuf *path, int flag);
 
-/* tries to remove the path with empty directories along it, ignores ENOENT */
+/*
+ * Tries to remove the path, along with leading empty directories so long as
+ * those empty directories are not startup_info->original_cwd.  Ignores
+ * ENOENT.
+ */
 int remove_path(const char *path);
 
 int fspathcmp(const char *a, const char *b);
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index be9ef903bd4..ce2efb9d30a 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -182,12 +182,12 @@ test_expect_success 'revert fails if cwd needs to be removed' '
 '
 
 test_expect_success 'rm does not clean cwd incidentally' '
-	test_incidental_dir_removal failure git rm bar/baz.t
+	test_incidental_dir_removal success git rm bar/baz.t
 '
 
 test_expect_success 'apply does not remove cwd incidentally' '
 	git diff HEAD HEAD~1 >patch &&
-	test_incidental_dir_removal failure git apply ../patch
+	test_incidental_dir_removal success git apply ../patch
 '
 
 test_incidental_untracked_dir_removal () {
@@ -271,12 +271,8 @@ test_expect_success '`rm -rf dir` even with only tracked files will remove somet
 	) &&
 
 	test_path_is_missing a/b/c/tracked &&
-	## We would prefer if a/b was still present, though empty, since it
-	## was the current working directory
-	#test_path_is_dir a/b
-	## But the current behavior is that it not only deletes the directory
-	## a/b as requested, but also goes and deletes a
-	test_path_is_missing a
+	test_path_is_missing a/b/c &&
+	test_path_is_dir a/b
 '
 
 test_expect_success 'git version continues working from a deleted dir' '
-- 
gitgitgadget

[PATCH v6 10/11] dir: new flag to remove_dir_recurse() to spare the original_cwd

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

remove_dir_recurse(), and its non-static wrapper called
remove_dir_recursively(), both take flags for modifying its behavior.
As with the previous commits, we would generally like to protect
the original_cwd, but we want to forced user commands (e.g. 'git rm -rf
...') or other special cases to remove it.  Add a flag for this purpose.
After reading through every caller of remove_dir_recursively() in the
current codebase, there was only one that should be adjusted and that
one only in a very unusual circumstance.  Add a pair of new testcases to
highlight that very specific case involving submodules && --git-dir &&
--work-tree.

Acked-by: Derrick Stolee <stolee@gmail.com>
Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/rm.c         |  3 ++-
 dir.c                | 12 +++++++++---
 dir.h                |  3 +++
 t/t2501-cwd-empty.sh |  5 -----
 4 files changed, 14 insertions(+), 9 deletions(-)

diff --git a/builtin/rm.c b/builtin/rm.c
index 3d0967cdc11..b4132e5d8ee 100644
--- a/builtin/rm.c
+++ b/builtin/rm.c
@@ -399,12 +399,13 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
 	if (!index_only) {
 		int removed = 0, gitmodules_modified = 0;
 		struct strbuf buf = STRBUF_INIT;
+		int flag = force ? REMOVE_DIR_PURGE_ORIGINAL_CWD : 0;
 		for (i = 0; i < list.nr; i++) {
 			const char *path = list.entry[i].name;
 			if (list.entry[i].is_submodule) {
 				strbuf_reset(&buf);
 				strbuf_addstr(&buf, path);
-				if (remove_dir_recursively(&buf, 0))
+				if (remove_dir_recursively(&buf, flag))
 					die(_("could not remove '%s'"), path);
 
 				removed = 1;
diff --git a/dir.c b/dir.c
index 97d6b71c872..52064345a6b 100644
--- a/dir.c
+++ b/dir.c
@@ -3204,6 +3204,7 @@ static int remove_dir_recurse(struct strbuf *path, int flag, int *kept_up)
 	int ret = 0, original_len = path->len, len, kept_down = 0;
 	int only_empty = (flag & REMOVE_DIR_EMPTY_ONLY);
 	int keep_toplevel = (flag & REMOVE_DIR_KEEP_TOPLEVEL);
+	int purge_original_cwd = (flag & REMOVE_DIR_PURGE_ORIGINAL_CWD);
 	struct object_id submodule_head;
 
 	if ((flag & REMOVE_DIR_KEEP_NESTED_GIT) &&
@@ -3259,9 +3260,14 @@ static int remove_dir_recurse(struct strbuf *path, int flag, int *kept_up)
 	closedir(dir);
 
 	strbuf_setlen(path, original_len);
-	if (!ret && !keep_toplevel && !kept_down)
-		ret = (!rmdir(path->buf) || errno == ENOENT) ? 0 : -1;
-	else if (kept_up)
+	if (!ret && !keep_toplevel && !kept_down) {
+		if (!purge_original_cwd &&
+		    startup_info->original_cwd &&
+		    !strcmp(startup_info->original_cwd, path->buf))
+			ret = -1; /* Do not remove current working directory */
+		else
+			ret = (!rmdir(path->buf) || errno == ENOENT) ? 0 : -1;
+	} else if (kept_up)
 		/*
 		 * report the uplevel that it is not an error that we
 		 * did not rmdir() our directory.
diff --git a/dir.h b/dir.h
index d6a5d03bec2..8e02dfb505d 100644
--- a/dir.h
+++ b/dir.h
@@ -495,6 +495,9 @@ int get_sparse_checkout_patterns(struct pattern_list *pl);
 /* Remove the contents of path, but leave path itself. */
 #define REMOVE_DIR_KEEP_TOPLEVEL 04
 
+/* Remove the_original_cwd too */
+#define REMOVE_DIR_PURGE_ORIGINAL_CWD 0x08
+
 /*
  * Remove path and its contents, recursively. flags is a combination
  * of the above REMOVE_DIR_* constants. Return 0 on success.
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index ce2efb9d30a..bc92230f2f2 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -291,11 +291,6 @@ test_submodule_removal () {
 	test_status=
 	test "$path_status" = dir && test_status=test_must_fail
 
-	# Actually, while path_status=dir && test_status=test_must_fail
-	# reflect our desired behavior, current behavior is:
-	path_status=missing
-	test_status=
-
 	test_when_finished "git reset --hard HEAD~1" &&
 	test_when_finished "rm -rf .git/modules/my_submodule" &&
 
-- 
gitgitgadget

[PATCH v6 11/11] t2501: simplify the tests since we can now assume desired behavior

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

We no longer are dealing with a mixture of previous and desired
behavior, so simplify the tests a bit.

Acked-by: Derrick Stolee <stolee@gmail.com>
Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t2501-cwd-empty.sh | 123 +++++++++++++------------------------------
 1 file changed, 36 insertions(+), 87 deletions(-)

diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index bc92230f2f2..f6d8d7d03d7 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -32,9 +32,6 @@ test_expect_success setup '
 '
 
 test_incidental_dir_removal () {
-	works=$1 &&
-	shift &&
-
 	test_when_finished "git reset --hard" &&
 
 	git checkout foo/bar/baz^{commit} &&
@@ -44,88 +41,57 @@ test_incidental_dir_removal () {
 		cd foo &&
 		"$@" &&
 
-		# Although we want pwd & git status to pass, test for existing
-		# rather than desired behavior.
-		if test "$works" = "success"
-		then
-			test-tool getcwd &&
-			git status --porcelain
-		else
-			! test-tool getcwd &&
-			test_might_fail git status --porcelain
-		fi
+		# Make sure foo still exists, and commands needing it work
+		test-tool getcwd &&
+		git status --porcelain
 	) &&
 	test_path_is_missing foo/bar/baz &&
 	test_path_is_missing foo/bar &&
 
-	# Although we want dir to be present, test for existing rather
-	# than desired behavior.
-	if test "$works" = "success"
-	then
-		test_path_is_dir foo
-	else
-		test_path_is_missing foo
-	fi
+	test_path_is_dir foo
 }
 
 test_required_dir_removal () {
-	works=$1 &&
-	shift &&
-
 	git checkout df_conflict^{commit} &&
 	test_when_finished "git clean -fdx" &&
 
 	(
 		cd dirORfile &&
 
-		# We'd like for the command to fail (much as it would if there
-		# was an untracked file there), and for the index and worktree
-		# to be left clean with pwd and git status working afterwards.
-		# But test for existing rather than desired behavior.
-		if test "$works" = "success"
-		then
-			test_must_fail "$@" 2>../error &&
-			grep "Refusing to remove.*current working directory" ../error &&
-
-			git diff --exit-code HEAD &&
-
-			test-tool getcwd &&
-			git status --porcelain
-		else
-			"$@" &&
-			! test-tool getcwd &&
-			test_might_fail git status --porcelain
-		fi
+		# Ensure command refuses to run
+		test_must_fail "$@" 2>../error &&
+		grep "Refusing to remove.*current working directory" ../error &&
+
+		# ...and that the index and working tree are left clean
+		git diff --exit-code HEAD &&
+
+		# Ensure that getcwd and git status do not error out (which
+		# they might if the current working directory had been removed)
+		test-tool getcwd &&
+		git status --porcelain
 	) &&
 
-	# Although we want dirORfile to be present, test for existing rather
-	# than desired behavior.
-	if test "$works" = "success"
-	then
-		test_path_is_dir dirORfile
-	else
-		test_path_is_file dirORfile
-	fi
+	test_path_is_dir dirORfile
 }
 
 test_expect_success 'checkout does not clean cwd incidentally' '
-	test_incidental_dir_removal success git checkout init
+	test_incidental_dir_removal git checkout init
 '
 
 test_expect_success 'checkout fails if cwd needs to be removed' '
-	test_required_dir_removal success git checkout fd_conflict
+	test_required_dir_removal git checkout fd_conflict
 '
 
 test_expect_success 'reset --hard does not clean cwd incidentally' '
-	test_incidental_dir_removal success git reset --hard init
+	test_incidental_dir_removal git reset --hard init
 '
 
 test_expect_success 'reset --hard fails if cwd needs to be removed' '
-	test_required_dir_removal success git reset --hard fd_conflict
+	test_required_dir_removal git reset --hard fd_conflict
 '
 
 test_expect_success 'merge does not clean cwd incidentally' '
-	test_incidental_dir_removal success git merge reverted
+	test_incidental_dir_removal git merge reverted
 '
 
 # This file uses some simple merges where
@@ -154,46 +120,43 @@ test_expect_success 'merge fails if cwd needs to be removed; recursive friendly'
 GIT_TEST_MERGE_ALGORITHM=ort
 
 test_expect_success 'merge fails if cwd needs to be removed' '
-	test_required_dir_removal success git merge fd_conflict
+	test_required_dir_removal git merge fd_conflict
 '
 
 test_expect_success 'cherry-pick does not clean cwd incidentally' '
-	test_incidental_dir_removal success git cherry-pick reverted
+	test_incidental_dir_removal git cherry-pick reverted
 '
 
 test_expect_success 'cherry-pick fails if cwd needs to be removed' '
-	test_required_dir_removal success git cherry-pick fd_conflict
+	test_required_dir_removal git cherry-pick fd_conflict
 '
 
 test_expect_success 'rebase does not clean cwd incidentally' '
-	test_incidental_dir_removal success git rebase reverted
+	test_incidental_dir_removal git rebase reverted
 '
 
 test_expect_success 'rebase fails if cwd needs to be removed' '
-	test_required_dir_removal success git rebase fd_conflict
+	test_required_dir_removal git rebase fd_conflict
 '
 
 test_expect_success 'revert does not clean cwd incidentally' '
-	test_incidental_dir_removal success git revert HEAD
+	test_incidental_dir_removal git revert HEAD
 '
 
 test_expect_success 'revert fails if cwd needs to be removed' '
-	test_required_dir_removal success git revert undo_fd_conflict
+	test_required_dir_removal git revert undo_fd_conflict
 '
 
 test_expect_success 'rm does not clean cwd incidentally' '
-	test_incidental_dir_removal success git rm bar/baz.t
+	test_incidental_dir_removal git rm bar/baz.t
 '
 
 test_expect_success 'apply does not remove cwd incidentally' '
 	git diff HEAD HEAD~1 >patch &&
-	test_incidental_dir_removal success git apply ../patch
+	test_incidental_dir_removal git apply ../patch
 '
 
 test_incidental_untracked_dir_removal () {
-	works=$1 &&
-	shift &&
-
 	test_when_finished "git reset --hard" &&
 
 	git checkout foo/bar/baz^{commit} &&
@@ -205,38 +168,24 @@ test_incidental_untracked_dir_removal () {
 		cd untracked &&
 		"$@" &&
 
-		# Although we want pwd & git status to pass, test for existing
-		# rather than desired behavior.
-		if test "$works" = "success"
-		then
-			test-tool getcwd &&
-			git status --porcelain
-		else
-			! test-tool getcwd &&
-			test_might_fail git status --porcelain
-		fi
+		# Make sure untracked still exists, and commands needing it work
+		test-tool getcwd &&
+		git status --porcelain
 	) &&
 	test_path_is_missing empty &&
 	test_path_is_missing untracked/random &&
 
-	# Although we want dir to be present, test for existing rather
-	# than desired behavior.
-	if test "$works" = "success"
-	then
-		test_path_is_dir untracked
-	else
-		test_path_is_missing untracked
-	fi
+	test_path_is_dir untracked
 }
 
 test_expect_success 'clean does not remove cwd incidentally' '
-	test_incidental_untracked_dir_removal success \
+	test_incidental_untracked_dir_removal \
 		git -C .. clean -fd -e warnings . >warnings &&
 	grep "Refusing to remove current working directory" warnings
 '
 
 test_expect_success 'stash does not remove cwd incidentally' '
-	test_incidental_untracked_dir_removal success \
+	test_incidental_untracked_dir_removal \
 		git stash --include-untracked
 '
 
-- 
gitgitgadget

[Bug] Rebase from worktree subdir is broken (was Re: [PATCH v5 07/11] rebase: do not attempt to remove startup_info->original_cwd)

[Glen Choo]

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Elijah Newren <newren@gmail.com>
>
> Since rebase spawns a `checkout` subprocess, make sure we run that from
> the startup_info->original_cwd directory, so that the checkout process
> knows to protect that directory.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  sequencer.c          | 2 ++
>  t/t2501-cwd-empty.sh | 4 ++--
>  2 files changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index ea96837cde3..83f257e7fa4 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -4228,6 +4228,8 @@ static int run_git_checkout(struct repository *r, struct replay_opts *opts,
>  
>  	cmd.git_cmd = 1;
>  
> +	if (startup_info->original_cwd)
> +		cmd.dir = startup_info->original_cwd;
>  	strvec_push(&cmd.args, "checkout");
>  	strvec_push(&cmd.args, commit);
>  	strvec_pushf(&cmd.env_array, GIT_REFLOG_ACTION "=%s", action);
> diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
> index b1182390ba3..52335a8afe9 100755
> --- a/t/t2501-cwd-empty.sh
> +++ b/t/t2501-cwd-empty.sh
> @@ -166,11 +166,11 @@ test_expect_success 'cherry-pick fails if cwd needs to be removed' '
>  '
>  
>  test_expect_success 'rebase does not clean cwd incidentally' '
> -	test_incidental_dir_removal failure git rebase reverted
> +	test_incidental_dir_removal success git rebase reverted
>  '
>  
>  test_expect_success 'rebase fails if cwd needs to be removed' '
> -	test_required_dir_removal failure git rebase fd_conflict
> +	test_required_dir_removal success git rebase fd_conflict
>  '
>  
>  test_expect_success 'revert does not clean cwd incidentally' '
> -- 
> gitgitgadget

This commit (which is already in master) introduces a bug that breaks
rebase when rebasing inside a subdirectory of a worktree. You can see
that the below test fails with:

  error: The following untracked working tree files would be overwritten by merge:
          a/b/c
  Please move or remove them before you merge.

This only affects subdirectories in worktrees, i.e. rebasing anywhere in
the `main-wt` directory is fine, and rebasing from the top of `other-wt`
is fine, but `other-wt/any/other/dir` fails.

I haven't tracked down the root cause yet, but judging from the commit,
I would suppose that the checkout is being spawned in the wrong
directory, causing the files to not be cleaned up.

---
 t/t3400-rebase.sh | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/t/t3400-rebase.sh b/t/t3400-rebase.sh
index 23dbd3c82e..8b8b66538b 100755
--- a/t/t3400-rebase.sh
+++ b/t/t3400-rebase.sh
@@ -416,4 +416,33 @@ test_expect_success MINGW,SYMLINKS_WINDOWS 'rebase when .git/logs is a symlink'
 	mv actual_logs .git/logs
 '
 
+test_expect_success 'rebase when inside worktree subdirectory' '
+	git init main-wt &&
+	(
+		cd main-wt &&
+		git commit --allow-empty -m "initial" &&
+		# create commit with foo/bar/baz
+		mkdir -p foo/bar &&
+		touch foo/bar/baz &&
+		git add foo/bar/baz &&
+		git commit -m "add foo/bar/baz" &&
+		# create commit with a/b/c
+		mkdir -p a/b &&
+		touch a/b/c &&
+		git add a/b/c &&
+		git commit -m "add a/b/c" &&
+		# create another branch for our other worktree
+		git branch other &&
+		git worktree add ../other-wt other &&
+		(
+			cd ../other-wt &&
+			mkdir -p random/dir &&
+			(
+				cd random/dir &&
+				git rebase --onto HEAD^^ HEAD^  # drops the HEAD^ commit
+			)
+		)
+	)
+'
+
 test_done
-- 
2.35.0.rc0.227.g00780c9af4-goog

Re: [Bug] Rebase from worktree subdir is broken (was Re: [PATCH v5 07/11] rebase: do not attempt to remove startup_info->original_cwd)

[Elijah Newren]

On Tue, Jan 25, 2022 at 12:27 PM Glen Choo <chooglen@google.com> wrote:
>
> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > Since rebase spawns a `checkout` subprocess, make sure we run that from
> > the startup_info->original_cwd directory, so that the checkout process
> > knows to protect that directory.
> >
> > Signed-off-by: Elijah Newren <newren@gmail.com>
> > ---
> >  sequencer.c          | 2 ++
> >  t/t2501-cwd-empty.sh | 4 ++--
> >  2 files changed, 4 insertions(+), 2 deletions(-)
> >
> > diff --git a/sequencer.c b/sequencer.c
> > index ea96837cde3..83f257e7fa4 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -4228,6 +4228,8 @@ static int run_git_checkout(struct repository *r, struct replay_opts *opts,
> >
> >       cmd.git_cmd = 1;
> >
> > +     if (startup_info->original_cwd)
> > +             cmd.dir = startup_info->original_cwd;
> >       strvec_push(&cmd.args, "checkout");
> >       strvec_push(&cmd.args, commit);
> >       strvec_pushf(&cmd.env_array, GIT_REFLOG_ACTION "=%s", action);
> > diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
> > index b1182390ba3..52335a8afe9 100755
> > --- a/t/t2501-cwd-empty.sh
> > +++ b/t/t2501-cwd-empty.sh
> > @@ -166,11 +166,11 @@ test_expect_success 'cherry-pick fails if cwd needs to be removed' '
> >  '
> >
> >  test_expect_success 'rebase does not clean cwd incidentally' '
> > -     test_incidental_dir_removal failure git rebase reverted
> > +     test_incidental_dir_removal success git rebase reverted
> >  '
> >
> >  test_expect_success 'rebase fails if cwd needs to be removed' '
> > -     test_required_dir_removal failure git rebase fd_conflict
> > +     test_required_dir_removal success git rebase fd_conflict
> >  '
> >
> >  test_expect_success 'revert does not clean cwd incidentally' '
> > --
> > gitgitgadget
>
> This commit (which is already in master) introduces a bug that breaks
> rebase when rebasing inside a subdirectory of a worktree. You can see
> that the below test fails with:
>
>   error: The following untracked working tree files would be overwritten by merge:
>           a/b/c
>   Please move or remove them before you merge.

Thanks for the detailed report -- with a full testcase!

> This only affects subdirectories in worktrees, i.e. rebasing anywhere in
> the `main-wt` directory is fine, and rebasing from the top of `other-wt`
> is fine, but `other-wt/any/other/dir` fails.
>
> I haven't tracked down the root cause yet, but judging from the commit,
> I would suppose that the checkout is being spawned in the wrong
> directory, causing the files to not be cleaned up.

There's nothing wrong with running checkout from a subdirectory.  It
is unfortunate that setup.c auto-discovers both the git directory and
the working tree, but sets GIT_DIR without setting GIT_WORK_TREE in
the case of a non-main worktree; it's not particularly friendly for
subcommands.  Of course, it's also unfortunate that sequencer still
forks subprocesses other than those requested by a user with e.g.
--exec.

But, anyway, I've got a patch that I'll send as soon as it passes CI
(https://github.com/git/git/pull/1205).

> ---
>  t/t3400-rebase.sh | 29 +++++++++++++++++++++++++++++
>  1 file changed, 29 insertions(+)
>
> diff --git a/t/t3400-rebase.sh b/t/t3400-rebase.sh
> index 23dbd3c82e..8b8b66538b 100755
> --- a/t/t3400-rebase.sh
> +++ b/t/t3400-rebase.sh
> @@ -416,4 +416,33 @@ test_expect_success MINGW,SYMLINKS_WINDOWS 'rebase when .git/logs is a symlink'
>         mv actual_logs .git/logs
>  '
>
> +test_expect_success 'rebase when inside worktree subdirectory' '
> +       git init main-wt &&
> +       (
> +               cd main-wt &&
> +               git commit --allow-empty -m "initial" &&
> +               # create commit with foo/bar/baz
> +               mkdir -p foo/bar &&
> +               touch foo/bar/baz &&
> +               git add foo/bar/baz &&
> +               git commit -m "add foo/bar/baz" &&
> +               # create commit with a/b/c
> +               mkdir -p a/b &&
> +               touch a/b/c &&
> +               git add a/b/c &&
> +               git commit -m "add a/b/c" &&
> +               # create another branch for our other worktree
> +               git branch other &&
> +               git worktree add ../other-wt other &&
> +               (
> +                       cd ../other-wt &&
> +                       mkdir -p random/dir &&
> +                       (
> +                               cd random/dir &&
> +                               git rebase --onto HEAD^^ HEAD^  # drops the HEAD^ commit
> +                       )
> +               )
> +       )
> +'
> +
>  test_done
> --
> 2.35.0.rc0.227.g00780c9af4-goog

Re: [Bug] Rebase from worktree subdir is broken (was Re: [PATCH v5 07/11] rebase: do not attempt to remove startup_info->original_cwd)

[Glen Choo]

Elijah Newren <newren@gmail.com> writes:

> On Tue, Jan 25, 2022 at 12:27 PM Glen Choo <chooglen@google.com> wrote:
>>
>> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>>
>> > From: Elijah Newren <newren@gmail.com>
>> >
>> > Since rebase spawns a `checkout` subprocess, make sure we run that from
>> > the startup_info->original_cwd directory, so that the checkout process
>> > knows to protect that directory.
>> >
>> > Signed-off-by: Elijah Newren <newren@gmail.com>
>> > ---
>> >  sequencer.c          | 2 ++
>> >  t/t2501-cwd-empty.sh | 4 ++--
>> >  2 files changed, 4 insertions(+), 2 deletions(-)
>> >
>> > diff --git a/sequencer.c b/sequencer.c
>> > index ea96837cde3..83f257e7fa4 100644
>> > --- a/sequencer.c
>> > +++ b/sequencer.c
>> > @@ -4228,6 +4228,8 @@ static int run_git_checkout(struct repository *r, struct replay_opts *opts,
>> >
>> >       cmd.git_cmd = 1;
>> >
>> > +     if (startup_info->original_cwd)
>> > +             cmd.dir = startup_info->original_cwd;
>> >       strvec_push(&cmd.args, "checkout");
>> >       strvec_push(&cmd.args, commit);
>> >       strvec_pushf(&cmd.env_array, GIT_REFLOG_ACTION "=%s", action);
>> > diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
>> > index b1182390ba3..52335a8afe9 100755
>> > --- a/t/t2501-cwd-empty.sh
>> > +++ b/t/t2501-cwd-empty.sh
>> > @@ -166,11 +166,11 @@ test_expect_success 'cherry-pick fails if cwd needs to be removed' '
>> >  '
>> >
>> >  test_expect_success 'rebase does not clean cwd incidentally' '
>> > -     test_incidental_dir_removal failure git rebase reverted
>> > +     test_incidental_dir_removal success git rebase reverted
>> >  '
>> >
>> >  test_expect_success 'rebase fails if cwd needs to be removed' '
>> > -     test_required_dir_removal failure git rebase fd_conflict
>> > +     test_required_dir_removal success git rebase fd_conflict
>> >  '
>> >
>> >  test_expect_success 'revert does not clean cwd incidentally' '
>> > --
>> > gitgitgadget
>>
>> This commit (which is already in master) introduces a bug that breaks
>> rebase when rebasing inside a subdirectory of a worktree. You can see
>> that the below test fails with:
>>
>>   error: The following untracked working tree files would be overwritten by merge:
>>           a/b/c
>>   Please move or remove them before you merge.
>
> Thanks for the detailed report -- with a full testcase!

Glad to be of help :)

>> This only affects subdirectories in worktrees, i.e. rebasing anywhere in
>> the `main-wt` directory is fine, and rebasing from the top of `other-wt`
>> is fine, but `other-wt/any/other/dir` fails.
>>
>> I haven't tracked down the root cause yet, but judging from the commit,
>> I would suppose that the checkout is being spawned in the wrong
>> directory, causing the files to not be cleaned up.
>
> There's nothing wrong with running checkout from a subdirectory.  It
> is unfortunate that setup.c auto-discovers both the git directory and
> the working tree, but sets GIT_DIR without setting GIT_WORK_TREE in
> the case of a non-main worktree; it's not particularly friendly for
> subcommands.  Of course, it's also unfortunate that sequencer still
> forks subprocesses other than those requested by a user with e.g.
> --exec.
>
> But, anyway, I've got a patch that I'll send as soon as it passes CI
> (https://github.com/git/git/pull/1205).

I wish I had seen this email before diving into debugging it myself
since your fix is more comprehensive, but it was a good learning
experience nonetheless.

Re: [Bug] Rebase from worktree subdir is broken (was Re: [PATCH v5 07/11] rebase: do not attempt to remove startup_info->original_cwd)

[Eric Sunshine]

On Tue, Jan 25, 2022 at 6:59 PM Elijah Newren <newren@gmail.com> wrote:
> On Tue, Jan 25, 2022 at 12:27 PM Glen Choo <chooglen@google.com> wrote:
> > This commit (which is already in master) introduces a bug that breaks
> > rebase when rebasing inside a subdirectory of a worktree. You can see
> > that the below test fails with:
>
> There's nothing wrong with running checkout from a subdirectory.  It
> is unfortunate that setup.c auto-discovers both the git directory and
> the working tree, but sets GIT_DIR without setting GIT_WORK_TREE in
> the case of a non-main worktree; it's not particularly friendly for
> subcommands.  Of course, it's also unfortunate that sequencer still
> forks subprocesses other than those requested by a user with e.g.
> --exec.
>
> But, anyway, I've got a patch that I'll send as soon as it passes CI
> (https://github.com/git/git/pull/1205).
>
> > +test_expect_success 'rebase when inside worktree subdirectory' '
> > +       git init main-wt &&
> > +       (
> > +               cd main-wt &&
> > +               git commit --allow-empty -m "initial" &&
> > +               # create commit with foo/bar/baz
> > +               mkdir -p foo/bar &&
> > +               touch foo/bar/baz &&
> > +               git add foo/bar/baz &&
> > +               git commit -m "add foo/bar/baz" &&
> > +               # create commit with a/b/c
> > +               mkdir -p a/b &&
> > +               touch a/b/c &&
> > +               git add a/b/c &&
> > +               git commit -m "add a/b/c" &&
> > +               # create another branch for our other worktree
> > +               git branch other &&
> > +               git worktree add ../other-wt other &&
> > +               (
> > +                       cd ../other-wt &&
> > +                       mkdir -p random/dir &&
> > +                       (
> > +                               cd random/dir &&
> > +                               git rebase --onto HEAD^^ HEAD^  # drops the HEAD^ commit
> > +                       )
> > +               )
> > +       )
> > +'

This is entirely minor, but all the inner subshells in this test are
superfluous. The outermost (...) will ensure that the working
directory is restored regardless of whether anything within its body
fails, no matter how much you `cd` around. Thus, all the inner
subshells could be dropped.

An alternative would be to close the outermost subshell immediately
after `git worktree add`:

    git init main-wt &&
    (
        cd main-wt &&
        ...
        git worktree add ../other-wt other
    ) &&
    mkdir -p other-wt/random/dir &&
    (
        cd other-wt/random/dir &&
        ...
    )

however, that may not buy you any clarity.

Re: [Bug] Rebase from worktree subdir is broken (was Re: [PATCH v5 07/11] rebase: do not attempt to remove startup_info->original_cwd)

[Eric Sunshine]

On Tue, Jan 25, 2022 at 7:32 PM Eric Sunshine <sunshine@sunshineco.com> wrote:
> On Tue, Jan 25, 2022 at 6:59 PM Elijah Newren <newren@gmail.com> wrote:
> > On Tue, Jan 25, 2022 at 12:27 PM Glen Choo <chooglen@google.com> wrote:
> > > +test_expect_success 'rebase when inside worktree subdirectory' '
> > > +       git init main-wt &&
> > > +       (
> > > +               cd main-wt &&
> > > +               git commit --allow-empty -m "initial" &&
> > > +               # create commit with foo/bar/baz
> > > +               mkdir -p foo/bar &&
> > > +               touch foo/bar/baz &&
> > > +               git add foo/bar/baz &&
> > > +               git commit -m "add foo/bar/baz" &&
> > > +               # create commit with a/b/c
> > > +               mkdir -p a/b &&
> > > +               touch a/b/c &&
> > > +               git add a/b/c &&
> > > +               git commit -m "add a/b/c" &&
>
> This is entirely minor, but all the inner subshells in this test are
> superfluous. [...]

One other minor comment: If the file's timestamp has no significance
to the test, then our style is to create the file with `>` rather than
`touch`, so:

    ... &&
    >foo/bar/baz &&
    ...
    >a/b/c &&
    ...

Re: [Bug] Rebase from worktree subdir is broken (was Re: [PATCH v5 07/11] rebase: do not attempt to remove startup_info->original_cwd)

[Elijah Newren]

On Tue, Jan 25, 2022 at 4:39 PM Eric Sunshine <sunshine@sunshineco.com> wrote:
>
> On Tue, Jan 25, 2022 at 7:32 PM Eric Sunshine <sunshine@sunshineco.com> wrote:
> > On Tue, Jan 25, 2022 at 6:59 PM Elijah Newren <newren@gmail.com> wrote:
> > > On Tue, Jan 25, 2022 at 12:27 PM Glen Choo <chooglen@google.com> wrote:
> > > > +test_expect_success 'rebase when inside worktree subdirectory' '
> > > > +       git init main-wt &&
> > > > +       (
> > > > +               cd main-wt &&
> > > > +               git commit --allow-empty -m "initial" &&
> > > > +               # create commit with foo/bar/baz
> > > > +               mkdir -p foo/bar &&
> > > > +               touch foo/bar/baz &&
> > > > +               git add foo/bar/baz &&
> > > > +               git commit -m "add foo/bar/baz" &&
> > > > +               # create commit with a/b/c
> > > > +               mkdir -p a/b &&
> > > > +               touch a/b/c &&
> > > > +               git add a/b/c &&
> > > > +               git commit -m "add a/b/c" &&
> >
> > This is entirely minor, but all the inner subshells in this test are
> > superfluous. [...]
>
> One other minor comment: If the file's timestamp has no significance
> to the test, then our style is to create the file with `>` rather than
> `touch`, so:
>
>     ... &&
>     >foo/bar/baz &&
>     ...
>     >a/b/c &&
>     ...

Ah, good point.  And while at it, we can replace the touch/add/commit
sequence with a simple test_commit call.

Re: [Bug] Rebase from worktree subdir is broken (was Re: [PATCH v5 07/11] rebase: do not attempt to remove startup_info->original_cwd)

[Glen Choo]

Elijah Newren <newren@gmail.com> writes:

> On Tue, Jan 25, 2022 at 4:39 PM Eric Sunshine <sunshine@sunshineco.com> wrote:
>>
>> On Tue, Jan 25, 2022 at 7:32 PM Eric Sunshine <sunshine@sunshineco.com> wrote:
>> > On Tue, Jan 25, 2022 at 6:59 PM Elijah Newren <newren@gmail.com> wrote:
>> > > On Tue, Jan 25, 2022 at 12:27 PM Glen Choo <chooglen@google.com> wrote:
>> > > > +test_expect_success 'rebase when inside worktree subdirectory' '
>> > > > +       git init main-wt &&
>> > > > +       (
>> > > > +               cd main-wt &&
>> > > > +               git commit --allow-empty -m "initial" &&
>> > > > +               # create commit with foo/bar/baz
>> > > > +               mkdir -p foo/bar &&
>> > > > +               touch foo/bar/baz &&
>> > > > +               git add foo/bar/baz &&
>> > > > +               git commit -m "add foo/bar/baz" &&
>> > > > +               # create commit with a/b/c
>> > > > +               mkdir -p a/b &&
>> > > > +               touch a/b/c &&
>> > > > +               git add a/b/c &&
>> > > > +               git commit -m "add a/b/c" &&
>> >
>> > This is entirely minor, but all the inner subshells in this test are
>> > superfluous. [...]
>>
>> One other minor comment: If the file's timestamp has no significance
>> to the test, then our style is to create the file with `>` rather than
>> `touch`, so:
>>
>>     ... &&
>>     >foo/bar/baz &&
>>     ...
>>     >a/b/c &&
>>     ...
>
> Ah, good point.  And while at it, we can replace the touch/add/commit
> sequence with a simple test_commit call.

Yes, thanks for pointing that out - I had copied my reproduction script
into the test case without paying enough attention.

Re: [Bug] Rebase from worktree subdir is broken (was Re: [PATCH v5 07/11] rebase: do not attempt to remove startup_info->original_cwd)

[Elijah Newren]

On Tue, Jan 25, 2022 at 5:15 PM Glen Choo <chooglen@google.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> > On Tue, Jan 25, 2022 at 4:39 PM Eric Sunshine <sunshine@sunshineco.com> wrote:
> >>
> >> On Tue, Jan 25, 2022 at 7:32 PM Eric Sunshine <sunshine@sunshineco.com> wrote:
> >> > On Tue, Jan 25, 2022 at 6:59 PM Elijah Newren <newren@gmail.com> wrote:
> >> > > On Tue, Jan 25, 2022 at 12:27 PM Glen Choo <chooglen@google.com> wrote:
> >> > > > +test_expect_success 'rebase when inside worktree subdirectory' '
> >> > > > +       git init main-wt &&
> >> > > > +       (
> >> > > > +               cd main-wt &&
> >> > > > +               git commit --allow-empty -m "initial" &&
> >> > > > +               # create commit with foo/bar/baz
> >> > > > +               mkdir -p foo/bar &&
> >> > > > +               touch foo/bar/baz &&
> >> > > > +               git add foo/bar/baz &&
> >> > > > +               git commit -m "add foo/bar/baz" &&
> >> > > > +               # create commit with a/b/c
> >> > > > +               mkdir -p a/b &&
> >> > > > +               touch a/b/c &&
> >> > > > +               git add a/b/c &&
> >> > > > +               git commit -m "add a/b/c" &&
> >> >
> >> > This is entirely minor, but all the inner subshells in this test are
> >> > superfluous. [...]
> >>
> >> One other minor comment: If the file's timestamp has no significance
> >> to the test, then our style is to create the file with `>` rather than
> >> `touch`, so:
> >>
> >>     ... &&
> >>     >foo/bar/baz &&
> >>     ...
> >>     >a/b/c &&
> >>     ...
> >
> > Ah, good point.  And while at it, we can replace the touch/add/commit
> > sequence with a simple test_commit call.
>
> Yes, thanks for pointing that out - I had copied my reproduction script
> into the test case without paying enough attention.

To be honest, it's the kind of thing others have reminded me of dozens
of times.  In fact, although I read your testcase, it didn't even
occur to me until Eric brought up ">" instead of "touch" (which was
something else I had been reminded of dozens of times before
learning).

To me, it's all small potatoes compared to coming up with a good
reproducible testcase; that was the hard work, and you did that.  You
even found a fix and called out the right things in the review for
others to look at.  So, it's all good.

Re: [Bug] Rebase from worktree subdir is broken (was Re: [PATCH v5 07/11] rebase: do not attempt to remove startup_info->original_cwd)

[Phillip Wood]

On 25/01/2022 23:59, Elijah Newren wrote:
> On Tue, Jan 25, 2022 at 12:27 PM Glen Choo <chooglen@google.com> wrote:
>>
>> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>>
>>> From: Elijah Newren <newren@gmail.com>
>>>
>>> Since rebase spawns a `checkout` subprocess, make sure we run that from
>>> the startup_info->original_cwd directory, so that the checkout process
>>> knows to protect that directory.
>>>
>>> Signed-off-by: Elijah Newren <newren@gmail.com>
>>> ---
>>>   sequencer.c          | 2 ++
>>>   t/t2501-cwd-empty.sh | 4 ++--
>>>   2 files changed, 4 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/sequencer.c b/sequencer.c
>>> index ea96837cde3..83f257e7fa4 100644
>>> --- a/sequencer.c
>>> +++ b/sequencer.c
>>> @@ -4228,6 +4228,8 @@ static int run_git_checkout(struct repository *r, struct replay_opts *opts,
>>>
>>>        cmd.git_cmd = 1;
>>>
>>> +     if (startup_info->original_cwd)
>>> +             cmd.dir = startup_info->original_cwd;
>>>        strvec_push(&cmd.args, "checkout");
>>>        strvec_push(&cmd.args, commit);
>>>        strvec_pushf(&cmd.env_array, GIT_REFLOG_ACTION "=%s", action);
>>> diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
>>> index b1182390ba3..52335a8afe9 100755
>>> --- a/t/t2501-cwd-empty.sh
>>> +++ b/t/t2501-cwd-empty.sh
>>> @@ -166,11 +166,11 @@ test_expect_success 'cherry-pick fails if cwd needs to be removed' '
>>>   '
>>>
>>>   test_expect_success 'rebase does not clean cwd incidentally' '
>>> -     test_incidental_dir_removal failure git rebase reverted
>>> +     test_incidental_dir_removal success git rebase reverted
>>>   '
>>>
>>>   test_expect_success 'rebase fails if cwd needs to be removed' '
>>> -     test_required_dir_removal failure git rebase fd_conflict
>>> +     test_required_dir_removal success git rebase fd_conflict
>>>   '
>>>
>>>   test_expect_success 'revert does not clean cwd incidentally' '
>>> --
>>> gitgitgadget
>>
>> This commit (which is already in master) introduces a bug that breaks
>> rebase when rebasing inside a subdirectory of a worktree. You can see
>> that the below test fails with:
>>
>>    error: The following untracked working tree files would be overwritten by merge:
>>            a/b/c
>>    Please move or remove them before you merge.
> 
> Thanks for the detailed report -- with a full testcase!
> 
>> This only affects subdirectories in worktrees, i.e. rebasing anywhere in
>> the `main-wt` directory is fine, and rebasing from the top of `other-wt`
>> is fine, but `other-wt/any/other/dir` fails.
>>
>> I haven't tracked down the root cause yet, but judging from the commit,
>> I would suppose that the checkout is being spawned in the wrong
>> directory, causing the files to not be cleaned up.
> 
> There's nothing wrong with running checkout from a subdirectory.  It
> is unfortunate that setup.c auto-discovers both the git directory and
> the working tree, but sets GIT_DIR without setting GIT_WORK_TREE in
> the case of a non-main worktree; it's not particularly friendly for
> subcommands.  Of course, it's also unfortunate that sequencer still
> forks subprocesses other than those requested by a user with e.g.
> --exec.
> 
> But, anyway, I've got a patch that I'll send as soon as it passes CI
> (https://github.com/git/git/pull/1205).

The patch hasn't come through to me on the mailing list yet, but it 
looks good. I thought we set both GIT_DIR and GIT_WORK_TREE when we were 
in a non-main worktree but obviously we don't. Eric do you happen to 
know if that is intentional?

As an aside I'm going to post an updated version of my series removing 
the call to 'git checkout' from sequencer.c later today but we 
definitely want to have this fix before that series gets merged.

Best Wishes

Phillip

>> ---
>>   t/t3400-rebase.sh | 29 +++++++++++++++++++++++++++++
>>   1 file changed, 29 insertions(+)
>>
>> diff --git a/t/t3400-rebase.sh b/t/t3400-rebase.sh
>> index 23dbd3c82e..8b8b66538b 100755
>> --- a/t/t3400-rebase.sh
>> +++ b/t/t3400-rebase.sh
>> @@ -416,4 +416,33 @@ test_expect_success MINGW,SYMLINKS_WINDOWS 'rebase when .git/logs is a symlink'
>>          mv actual_logs .git/logs
>>   '
>>
>> +test_expect_success 'rebase when inside worktree subdirectory' '
>> +       git init main-wt &&
>> +       (
>> +               cd main-wt &&
>> +               git commit --allow-empty -m "initial" &&
>> +               # create commit with foo/bar/baz
>> +               mkdir -p foo/bar &&
>> +               touch foo/bar/baz &&
>> +               git add foo/bar/baz &&
>> +               git commit -m "add foo/bar/baz" &&
>> +               # create commit with a/b/c
>> +               mkdir -p a/b &&
>> +               touch a/b/c &&
>> +               git add a/b/c &&
>> +               git commit -m "add a/b/c" &&
>> +               # create another branch for our other worktree
>> +               git branch other &&
>> +               git worktree add ../other-wt other &&
>> +               (
>> +                       cd ../other-wt &&
>> +                       mkdir -p random/dir &&
>> +                       (
>> +                               cd random/dir &&
>> +                               git rebase --onto HEAD^^ HEAD^  # drops the HEAD^ commit
>> +                       )
>> +               )
>> +       )
>> +'
>> +
>>   test_done
>> --
>> 2.35.0.rc0.227.g00780c9af4-goog

Re: [Bug] Rebase from worktree subdir is broken (was Re: [PATCH v5 07/11] rebase: do not attempt to remove startup_info->original_cwd)

[Eric Sunshine]

On Wed, Jan 26, 2022 at 6:00 AM Phillip Wood <phillip.wood123@gmail.com> wrote:
> On 25/01/2022 23:59, Elijah Newren wrote:
> > There's nothing wrong with running checkout from a subdirectory.  It
> > is unfortunate that setup.c auto-discovers both the git directory and
> > the working tree, but sets GIT_DIR without setting GIT_WORK_TREE in
> > the case of a non-main worktree; it's not particularly friendly for
> > subcommands.  Of course, it's also unfortunate that sequencer still
> > forks subprocesses other than those requested by a user with e.g.
> > --exec.
> >
> > But, anyway, I've got a patch that I'll send as soon as it passes CI
> > (https://github.com/git/git/pull/1205).
>
> The patch hasn't come through to me on the mailing list yet, but it
> looks good. I thought we set both GIT_DIR and GIT_WORK_TREE when we were
> in a non-main worktree but obviously we don't. Eric do you happen to
> know if that is intentional?

As far as I know, there is no reason to set GIT_DIR and GIT_WORK_TREE,
in general, when in a linked worktree since each worktree has its own
.git file ("gitfile") which tells Git commands where the repository is
and signals that that directory itself (which contains the .git file)
is indeed a Git worktree.

Re: [Bug] Rebase from worktree subdir is broken (was Re: [PATCH v5 07/11] rebase: do not attempt to remove startup_info->original_cwd)

[Elijah Newren]

On Tue, Jan 25, 2022 at 4:30 PM Glen Choo <chooglen@google.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> > On Tue, Jan 25, 2022 at 12:27 PM Glen Choo <chooglen@google.com> wrote:
> >>
> >> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
> >>
> >> > From: Elijah Newren <newren@gmail.com>
> >> >
> >> > Since rebase spawns a `checkout` subprocess, make sure we run that from
> >> > the startup_info->original_cwd directory, so that the checkout process
> >> > knows to protect that directory.
> >> >
> >> > Signed-off-by: Elijah Newren <newren@gmail.com>
> >> > ---
> >> >  sequencer.c          | 2 ++
> >> >  t/t2501-cwd-empty.sh | 4 ++--
> >> >  2 files changed, 4 insertions(+), 2 deletions(-)
> >> >
> >> > diff --git a/sequencer.c b/sequencer.c
> >> > index ea96837cde3..83f257e7fa4 100644
> >> > --- a/sequencer.c
> >> > +++ b/sequencer.c
> >> > @@ -4228,6 +4228,8 @@ static int run_git_checkout(struct repository *r, struct replay_opts *opts,
> >> >
> >> >       cmd.git_cmd = 1;
> >> >
> >> > +     if (startup_info->original_cwd)
> >> > +             cmd.dir = startup_info->original_cwd;
> >> >       strvec_push(&cmd.args, "checkout");
> >> >       strvec_push(&cmd.args, commit);
> >> >       strvec_pushf(&cmd.env_array, GIT_REFLOG_ACTION "=%s", action);
> >> > diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
> >> > index b1182390ba3..52335a8afe9 100755
> >> > --- a/t/t2501-cwd-empty.sh
> >> > +++ b/t/t2501-cwd-empty.sh
> >> > @@ -166,11 +166,11 @@ test_expect_success 'cherry-pick fails if cwd needs to be removed' '
> >> >  '
> >> >
> >> >  test_expect_success 'rebase does not clean cwd incidentally' '
> >> > -     test_incidental_dir_removal failure git rebase reverted
> >> > +     test_incidental_dir_removal success git rebase reverted
> >> >  '
> >> >
> >> >  test_expect_success 'rebase fails if cwd needs to be removed' '
> >> > -     test_required_dir_removal failure git rebase fd_conflict
> >> > +     test_required_dir_removal success git rebase fd_conflict
> >> >  '
> >> >
> >> >  test_expect_success 'revert does not clean cwd incidentally' '
> >> > --
> >> > gitgitgadget
> >>
> >> This commit (which is already in master) introduces a bug that breaks
> >> rebase when rebasing inside a subdirectory of a worktree. You can see
> >> that the below test fails with:
> >>
> >>   error: The following untracked working tree files would be overwritten by merge:
> >>           a/b/c
> >>   Please move or remove them before you merge.
> >
> > Thanks for the detailed report -- with a full testcase!
>
> Glad to be of help :)
>
> >> This only affects subdirectories in worktrees, i.e. rebasing anywhere in
> >> the `main-wt` directory is fine, and rebasing from the top of `other-wt`
> >> is fine, but `other-wt/any/other/dir` fails.
> >>
> >> I haven't tracked down the root cause yet, but judging from the commit,
> >> I would suppose that the checkout is being spawned in the wrong
> >> directory, causing the files to not be cleaned up.
> >
> > There's nothing wrong with running checkout from a subdirectory.  It
> > is unfortunate that setup.c auto-discovers both the git directory and
> > the working tree, but sets GIT_DIR without setting GIT_WORK_TREE in
> > the case of a non-main worktree; it's not particularly friendly for
> > subcommands.  Of course, it's also unfortunate that sequencer still
> > forks subprocesses other than those requested by a user with e.g.
> > --exec.
> >
> > But, anyway, I've got a patch that I'll send as soon as it passes CI
> > (https://github.com/git/git/pull/1205).
>
> I wish I had seen this email before diving into debugging it myself
> since your fix is more comprehensive, but it was a good learning
> experience nonetheless.

Yeah, sorry, it took me quite a bit of debugging as well before
figuring all this stuff out, so I don't think I sent the email in time
to save you the work.

Re: [Bug] Rebase from worktree subdir is broken (was Re: [PATCH v5 07/11] rebase: do not attempt to remove startup_info->original_cwd)

[Phillip Wood]

On 26/01/2022 17:53, Eric Sunshine wrote:
> On Wed, Jan 26, 2022 at 6:00 AM Phillip Wood <phillip.wood123@gmail.com> wrote:
>> On 25/01/2022 23:59, Elijah Newren wrote:
>>> There's nothing wrong with running checkout from a subdirectory.  It
>>> is unfortunate that setup.c auto-discovers both the git directory and
>>> the working tree, but sets GIT_DIR without setting GIT_WORK_TREE in
>>> the case of a non-main worktree; it's not particularly friendly for
>>> subcommands.  Of course, it's also unfortunate that sequencer still
>>> forks subprocesses other than those requested by a user with e.g.
>>> --exec.
>>>
>>> But, anyway, I've got a patch that I'll send as soon as it passes CI
>>> (https://github.com/git/git/pull/1205).
>>
>> The patch hasn't come through to me on the mailing list yet, but it
>> looks good. I thought we set both GIT_DIR and GIT_WORK_TREE when we were
>> in a non-main worktree but obviously we don't. Eric do you happen to
>> know if that is intentional?
> 
> As far as I know, there is no reason to set GIT_DIR and GIT_WORK_TREE,
> in general, when in a linked worktree since each worktree has its own
> .git file ("gitfile") which tells Git commands where the repository is
> and signals that that directory itself (which contains the .git file)
> is indeed a Git worktree.

Oh of course I should have thought of that. Now I'm wondering why we're 
exporting GIT_DIR - I should go and read the code for myself.

Thanks

Phillip

Re: [Bug] Rebase from worktree subdir is broken (was Re: [PATCH v5 07/11] rebase: do not attempt to remove startup_info->original_cwd)

[Elijah Newren]

On Wed, Jan 26, 2022 at 9:53 AM Eric Sunshine <sunshine@sunshineco.com> wrote:
>
> On Wed, Jan 26, 2022 at 6:00 AM Phillip Wood <phillip.wood123@gmail.com> wrote:
> > On 25/01/2022 23:59, Elijah Newren wrote:
> > > There's nothing wrong with running checkout from a subdirectory.  It
> > > is unfortunate that setup.c auto-discovers both the git directory and
> > > the working tree, but sets GIT_DIR without setting GIT_WORK_TREE in
> > > the case of a non-main worktree; it's not particularly friendly for
> > > subcommands.  Of course, it's also unfortunate that sequencer still
> > > forks subprocesses other than those requested by a user with e.g.
> > > --exec.
> > >
> > > But, anyway, I've got a patch that I'll send as soon as it passes CI
> > > (https://github.com/git/git/pull/1205).
> >
> > The patch hasn't come through to me on the mailing list yet, but it
> > looks good. I thought we set both GIT_DIR and GIT_WORK_TREE when we were
> > in a non-main worktree but obviously we don't. Eric do you happen to
> > know if that is intentional?
>
> As far as I know, there is no reason to set GIT_DIR and GIT_WORK_TREE,
> in general, when in a linked worktree since each worktree has its own
> .git file ("gitfile") which tells Git commands where the repository is
> and signals that that directory itself (which contains the .git file)
> is indeed a Git worktree.

Oh, interesting.  Not setting GIT_DIR either does sound a bit better.

...though after digging for a while, it turns out to be a bit more
involved than I thought.  Although the below patch passes our current
testsuite and fixes the reported bug, I'm worried I've missed some cases
not tested by the testsuite.

Not sure if we want to pursue this, drop it, or something else.  Thoughts?

-- >8 --
Subject: [RFC/POC PATCH] setup: do not pre-emptively set GIT_DIR based on discovery

setup_git_directory_gently() handles a few cases, but when it discovers
the git directory and finds it is not ".git", it sets and exports the
GIT_DIR environment variable to the discovered value.  It does not set
GIT_WORK_TREE, though, even when it also discovers it.  This has two
drawbacks:

  * Since Git assumes the working tree is '.' when GIT_DIR is set and
    GIT_WORK_TREE isn't, any subprocesses called by Git which need to
    operate on the working tree either need to be careful to always
    invoke with their cwd being the toplevel of the actual working tree
    (and potentially losing information about 'prefix'), or take care to
    also specify the GIT_WORK_TREE environment value when forking.

  * The GIT_DIR environment variable behaves somewhat like an implicit
    'the_repository'; it's another bit of global state that is
    potentially problematic for libifying efforts of libgit.

Avoid these issues by no longer exporting GIT_DIR from the setup code
(other locations like the parser handling the --git-dir flag will still
set it, though).

Some comments on the various code changes:
   * clone/push/fetch related:
     * there are *many* subprocesses involved in fetch/push and friends,
       and they typically need to know the GIT_DIR they are operating on
     * this involves: fetch-patch.c, connected.c, bundle.c, clone.c,
       transport-helper.c, receive-pack.c, upload-pack.c
     * this accounts for the majority of this patch
     * much of this work could be avoided by having enter_repo() call
       xsetenv(GIT_DIR_ENVIRONMENT, ".", 1) just after its set_git_dir()
       call, but I don't know if that'd be considered a half measure
   * rev-parse.c: substitute usage of the GIT_DIR environment variable
     with the_repository->gitdir that should have the same value
     (exept when the latter is ".git" the former will be NULL, so there
     are some small logic tweaks).
   * run-command: hooks have been operating under the assumption that
     GIT_DIR is set for a long time.  This is possibly
     counter-productive for the same reasons listed above, and it has
     certainly been brought up before on the list, but backward
     compatibility concerns led me to just propagate it to avoid
     breaking anything.
   * test-subprocess: not sure if we should automatically export
     GIT_DIR, but the existing code and one test in the repo relied on
     it, so I just included it.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/clone.c            |  7 ++++++-
 builtin/receive-pack.c     | 12 ++++++++++++
 builtin/rev-parse.c        |  8 +++-----
 bundle.c                   |  2 ++
 connected.c                |  2 ++
 environment.c              |  1 -
 fetch-pack.c               |  6 ++++++
 run-command.c              |  2 ++
 t/helper/test-subprocess.c |  2 ++
 transport-helper.c         |  3 +++
 upload-pack.c              |  2 ++
 11 files changed, 40 insertions(+), 7 deletions(-)

diff --git a/builtin/clone.c b/builtin/clone.c
index 727e16e0ae..5304939cec 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -836,13 +836,18 @@ static void dissociate_from_references(void)
 {
 	static const char* argv[] = { "repack", "-a", "-d", NULL };
 	char *alternates = git_pathdup("objects/info/alternates");
+	struct strvec env = STRVEC_INIT;
+	strvec_pushf(&env, GIT_DIR_ENVIRONMENT "=%s", the_repository->gitdir);
 
 	if (!access(alternates, F_OK)) {
-		if (run_command_v_opt(argv, RUN_GIT_CMD|RUN_COMMAND_NO_STDIN))
+		if (run_command_v_opt_cd_env(argv,
+					     RUN_GIT_CMD|RUN_COMMAND_NO_STDIN,
+					     NULL, env.v))
 			die(_("cannot repack to clean up"));
 		if (unlink(alternates) && errno != ENOENT)
 			die_errno(_("cannot unlink temporary alternates file"));
 	}
+	strvec_clear(&env);
 	free(alternates);
 }
 
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 9f4a0b816c..b5d2633b78 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -825,6 +825,8 @@ static int run_and_feed_hook(const char *hook_name, feed_fn feed,
 	proc.stdout_to_stderr = 1;
 	proc.trace2_hook_name = hook_name;
 
+	strvec_pushf(&proc.env_array, "%s=%s",
+		     GIT_DIR_ENVIRONMENT, the_repository->gitdir);
 	if (feed_state->push_options) {
 		int i;
 		for (i = 0; i < feed_state->push_options->nr; i++)
@@ -953,6 +955,8 @@ static int run_update_hook(struct command *cmd)
 	strvec_push(&proc.args, cmd->ref_name);
 	strvec_push(&proc.args, oid_to_hex(&cmd->old_oid));
 	strvec_push(&proc.args, oid_to_hex(&cmd->new_oid));
+	strvec_pushf(&proc.env_array, "%s=%s",
+		     GIT_DIR_ENVIRONMENT, the_repository->gitdir);
 
 	proc.no_stdin = 1;
 	proc.stdout_to_stderr = 1;
@@ -1128,6 +1132,8 @@ static int run_proc_receive_hook(struct command *commands,
 	}
 
 	strvec_push(&proc.args, hook_path);
+	strvec_pushf(&proc.env_array, "%s=%s",
+		     GIT_DIR_ENVIRONMENT, the_repository->gitdir);
 	proc.in = -1;
 	proc.out = -1;
 	proc.trace2_hook_name = "proc-receive";
@@ -1645,6 +1651,8 @@ static void run_update_post_hook(struct command *commands)
 	proc.stdout_to_stderr = 1;
 	proc.err = use_sideband ? -1 : 0;
 	proc.trace2_hook_name = "post-update";
+	strvec_pushf(&proc.env_array, "%s=%s",
+		     GIT_DIR_ENVIRONMENT, the_repository->gitdir);
 
 	if (!start_command(&proc)) {
 		if (use_sideband)
@@ -2236,6 +2244,8 @@ static const char *unpack(int err_fd, struct shallow_info *si)
 		child.no_stdout = 1;
 		child.err = err_fd;
 		child.git_cmd = 1;
+		strvec_pushf(&child.env_array, "%s=%s",
+			     GIT_DIR_ENVIRONMENT, the_repository->gitdir);
 		status = run_command(&child);
 		if (status)
 			return "unpack-objects abnormal exit";
@@ -2267,6 +2277,8 @@ static const char *unpack(int err_fd, struct shallow_info *si)
 		child.out = -1;
 		child.err = err_fd;
 		child.git_cmd = 1;
+		strvec_pushf(&child.env_array, "%s=%s",
+			     GIT_DIR_ENVIRONMENT, the_repository->gitdir);
 		status = start_command(&child);
 		if (status)
 			return "index-pack fork failed";
diff --git a/builtin/rev-parse.c b/builtin/rev-parse.c
index 8480a59f57..1d578d48a1 100644
--- a/builtin/rev-parse.c
+++ b/builtin/rev-parse.c
@@ -930,12 +930,12 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix)
 			}
 			if (!strcmp(arg, "--git-dir") ||
 			    !strcmp(arg, "--absolute-git-dir")) {
-				const char *gitdir = getenv(GIT_DIR_ENVIRONMENT);
+				const char *gitdir = the_repository->gitdir;
 				char *cwd;
 				int len;
 				enum format_type wanted = format;
 				if (arg[2] == 'g') {	/* --git-dir */
-					if (gitdir) {
+					if (strcmp(gitdir, ".git")) {
 						print_path(gitdir, prefix, format, DEFAULT_UNMODIFIED);
 						continue;
 					}
@@ -945,9 +945,7 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix)
 					}
 				} else {		/* --absolute-git-dir */
 					wanted = FORMAT_CANONICAL;
-					if (!gitdir && !prefix)
-						gitdir = ".git";
-					if (gitdir) {
+					if (strcmp(gitdir, ".git") || !prefix) {
 						struct strbuf realpath = STRBUF_INIT;
 						strbuf_realpath(&realpath, gitdir, 1);
 						puts(realpath.buf);
diff --git a/bundle.c b/bundle.c
index a0bb687b0f..897d36c83a 100644
--- a/bundle.c
+++ b/bundle.c
@@ -573,6 +573,8 @@ int unbundle(struct repository *r, struct bundle_header *header,
 {
 	struct child_process ip = CHILD_PROCESS_INIT;
 	strvec_pushl(&ip.args, "index-pack", "--fix-thin", "--stdin", NULL);
+	strvec_pushf(&ip.env_array, "%s=%s",
+		     GIT_DIR_ENVIRONMENT, the_repository->gitdir);
 
 	if (extra_index_pack_args) {
 		strvec_pushv(&ip.args, extra_index_pack_args->v);
diff --git a/connected.c b/connected.c
index ed3025e7a2..3dfc6278e8 100644
--- a/connected.c
+++ b/connected.c
@@ -109,6 +109,8 @@ int check_connected(oid_iterate_fn fn, void *cb_data,
 			     _("Checking connectivity"));
 
 	rev_list.git_cmd = 1;
+	strvec_pushf(&rev_list.env_array, "%s=%s",
+		     GIT_DIR_ENVIRONMENT, the_repository->gitdir);
 	if (opt->env)
 		strvec_pushv(&rev_list.env_array, opt->env);
 	rev_list.in = -1;
diff --git a/environment.c b/environment.c
index fd0501e77a..db31c37a62 100644
--- a/environment.c
+++ b/environment.c
@@ -327,7 +327,6 @@ char *get_graft_file(struct repository *r)
 
 static void set_git_dir_1(const char *path)
 {
-	xsetenv(GIT_DIR_ENVIRONMENT, path, 1);
 	setup_git_env(path);
 }
 
diff --git a/fetch-pack.c b/fetch-pack.c
index dd6ec449f2..8b59014c3a 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -892,6 +892,8 @@ static int get_pack(struct fetch_pack_args *args,
 		cmd_name = "index-pack";
 		strvec_push(&cmd.args, cmd_name);
 		strvec_push(&cmd.args, "--stdin");
+		strvec_pushf(&cmd.env_array, "%s=%s",
+			     GIT_DIR_ENVIRONMENT, the_repository->gitdir);
 		if (!args->quiet && !args->no_progress)
 			strvec_push(&cmd.args, "-v");
 		if (args->use_thin_pack)
@@ -927,6 +929,8 @@ static int get_pack(struct fetch_pack_args *args,
 	}
 	else {
 		cmd_name = "unpack-objects";
+		strvec_pushf(&cmd.env_array, "%s=%s",
+			     GIT_DIR_ENVIRONMENT, the_repository->gitdir);
 		strvec_push(&cmd.args, cmd_name);
 		if (args->quiet || args->no_progress)
 			strvec_push(&cmd.args, "-q");
@@ -1703,6 +1707,8 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
 			strvec_pushf(&cmd.args, "--index-pack-arg=%s",
 				     index_pack_args.v[j]);
 		strvec_push(&cmd.args, uri);
+		strvec_pushf(&cmd.env_array, "%s=%s",
+			     GIT_DIR_ENVIRONMENT, the_repository->gitdir);
 		cmd.git_cmd = 1;
 		cmd.no_stdin = 1;
 		cmd.out = -1;
diff --git a/run-command.c b/run-command.c
index 69dde42f1e..0fb21f92bf 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1319,6 +1319,8 @@ int run_hook_ve(const char *const *env, const char *name, va_list args)
 	strvec_push(&hook.args, p);
 	while ((p = va_arg(args, const char *)))
 		strvec_push(&hook.args, p);
+	strvec_pushf(&hook.env_array, "%s=%s",
+		     GIT_DIR_ENVIRONMENT, the_repository->gitdir);
 	if (env)
 		strvec_pushv(&hook.env_array, (const char **)env);
 	hook.no_stdin = 1;
diff --git a/t/helper/test-subprocess.c b/t/helper/test-subprocess.c
index ff22f2fa2c..d58bf7e18d 100644
--- a/t/helper/test-subprocess.c
+++ b/t/helper/test-subprocess.c
@@ -16,5 +16,7 @@ int cmd__subprocess(int argc, const char **argv)
 	}
 	cp.git_cmd = 1;
 	strvec_pushv(&cp.args, (const char **)argv + 1);
+	strvec_pushf(&cp.env_array, "%s=%s",
+		     GIT_DIR_ENVIRONMENT, the_repository->gitdir);
 	return run_command(&cp);
 }
diff --git a/transport-helper.c b/transport-helper.c
index a0297b0986..d26b5ecd10 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -448,6 +448,9 @@ static int get_importer(struct transport *transport, struct child_process *fasti
 		strvec_pushf(&fastimport->args, "--cat-blob-fd=%d", cat_blob_fd);
 	}
 	fastimport->git_cmd = 1;
+	if (have_git_dir())
+		strvec_pushf(&fastimport->env_array, "%s=%s",
+			     GIT_DIR_ENVIRONMENT, get_git_dir());
 
 	code = start_command(fastimport);
 	return code;
diff --git a/upload-pack.c b/upload-pack.c
index 8acc98741b..7bf36d6d3a 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -295,6 +295,8 @@ static void create_pack_file(struct upload_pack_data *pack_data,
 		strvec_push(&pack_objects.args, "--shallow-file");
 		strvec_push(&pack_objects.args, "");
 	}
+	strvec_pushf(&pack_objects.env_array, "%s=%s",
+		     GIT_DIR_ENVIRONMENT, the_repository->gitdir);
 	strvec_push(&pack_objects.args, "pack-objects");
 	strvec_push(&pack_objects.args, "--revs");
 	if (pack_data->use_thin_pack)
-- 
2.35.0.2.g9420af4777.dirty

Re: [Bug] Rebase from worktree subdir is broken (was Re: [PATCH v5 07/11] rebase: do not attempt to remove startup_info->original_cwd)

[Eric Sunshine]

On Thu, Jan 27, 2022 at 3:03 PM Elijah Newren <newren@gmail.com> wrote:
> On Wed, Jan 26, 2022 at 9:53 AM Eric Sunshine <sunshine@sunshineco.com> wrote:
> > As far as I know, there is no reason to set GIT_DIR and GIT_WORK_TREE,
> > in general, when in a linked worktree since each worktree has its own
> > .git file ("gitfile") which tells Git commands where the repository is
> > and signals that that directory itself (which contains the .git file)
> > is indeed a Git worktree.
>
> Oh, interesting.  Not setting GIT_DIR either does sound a bit better.
>
> ...though after digging for a while, it turns out to be a bit more
> involved than I thought.  Although the below patch passes our current
> testsuite and fixes the reported bug, I'm worried I've missed some cases
> not tested by the testsuite.
>
> Not sure if we want to pursue this, drop it, or something else.  Thoughts?

It's an enticing idea, though I have no deep knowledge about all the
possible interactions which may be impacted by such a change. Duy had
a deep understanding of how all this worked, and probably Peff, as
well, but they aren't around to offer opinions.

set_git_dir() has been setting the GIT_DIR environment variable ever
since it (the function) was introduced by d7ac12b25d (Add
set_git_dir() function, 2007-08-01). Unfortunately, the commit message
doesn't explain why it does so.

More below...

> -- >8 --
> Subject: [RFC/POC PATCH] setup: do not pre-emptively set GIT_DIR based on discovery
>
> Some comments on the various code changes:
>    * clone/push/fetch related:
>      * there are *many* subprocesses involved in fetch/push and friends,
>        and they typically need to know the GIT_DIR they are operating on
>      * this involves: fetch-patch.c, connected.c, bundle.c, clone.c,
>        transport-helper.c, receive-pack.c, upload-pack.c
>      * this accounts for the majority of this patch
>      * much of this work could be avoided by having enter_repo() call
>        xsetenv(GIT_DIR_ENVIRONMENT, ".", 1) just after its set_git_dir()
>        call, but I don't know if that'd be considered a half measure

It does feel a bit like a bandaid to insert new code at these
locations to set GIT_DIR manually. It's not clear to readers why
GIT_DIR is needed for these specific cases, nor what the impact is
when it is not set. Thus, one wonders if such a blanket approach is
indeed required or if a more narrow and directed fix can be applied,
such as calling subprograms with an explicit --git-dir= rather than
setting GIT_DIR with its potentially more broad and
difficult-to-reason-about impact.

> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
> diff --git a/builtin/clone.c b/builtin/clone.c
> @@ -836,13 +836,18 @@ static void dissociate_from_references(void)
>  {
> +       struct strvec env = STRVEC_INIT;
> +       strvec_pushf(&env, GIT_DIR_ENVIRONMENT "=%s", the_repository->gitdir);

Minor inconsistency: all the other similar changes in this patch use
"%s=%s" and then pass in GIT_DIR_ENVIRONMENT to be interpolated by
`%s`.

> diff --git a/builtin/rev-parse.c b/builtin/rev-parse.c
> @@ -930,12 +930,12 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix)
> -                               const char *gitdir = getenv(GIT_DIR_ENVIRONMENT);
> +                               const char *gitdir = the_repository->gitdir;
>                                 if (arg[2] == 'g') {    /* --git-dir */
> -                                       if (gitdir) {
> +                                       if (strcmp(gitdir, ".git")) {
> @@ -945,9 +945,7 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix)
> -                                       if (!gitdir && !prefix)
> -                                               gitdir = ".git";
> -                                       if (gitdir) {
> +                                       if (strcmp(gitdir, ".git") || !prefix) {

The meaning here becomes more obscure with this change applied. In the
original code, it was obvious enough that non-NULL `gitdir` meant that
the GIT_DIR environment variable had a value, but
`strcmp(gitdir,".git")` probably doesn't convey much to readers of
this code? Assigning the result of the strcmp() to a well-named
variable could go a long way toward making the meaning clearer. Or, an
in-code comment might be warranted.

> diff --git a/environment.c b/environment.c
> @@ -327,7 +327,6 @@ char *get_graft_file(struct repository *r)
>  static void set_git_dir_1(const char *path)
>  {
> -       xsetenv(GIT_DIR_ENVIRONMENT, path, 1);
>         setup_git_env(path);
>  }

Re: [Bug] Rebase from worktree subdir is broken (was Re: [PATCH v5 07/11] rebase: do not attempt to remove startup_info->original_cwd)

[Eric Sunshine]

On Sat, Feb 5, 2022 at 6:23 AM Eric Sunshine <sunshine@sunshineco.com> wrote:
> On Thu, Jan 27, 2022 at 3:03 PM Elijah Newren <newren@gmail.com> wrote:
> > Some comments on the various code changes:
> >    * clone/push/fetch related:
> >      * there are *many* subprocesses involved in fetch/push and friends,
> >        and they typically need to know the GIT_DIR they are operating on
> >      * this involves: fetch-patch.c, connected.c, bundle.c, clone.c,
> >        transport-helper.c, receive-pack.c, upload-pack.c
> >      * this accounts for the majority of this patch
>
> It does feel a bit like a bandaid to insert new code at these
> locations to set GIT_DIR manually. It's not clear to readers why
> GIT_DIR is needed for these specific cases, nor what the impact is
> when it is not set. Thus, one wonders if such a blanket approach is
> indeed required or if a more narrow and directed fix can be applied,
> such as calling subprograms with an explicit --git-dir= rather than
> setting GIT_DIR with its potentially more broad and
> difficult-to-reason-about impact.

I meant to ask here what was the nature of the various failures you
were seeing without GIT_DIR being set, and whether you had considered
tackling those failures with --git-dir= instead of GIT_DIR. If so,
were the problems too difficult to overcome by --git-dir= alone?
Fleshing out the commit message with such information might be
worthwhile.

By the way, I also didn't mean to imply that the
"difficult-to-reason-about GIT_DIR interaction in relation to
subprograms" problem is new to this patch; it isn't. That problem has
been around for ages (at least since 2007-08-01) but it wasn't as
obvious since the setting of GIT_DIR was so far removed from other
code which runs subprograms, thus readers likely wouldn't be thinking
about GIT_DIR when reading the code which runs subprograms. This patch
only makes the problem more obvious since readers now see the
sequence:

    (1) set GIT_DIR
    (2) launch subprogram

So, a reader is more likely to wonder why GIT_DIR is needed for the
subprogram and what possible wider side-effects it might have.

Re: [Bug] Rebase from worktree subdir is broken (was Re: [PATCH v5 07/11] rebase: do not attempt to remove startup_info->original_cwd)

[Elijah Newren]

On Sat, Feb 5, 2022 at 3:42 AM Eric Sunshine <sunshine@sunshineco.com> wrote:
>
> On Sat, Feb 5, 2022 at 6:23 AM Eric Sunshine <sunshine@sunshineco.com> wrote:
> > On Thu, Jan 27, 2022 at 3:03 PM Elijah Newren <newren@gmail.com> wrote:
> > > Some comments on the various code changes:
> > >    * clone/push/fetch related:
> > >      * there are *many* subprocesses involved in fetch/push and friends,
> > >        and they typically need to know the GIT_DIR they are operating on
> > >      * this involves: fetch-patch.c, connected.c, bundle.c, clone.c,
> > >        transport-helper.c, receive-pack.c, upload-pack.c
> > >      * this accounts for the majority of this patch
> >
> > It does feel a bit like a bandaid to insert new code at these
> > locations to set GIT_DIR manually. It's not clear to readers why
> > GIT_DIR is needed for these specific cases, nor what the impact is
> > when it is not set. Thus, one wonders if such a blanket approach is
> > indeed required or if a more narrow and directed fix can be applied,
> > such as calling subprograms with an explicit --git-dir= rather than
> > setting GIT_DIR with its potentially more broad and
> > difficult-to-reason-about impact.
>
> I meant to ask here what was the nature of the various failures you
> were seeing without GIT_DIR being set, and whether you had considered
> tackling those failures with --git-dir= instead of GIT_DIR. If so,
> were the problems too difficult to overcome by --git-dir= alone?
> Fleshing out the commit message with such information might be
> worthwhile.

clone, receive-pack, etc. will often spawn subprocesses such as
repack, or index-pack.  Those kinds of commands will not have been
invoked from within a particular git directory, so discovery in the
subprocess either doesn't find the git directory or finds the wrong
one (e.g. testcases where you're in a repo and do `git clone .
my-clone` would cause subprocesses to use discovery and find the outer
git repo rather than the 'my-clone' they are supposed to be working
on).

I didn't look at just --git-dir=.  I probably could have, but since I
was just trying to get a feel for how big of a change it was, and
there's the possibility of nested subprocesses (clone forking some
http-fetch thingy, which forks an index-pack subprocess, etc.), so I
just did the environment variable to save the work from diving down
the hierarchy.

I agree that if we want to make this kind of change, a better commit
message would probably be in order, and perhaps trying out --git-dir=
instead of setting the environment variable.  However, after seeing
the patch myself and thinking about it for another week, this just
doesn't seem like a change that's worth it to me.  If someone else
wants to pick up the patch and run with it, they should feel free, but
I'm not motivated enough to do so.

> By the way, I also didn't mean to imply that the
> "difficult-to-reason-about GIT_DIR interaction in relation to
> subprograms" problem is new to this patch; it isn't. That problem has
> been around for ages (at least since 2007-08-01) but it wasn't as
> obvious since the setting of GIT_DIR was so far removed from other
> code which runs subprograms, thus readers likely wouldn't be thinking
> about GIT_DIR when reading the code which runs subprograms. This patch
> only makes the problem more obvious since readers now see the
> sequence:
>
>     (1) set GIT_DIR
>     (2) launch subprogram
>
> So, a reader is more likely to wonder why GIT_DIR is needed for the
> subprogram and what possible wider side-effects it might have.

Yes, well put.

Re: [PATCH v6 01/11] t2501: add various tests for removing the current working directory

[Ævar Arnfjörð Bjarmason]

On Thu, Dec 09 2021, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
> [...]
> +test_expect_success 'git version continues working from a deleted dir' '
> +	mkdir tmp &&
> +	(
> +		cd tmp &&
> +		rm -rf ../tmp &&
> +		git version
> +	)
> +'

There's a (minor) regression here in v2.35.0 on AIX (as of 8a0d52dfd87
(t2501: add various tests for removing the current working directory,
2021-12-09)). The tooling on that OS will also refuse to remove the cwd:
    
    + mkdir tmp
    + cd tmp
    + rm -rf ../tmp
    rm: Cannot remove the current directory ../tmp.
    error: last command exited with $?=2
    not ok 21 - git version continues working from a deleted dir
    #

Just splitting that into a prereq that checks if the first two lines of
the sub-shell works should be a solution in this case, but I only had
time for this quick bug report :)