[PATCH 0/6] git-rm absorbs submodule git directory before deletion

[PATCH 0/6] git-rm absorbs submodule git directory before deletion

[Stefan Beller]

The "checkout --recurse-submodules" series got too large to comfortably send
it out for review, so I had to break it up into smaller series'; this is the
first subseries, but it makes sense on its own.

This series teaches git-rm to absorb the git directory of a submodule instead
of failing and complaining about the git directory preventing deletion.
 
It applies on origin/sb/submodule-embed-gitdir.

Any feedback welcome!

Thanks,
Stefan

Stefan Beller (6):
  submodule.h: add extern keyword to functions
  submodule: modernize ok_to_remove_submodule to use argv_array
  submodule: add flags to ok_to_remove_submodule
  ok_to_remove_submodule: absorb the submodule git dir
  t3600: slightly modernize style
  rm: add absorb a submodules git dir before deletion

 builtin/rm.c  |  21 +++-----
 cache.h       |   2 +
 entry.c       |   5 ++
 submodule.c   |  77 +++++++++++++++++++++++-----
 submodule.h   |  64 ++++++++++++++---------
 t/t3600-rm.sh | 159 +++++++++++++++++++++++----------------------------------
 6 files changed, 182 insertions(+), 146 deletions(-)

-- 
2.11.0.rc2.35.g7af3268

[PATCH 1/6] submodule.h: add extern keyword to functions

[Stefan Beller]

As the upcoming series will add a lot of functions to the submodule
header, let's first make the header consistent to the rest of the project
by adding the extern keyword to functions.

As per the CodingGuidelines we try to stay below 80 characters per line,
so adapt all those functions to stay below 80 characters that are already
using more than one line.  Those function using just one line are better
kept in one line than breaking them up into multiple lines just for the
goal of staying below the character limit as it makes grepping
for functions easier if they are one liners.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 submodule.h | 55 ++++++++++++++++++++++++++++++-------------------------
 1 file changed, 30 insertions(+), 25 deletions(-)

diff --git a/submodule.h b/submodule.h
index 6229054b99..61fb610749 100644
--- a/submodule.h
+++ b/submodule.h
@@ -29,50 +29,55 @@ struct submodule_update_strategy {
 };
 #define SUBMODULE_UPDATE_STRATEGY_INIT {SM_UPDATE_UNSPECIFIED, NULL}
 
-int is_staging_gitmodules_ok(void);
-int update_path_in_gitmodules(const char *oldpath, const char *newpath);
-int remove_path_from_gitmodules(const char *path);
-void stage_updated_gitmodules(void);
-void set_diffopt_flags_from_submodule_config(struct diff_options *diffopt,
+extern int is_staging_gitmodules_ok(void);
+extern int update_path_in_gitmodules(const char *oldpath, const char *newpath);
+extern int remove_path_from_gitmodules(const char *path);
+extern void stage_updated_gitmodules(void);
+extern void set_diffopt_flags_from_submodule_config(struct diff_options *,
 		const char *path);
-int submodule_config(const char *var, const char *value, void *cb);
-void gitmodules_config(void);
-int parse_submodule_update_strategy(const char *value,
+extern int submodule_config(const char *var, const char *value, void *cb);
+extern void gitmodules_config(void);
+extern int parse_submodule_update_strategy(const char *value,
 		struct submodule_update_strategy *dst);
-const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);
-void handle_ignore_submodules_arg(struct diff_options *diffopt, const char *);
-void show_submodule_summary(FILE *f, const char *path,
+extern const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);
+extern void handle_ignore_submodules_arg(struct diff_options *, const char *);
+extern void show_submodule_summary(FILE *f, const char *path,
 		const char *line_prefix,
 		struct object_id *one, struct object_id *two,
 		unsigned dirty_submodule, const char *meta,
 		const char *del, const char *add, const char *reset);
-void show_submodule_inline_diff(FILE *f, const char *path,
+extern void show_submodule_inline_diff(FILE *f, const char *path,
 		const char *line_prefix,
 		struct object_id *one, struct object_id *two,
 		unsigned dirty_submodule, const char *meta,
 		const char *del, const char *add, const char *reset,
 		const struct diff_options *opt);
-void set_config_fetch_recurse_submodules(int value);
-void check_for_new_submodule_commits(unsigned char new_sha1[20]);
-int fetch_populated_submodules(const struct argv_array *options,
+extern void set_config_fetch_recurse_submodules(int value);
+extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);
+extern int fetch_populated_submodules(const struct argv_array *options,
 			       const char *prefix, int command_line_option,
 			       int quiet, int max_parallel_jobs);
-unsigned is_submodule_modified(const char *path, int ignore_untracked);
-int submodule_uses_gitfile(const char *path);
-int ok_to_remove_submodule(const char *path);
-int merge_submodule(unsigned char result[20], const char *path, const unsigned char base[20],
-		    const unsigned char a[20], const unsigned char b[20], int search);
-int find_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_name,
-		struct string_list *needs_pushing);
-int push_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_name);
-int parallel_submodules(void);
+extern unsigned is_submodule_modified(const char *path, int ignore_untracked);
+extern int submodule_uses_gitfile(const char *path);
+extern int ok_to_remove_submodule(const char *path);
+extern int merge_submodule(unsigned char result[20], const char *path,
+			   const unsigned char base[20],
+			   const unsigned char a[20],
+			   const unsigned char b[20], int search);
+extern int find_unpushed_submodules(unsigned char new_sha1[20],
+				    const char *remotes_name,
+				    struct string_list *needs_pushing);
+extern int push_unpushed_submodules(unsigned char new_sha1[20],
+				    const char *remotes_name);
+extern void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);
+extern int parallel_submodules(void);
 
 /*
  * Prepare the "env_array" parameter of a "struct child_process" for executing
  * a submodule by clearing any repo-specific envirionment variables, but
  * retaining any config in the environment.
  */
-void prepare_submodule_repo_env(struct argv_array *out);
+extern void prepare_submodule_repo_env(struct argv_array *out);
 
 #define ABSORB_GITDIR_RECURSE_SUBMODULES (1<<0)
 extern void absorb_git_dir_into_superproject(const char *prefix,
-- 
2.11.0.rc2.35.g7af3268

[PATCH 2/6] submodule: modernize ok_to_remove_submodule to use argv_array

[Stefan Beller]

Instead of constructing the NULL terminated array ourselves, we
should make use of the argv_array infrastructure.

While at it, adapt the error messages to reflect the actual invocation.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 submodule.c | 14 ++++----------
 1 file changed, 4 insertions(+), 10 deletions(-)

diff --git a/submodule.c b/submodule.c
index 45ccfb7ab4..9f0b544ebe 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1023,13 +1023,6 @@ int ok_to_remove_submodule(const char *path)
 {
 	ssize_t len;
 	struct child_process cp = CHILD_PROCESS_INIT;
-	const char *argv[] = {
-		"status",
-		"--porcelain",
-		"-u",
-		"--ignore-submodules=none",
-		NULL,
-	};
 	struct strbuf buf = STRBUF_INIT;
 	int ok_to_remove = 1;
 
@@ -1039,14 +1032,15 @@ int ok_to_remove_submodule(const char *path)
 	if (!submodule_uses_gitfile(path))
 		return 0;
 
-	cp.argv = argv;
+	argv_array_pushl(&cp.args, "status", "--porcelain", "-u",
+				   "--ignore-submodules=none", NULL);
 	prepare_submodule_repo_env(&cp.env_array);
 	cp.git_cmd = 1;
 	cp.no_stdin = 1;
 	cp.out = -1;
 	cp.dir = path;
 	if (start_command(&cp))
-		die("Could not run 'git status --porcelain -uall --ignore-submodules=none' in submodule %s", path);
+		die(_("could not run 'git status --porcelain -u --ignore-submodules=none' in submodule %s"), path);
 
 	len = strbuf_read(&buf, cp.out, 1024);
 	if (len > 2)
@@ -1054,7 +1048,7 @@ int ok_to_remove_submodule(const char *path)
 	close(cp.out);
 
 	if (finish_command(&cp))
-		die("'git status --porcelain -uall --ignore-submodules=none' failed in submodule %s", path);
+		die(_("'git status --porcelain -u --ignore-submodules=none' failed in submodule %s"), path);
 
 	strbuf_release(&buf);
 	return ok_to_remove;
-- 
2.11.0.rc2.35.g7af3268

[PATCH 3/6] submodule: add flags to ok_to_remove_submodule

[Stefan Beller]

In different contexts the question whether deleting a submodule is ok
to remove may be answered differently.

In 293ab15eea (submodule: teach rm to remove submodules unless they
contain a git directory, 2012-09-26) a case was made that we can safely
ignore ignored untracked files for removal as we explicitely ask for the
removal of the submodule.

In a later patch we want to remove submodules even when the user doesn't
explicitly ask for it (e.g. checking out a tree-ish in which the submodule
doesn't exist).  In that case we want to be more careful when it comes
to deletion of untracked files. As of this patch it is unclear how this
will be implemented exactly, so we'll offer flags in which the caller
can specify how the different untracked files ought to be handled.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/rm.c |  3 ++-
 submodule.c  | 23 +++++++++++++++++++----
 submodule.h  |  5 ++++-
 3 files changed, 25 insertions(+), 6 deletions(-)

diff --git a/builtin/rm.c b/builtin/rm.c
index 3f3e24eb36..fdd7183f61 100644
--- a/builtin/rm.c
+++ b/builtin/rm.c
@@ -187,7 +187,8 @@ static int check_local_mod(struct object_id *head, int index_only)
 		 */
 		if (ce_match_stat(ce, &st, 0) ||
 		    (S_ISGITLINK(ce->ce_mode) &&
-		     !ok_to_remove_submodule(ce->name)))
+		     !ok_to_remove_submodule(ce->name,
+				SUBMODULE_REMOVAL_IGNORE_IGNORED_UNTRACKED)))
 			local_changes = 1;
 
 		/*
diff --git a/submodule.c b/submodule.c
index 9f0b544ebe..2d13744b06 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1019,7 +1019,7 @@ int submodule_uses_gitfile(const char *path)
 	return 1;
 }
 
-int ok_to_remove_submodule(const char *path)
+int ok_to_remove_submodule(const char *path, unsigned flags)
 {
 	ssize_t len;
 	struct child_process cp = CHILD_PROCESS_INIT;
@@ -1032,15 +1032,27 @@ int ok_to_remove_submodule(const char *path)
 	if (!submodule_uses_gitfile(path))
 		return 0;
 
-	argv_array_pushl(&cp.args, "status", "--porcelain", "-u",
+	argv_array_pushl(&cp.args, "status", "--porcelain",
 				   "--ignore-submodules=none", NULL);
+
+	if (flags & SUBMODULE_REMOVAL_IGNORE_UNTRACKED)
+		argv_array_push(&cp.args, "-uno");
+	else
+		argv_array_push(&cp.args, "-uall");
+
+	if (!(flags & SUBMODULE_REMOVAL_IGNORE_IGNORED_UNTRACKED))
+		argv_array_push(&cp.args, "--ignored");
+
 	prepare_submodule_repo_env(&cp.env_array);
 	cp.git_cmd = 1;
 	cp.no_stdin = 1;
 	cp.out = -1;
 	cp.dir = path;
 	if (start_command(&cp))
-		die(_("could not run 'git status --porcelain -u --ignore-submodules=none' in submodule %s"), path);
+		die(_("could not run 'git status --porcelain --ignore-submodules=none %s %s' in submodule '%s'"),
+			(flags & SUBMODULE_REMOVAL_IGNORE_UNTRACKED) ? "-uno" : "-uall",
+			(!(flags & SUBMODULE_REMOVAL_IGNORE_IGNORED_UNTRACKED)) ? "--ignored" : "",
+			path);
 
 	len = strbuf_read(&buf, cp.out, 1024);
 	if (len > 2)
@@ -1048,7 +1060,10 @@ int ok_to_remove_submodule(const char *path)
 	close(cp.out);
 
 	if (finish_command(&cp))
-		die(_("'git status --porcelain -u --ignore-submodules=none' failed in submodule %s"), path);
+		die(_("'git status --porcelain --ignore-submodules=none %s %s' failed in submodule '%s'"),
+			(flags & SUBMODULE_REMOVAL_IGNORE_UNTRACKED) ? "-uno" : "-uall",
+			(!(flags & SUBMODULE_REMOVAL_IGNORE_IGNORED_UNTRACKED)) ? "--ignored" : "",
+			path);
 
 	strbuf_release(&buf);
 	return ok_to_remove;
diff --git a/submodule.h b/submodule.h
index 61fb610749..3ed3aa479a 100644
--- a/submodule.h
+++ b/submodule.h
@@ -59,7 +59,10 @@ extern int fetch_populated_submodules(const struct argv_array *options,
 			       int quiet, int max_parallel_jobs);
 extern unsigned is_submodule_modified(const char *path, int ignore_untracked);
 extern int submodule_uses_gitfile(const char *path);
-extern int ok_to_remove_submodule(const char *path);
+
+#define SUBMODULE_REMOVAL_IGNORE_UNTRACKED (1<<0)
+#define SUBMODULE_REMOVAL_IGNORE_IGNORED_UNTRACKED (1<<1)
+extern int ok_to_remove_submodule(const char *path, unsigned flags);
 extern int merge_submodule(unsigned char result[20], const char *path,
 			   const unsigned char base[20],
 			   const unsigned char a[20],
-- 
2.11.0.rc2.35.g7af3268

[PATCH 4/6] ok_to_remove_submodule: absorb the submodule git dir

[Stefan Beller]

When deciding if a submodule can be deleted a major
reason to say no is the git directory of the submodule
being contained in the submodules working directory.

Instead of failing migrate the git directory into the
superproject and proceed with the other checks.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 submodule.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/submodule.c b/submodule.c
index 2d13744b06..e42efa2337 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1026,11 +1026,22 @@ int ok_to_remove_submodule(const char *path, unsigned flags)
 	struct strbuf buf = STRBUF_INIT;
 	int ok_to_remove = 1;
 
+	/* Is it there? */
 	if (!file_exists(path) || is_empty_dir(path))
 		return 1;
 
-	if (!submodule_uses_gitfile(path))
-		return 0;
+	/* Does it have a .git directory? */
+	if (!submodule_uses_gitfile(path)) {
+		absorb_git_dir_into_superproject("", path,
+			ABSORB_GITDIR_RECURSE_SUBMODULES);
+
+		/*
+		 * We should be using a gitfile by now. Let's double
+		 * check as losing the git dir would be fatal.
+		 */
+		if (!submodule_uses_gitfile(path))
+			return 0;
+	}
 
 	argv_array_pushl(&cp.args, "status", "--porcelain",
 				   "--ignore-submodules=none", NULL);
-- 
2.11.0.rc2.35.g7af3268

[PATCH 5/6] t3600: slightly modernize style

[Stefan Beller]

Remove the space between redirection and file name.
Also remove unnecessary invocations of subshells, such as

	(cd submod &&
		echo X >untracked
	) &&

as there is no point of having the shell for functional purposes.
In case of a single Git command use the `-C` option to let Git cd into
the directory.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 t/t3600-rm.sh | 122 ++++++++++++++++++++++++----------------------------------
 1 file changed, 50 insertions(+), 72 deletions(-)

diff --git a/t/t3600-rm.sh b/t/t3600-rm.sh
index 14f0edca2b..5e5a16c863 100755
--- a/t/t3600-rm.sh
+++ b/t/t3600-rm.sh
@@ -111,21 +111,21 @@ test_expect_success 'Remove nonexistent file with --ignore-unmatch' '
 '
 
 test_expect_success '"rm" command printed' '
-	echo frotz > test-file &&
+	echo frotz >test-file &&
 	git add test-file &&
 	git commit -m "add file for rm test" &&
-	git rm test-file > rm-output &&
+	git rm test-file >rm-output &&
 	test $(grep "^rm " rm-output | wc -l) = 1 &&
 	rm -f test-file rm-output &&
 	git commit -m "remove file from rm test"
 '
 
 test_expect_success '"rm" command suppressed with --quiet' '
-	echo frotz > test-file &&
+	echo frotz >test-file &&
 	git add test-file &&
 	git commit -m "add file for rm --quiet test" &&
-	git rm --quiet test-file > rm-output &&
-	test $(wc -l < rm-output) = 0 &&
+	git rm --quiet test-file >rm-output &&
+	test_must_be_empty rm-output &&
 	rm -f test-file rm-output &&
 	git commit -m "remove file from rm --quiet test"
 '
@@ -221,7 +221,7 @@ test_expect_success 'Call "rm" from outside the work tree' '
 	mkdir repo &&
 	(cd repo &&
 	 git init &&
-	 echo something > somefile &&
+	 echo something >somefile &&
 	 git add somefile &&
 	 git commit -m "add a file" &&
 	 (cd .. &&
@@ -287,7 +287,7 @@ test_expect_success 'rm removes empty submodules from work tree' '
 	git commit -m "add submodule" &&
 	git rm submod &&
 	test ! -e submod &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect actual &&
 	test_must_fail git config -f .gitmodules submodule.sub.url &&
 	test_must_fail git config -f .gitmodules submodule.sub.path
@@ -298,7 +298,7 @@ test_expect_success 'rm removes removed submodule from index and .gitmodules' '
 	git submodule update &&
 	rm -rf submod &&
 	git rm submod &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect actual &&
 	test_must_fail git config -f .gitmodules submodule.sub.url &&
 	test_must_fail git config -f .gitmodules submodule.sub.path
@@ -309,7 +309,7 @@ test_expect_success 'rm removes work tree of unmodified submodules' '
 	git submodule update &&
 	git rm submod &&
 	test ! -d submod &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect actual &&
 	test_must_fail git config -f .gitmodules submodule.sub.url &&
 	test_must_fail git config -f .gitmodules submodule.sub.path
@@ -320,7 +320,7 @@ test_expect_success 'rm removes a submodule with a trailing /' '
 	git submodule update &&
 	git rm submod/ &&
 	test ! -d submod &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect actual
 '
 
@@ -335,17 +335,15 @@ test_expect_success 'rm succeeds when given a directory with a trailing /' '
 test_expect_success 'rm of a populated submodule with different HEAD fails unless forced' '
 	git reset --hard &&
 	git submodule update &&
-	(cd submod &&
-		git checkout HEAD^
-	) &&
+	git -C submod checkout HEAD^ &&
 	test_must_fail git rm submod &&
 	test -d submod &&
 	test -f submod/.git &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect.modified actual &&
 	git rm -f submod &&
 	test ! -d submod &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect actual &&
 	test_must_fail git config -f .gitmodules submodule.sub.url &&
 	test_must_fail git config -f .gitmodules submodule.sub.path
@@ -418,34 +416,30 @@ test_expect_success 'rm issues a warning when section is not found in .gitmodule
 test_expect_success 'rm of a populated submodule with modifications fails unless forced' '
 	git reset --hard &&
 	git submodule update &&
-	(cd submod &&
-		echo X >empty
-	) &&
+	echo X >submod/empty &&
 	test_must_fail git rm submod &&
 	test -d submod &&
 	test -f submod/.git &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect.modified actual &&
 	git rm -f submod &&
 	test ! -d submod &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect actual
 '
 
 test_expect_success 'rm of a populated submodule with untracked files fails unless forced' '
 	git reset --hard &&
 	git submodule update &&
-	(cd submod &&
-		echo X >untracked
-	) &&
+	echo X >submod/untracked &&
 	test_must_fail git rm submod &&
 	test -d submod &&
 	test -f submod/.git &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect.modified actual &&
 	git rm -f submod &&
 	test ! -d submod &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect actual
 '
 
@@ -461,16 +455,12 @@ test_expect_success 'setup submodule conflict' '
 	git add nitfol &&
 	git commit -m "added nitfol 2" &&
 	git checkout -b conflict1 master &&
-	(cd submod &&
-		git fetch &&
-		git checkout branch1
-	) &&
+	git -C submod fetch &&
+	git -C submod checkout branch1 &&
 	git add submod &&
 	git commit -m "submod 1" &&
 	git checkout -b conflict2 master &&
-	(cd submod &&
-		git checkout branch2
-	) &&
+	git -C submod checkout branch2 &&
 	git add submod &&
 	git commit -m "submod 2"
 '
@@ -486,7 +476,7 @@ test_expect_success 'rm removes work tree of unmodified conflicted submodule' '
 	test_must_fail git merge conflict2 &&
 	git rm submod &&
 	test ! -d submod &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect actual
 '
 
@@ -494,18 +484,16 @@ test_expect_success 'rm of a conflicted populated submodule with different HEAD
 	git checkout conflict1 &&
 	git reset --hard &&
 	git submodule update &&
-	(cd submod &&
-		git checkout HEAD^
-	) &&
+	git -C submod checkout HEAD^ &&
 	test_must_fail git merge conflict2 &&
 	test_must_fail git rm submod &&
 	test -d submod &&
 	test -f submod/.git &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect.conflict actual &&
 	git rm -f submod &&
 	test ! -d submod &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect actual &&
 	test_must_fail git config -f .gitmodules submodule.sub.url &&
 	test_must_fail git config -f .gitmodules submodule.sub.path
@@ -515,18 +503,16 @@ test_expect_success 'rm of a conflicted populated submodule with modifications f
 	git checkout conflict1 &&
 	git reset --hard &&
 	git submodule update &&
-	(cd submod &&
-		echo X >empty
-	) &&
+	echo X >submod/empty &&
 	test_must_fail git merge conflict2 &&
 	test_must_fail git rm submod &&
 	test -d submod &&
 	test -f submod/.git &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect.conflict actual &&
 	git rm -f submod &&
 	test ! -d submod &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect actual &&
 	test_must_fail git config -f .gitmodules submodule.sub.url &&
 	test_must_fail git config -f .gitmodules submodule.sub.path
@@ -536,18 +522,16 @@ test_expect_success 'rm of a conflicted populated submodule with untracked files
 	git checkout conflict1 &&
 	git reset --hard &&
 	git submodule update &&
-	(cd submod &&
-		echo X >untracked
-	) &&
+	echo X >submod/untracked &&
 	test_must_fail git merge conflict2 &&
 	test_must_fail git rm submod &&
 	test -d submod &&
 	test -f submod/.git &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect.conflict actual &&
 	git rm -f submod &&
 	test ! -d submod &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect actual
 '
 
@@ -564,12 +548,12 @@ test_expect_success 'rm of a conflicted populated submodule with a .git director
 	test_must_fail git rm submod &&
 	test -d submod &&
 	test -d submod/.git &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect.conflict actual &&
 	test_must_fail git rm -f submod &&
 	test -d submod &&
 	test -d submod/.git &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect.conflict actual &&
 	git merge --abort &&
 	rm -rf submod
@@ -581,7 +565,7 @@ test_expect_success 'rm of a conflicted unpopulated submodule succeeds' '
 	test_must_fail git merge conflict2 &&
 	git rm submod &&
 	test ! -d submod &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect actual
 '
 
@@ -597,12 +581,12 @@ test_expect_success 'rm of a populated submodule with a .git directory fails eve
 	test_must_fail git rm submod &&
 	test -d submod &&
 	test -d submod/.git &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	! test -s actual &&
 	test_must_fail git rm -f submod &&
 	test -d submod &&
 	test -d submod/.git &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	! test -s actual &&
 	rm -rf submod
 '
@@ -629,58 +613,52 @@ test_expect_success 'setup subsubmodule' '
 test_expect_success 'rm recursively removes work tree of unmodified submodules' '
 	git rm submod &&
 	test ! -d submod &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect actual
 '
 
 test_expect_success 'rm of a populated nested submodule with different nested HEAD fails unless forced' '
 	git reset --hard &&
 	git submodule update --recursive &&
-	(cd submod/subsubmod &&
-		git checkout HEAD^
-	) &&
+	git -C submod/subsubmod checkout HEAD^ &&
 	test_must_fail git rm submod &&
 	test -d submod &&
 	test -f submod/.git &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect.modified actual &&
 	git rm -f submod &&
 	test ! -d submod &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect actual
 '
 
 test_expect_success 'rm of a populated nested submodule with nested modifications fails unless forced' '
 	git reset --hard &&
 	git submodule update --recursive &&
-	(cd submod/subsubmod &&
-		echo X >empty
-	) &&
+	echo X >submod/subsubmod/empty &&
 	test_must_fail git rm submod &&
 	test -d submod &&
 	test -f submod/.git &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect.modified actual &&
 	git rm -f submod &&
 	test ! -d submod &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect actual
 '
 
 test_expect_success 'rm of a populated nested submodule with nested untracked files fails unless forced' '
 	git reset --hard &&
 	git submodule update --recursive &&
-	(cd submod/subsubmod &&
-		echo X >untracked
-	) &&
+	echo X >submod/subsubmod/untracked &&
 	test_must_fail git rm submod &&
 	test -d submod &&
 	test -f submod/.git &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect.modified actual &&
 	git rm -f submod &&
 	test ! -d submod &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	test_cmp expect actual
 '
 
@@ -695,12 +673,12 @@ test_expect_success 'rm of a populated nested submodule with a nested .git direc
 	test_must_fail git rm submod &&
 	test -d submod &&
 	test -d submod/subsubmod/.git &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	! test -s actual &&
 	test_must_fail git rm -f submod &&
 	test -d submod &&
 	test -d submod/subsubmod/.git &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	! test -s actual &&
 	rm -rf submod
 '
@@ -716,7 +694,7 @@ test_expect_success 'checking out a commit after submodule removal needs manual
 	echo "?? submod/" >expected &&
 	test_cmp expected actual &&
 	rm -rf submod &&
-	git status -s -uno --ignore-submodules=none > actual &&
+	git status -s -uno --ignore-submodules=none >actual &&
 	! test -s actual
 '
 
-- 
2.11.0.rc2.35.g7af3268

[PATCH 6/6] rm: add absorb a submodules git dir before deletion

[Stefan Beller]

When deleting a submodule we need to keep the actual git directory around,
such that we do not lose local changes in there and at a later checkout
of the submodule we don't need to clone it again.

Implement `depopulate_submodule`, that migrates the git directory before
deletion of a submodule and afterwards the equivalent of "rm -rf", which
is already found in entry.c, so expose that and for clarity add a suffix
"_or_dir" to it.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/rm.c  | 18 ++++++------------
 cache.h       |  2 ++
 entry.c       |  5 +++++
 submodule.c   | 31 +++++++++++++++++++++++++++++++
 submodule.h   |  6 ++++++
 t/t3600-rm.sh | 41 ++++++++++++++++-------------------------
 6 files changed, 66 insertions(+), 37 deletions(-)

diff --git a/builtin/rm.c b/builtin/rm.c
index fdd7183f61..f8c5e9b6c6 100644
--- a/builtin/rm.c
+++ b/builtin/rm.c
@@ -400,18 +400,12 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
 						continue;
 					}
 				} else {
-					strbuf_reset(&buf);
-					strbuf_addstr(&buf, path);
-					if (!remove_dir_recursively(&buf, 0)) {
-						removed = 1;
-						if (!remove_path_from_gitmodules(path))
-							gitmodules_modified = 1;
-						strbuf_release(&buf);
-						continue;
-					} else if (!file_exists(path))
-						/* Submodule was removed by user */
-						if (!remove_path_from_gitmodules(path))
-							gitmodules_modified = 1;
+					if (file_exists(path))
+						depopulate_submodule(path);
+					removed = 1;
+					if (!remove_path_from_gitmodules(path))
+						gitmodules_modified = 1;
+					continue;
 					/* Fallthrough and let remove_path() fail. */
 				}
 			}
diff --git a/cache.h b/cache.h
index a50a61a197..b645ca2f9a 100644
--- a/cache.h
+++ b/cache.h
@@ -2018,4 +2018,6 @@ void sleep_millisec(int millisec);
  */
 void safe_create_dir(const char *dir, int share);
 
+extern void remove_directory_or_die(struct strbuf *path);
+
 #endif /* CACHE_H */
diff --git a/entry.c b/entry.c
index c6eea240b6..02c4ac9f22 100644
--- a/entry.c
+++ b/entry.c
@@ -73,6 +73,11 @@ static void remove_subtree(struct strbuf *path)
 		die_errno("cannot rmdir '%s'", path->buf);
 }
 
+void remove_directory_or_die(struct strbuf *path)
+{
+	remove_subtree(path);
+}
+
 static int create_file(const char *path, unsigned int mode)
 {
 	mode = (mode & 0100) ? 0777 : 0666;
diff --git a/submodule.c b/submodule.c
index e42efa2337..3770ecb7b9 100644
--- a/submodule.c
+++ b/submodule.c
@@ -308,6 +308,37 @@ static void print_submodule_summary(struct rev_info *rev, FILE *f,
 	strbuf_release(&sb);
 }
 
+void depopulate_submodule(const char *path)
+{
+	struct strbuf pathbuf = STRBUF_INIT;
+	char *dot_git = xstrfmt("%s/.git", path);
+
+	/* Is it populated? */
+	if (!resolve_gitdir(dot_git))
+		goto out;
+
+	/* Does it have a .git directory? */
+	if (!submodule_uses_gitfile(path)) {
+		absorb_git_dir_into_superproject("", path,
+			ABSORB_GITDIR_RECURSE_SUBMODULES);
+
+		if (!submodule_uses_gitfile(path)) {
+			/*
+			 * We should be using a gitfile by now. Let's double
+			 * check as losing the git dir would be fatal.
+			 */
+			die("BUG: could not absorb git directory for '%s'", path);
+		}
+	}
+
+	strbuf_addstr(&pathbuf, path);
+	remove_directory_or_die(&pathbuf);
+
+out:
+	strbuf_release(&pathbuf);
+	free(dot_git);
+}
+
 /* Helper function to display the submodule header line prior to the full
  * summary output. If it can locate the submodule objects directory it will
  * attempt to lookup both the left and right commits and put them into the
diff --git a/submodule.h b/submodule.h
index 3ed3aa479a..516e377a12 100644
--- a/submodule.h
+++ b/submodule.h
@@ -53,6 +53,12 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,
 		const char *del, const char *add, const char *reset,
 		const struct diff_options *opt);
 extern void set_config_fetch_recurse_submodules(int value);
+
+/*
+ * Removes a submodule from a given path. When the submodule contains its
+ * git directory instead of a gitlink, migrate that first into the superproject.
+ */
+extern void depopulate_submodule(const char *path);
 extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);
 extern int fetch_populated_submodules(const struct argv_array *options,
 			       const char *prefix, int command_line_option,
diff --git a/t/t3600-rm.sh b/t/t3600-rm.sh
index 5e5a16c863..5aa6db584c 100755
--- a/t/t3600-rm.sh
+++ b/t/t3600-rm.sh
@@ -569,26 +569,22 @@ test_expect_success 'rm of a conflicted unpopulated submodule succeeds' '
 	test_cmp expect actual
 '
 
-test_expect_success 'rm of a populated submodule with a .git directory fails even when forced' '
+test_expect_success 'rm of a populated submodule with a .git directory migrates git dir' '
 	git checkout -f master &&
 	git reset --hard &&
 	git submodule update &&
 	(cd submod &&
 		rm .git &&
 		cp -R ../.git/modules/sub .git &&
-		GIT_WORK_TREE=. git config --unset core.worktree
+		GIT_WORK_TREE=. git config --unset core.worktree &&
+		rm -r ../.git/modules/sub
 	) &&
-	test_must_fail git rm submod &&
-	test -d submod &&
-	test -d submod/.git &&
-	git status -s -uno --ignore-submodules=none >actual &&
-	! test -s actual &&
-	test_must_fail git rm -f submod &&
-	test -d submod &&
-	test -d submod/.git &&
+	git rm submod 2>output.err &&
+	! test -d submod &&
+	! test -d submod/.git &&
 	git status -s -uno --ignore-submodules=none >actual &&
-	! test -s actual &&
-	rm -rf submod
+	test -s actual &&
+	test_i18ngrep Migrating output.err
 '
 
 cat >expect.deepmodified <<EOF
@@ -667,27 +663,22 @@ test_expect_success 'rm of a populated nested submodule with a nested .git direc
 	git submodule update --recursive &&
 	(cd submod/subsubmod &&
 		rm .git &&
-		cp -R ../../.git/modules/sub/modules/sub .git &&
+		mv ../../.git/modules/sub/modules/sub .git &&
 		GIT_WORK_TREE=. git config --unset core.worktree
 	) &&
-	test_must_fail git rm submod &&
-	test -d submod &&
-	test -d submod/subsubmod/.git &&
-	git status -s -uno --ignore-submodules=none >actual &&
-	! test -s actual &&
-	test_must_fail git rm -f submod &&
-	test -d submod &&
-	test -d submod/subsubmod/.git &&
+	git rm submod 2>output.err &&
+	! test -d submod &&
+	! test -d submod/subsubmod/.git &&
 	git status -s -uno --ignore-submodules=none >actual &&
-	! test -s actual &&
-	rm -rf submod
+	test -s actual &&
+	test_i18ngrep Migrating output.err
 '
 
 test_expect_success 'checking out a commit after submodule removal needs manual updates' '
-	git commit -m "submodule removal" submod &&
+	git commit -m "submodule removal" submod .gitmodules &&
 	git checkout HEAD^ &&
 	git submodule update &&
-	git checkout -q HEAD^ 2>actual &&
+	git checkout -q HEAD^ &&
 	git checkout -q master 2>actual &&
 	test_i18ngrep "^warning: unable to rmdir submod:" actual &&
 	git status -s submod >actual &&
-- 
2.11.0.rc2.35.g7af3268

Re: [PATCH 6/6] rm: add absorb a submodules git dir before deletion

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 915 bytes --]

On Mon, Dec 12, 2016 at 05:40:55PM -0800, Stefan Beller wrote:
> When deleting a submodule we need to keep the actual git directory around,
> such that we do not lose local changes in there and at a later checkout
> of the submodule we don't need to clone it again.
> 
> Implement `depopulate_submodule`, that migrates the git directory before
> deletion of a submodule and afterwards the equivalent of "rm -rf", which
> is already found in entry.c, so expose that and for clarity add a suffix
> "_or_dir" to it.

I think you might have meant "_or_die" here.

Other than that, I didn't see anything that stuck out to me in this
series as obviously wrong or broken, but this isn't an area of the code
I'm super familiar with.
-- 
brian m. carlson / brian with sandals: Houston, Texas, US
+1 832 623 2791 | https://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: https://keybase.io/bk2204

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 868 bytes --]

Re: [PATCH 6/6] rm: add absorb a submodules git dir before deletion

[Stefan Beller]

On Mon, Dec 12, 2016 at 7:28 PM, brian m. carlson
<sandals@crustytoothpaste.net> wrote:
> On Mon, Dec 12, 2016 at 05:40:55PM -0800, Stefan Beller wrote:
>> When deleting a submodule we need to keep the actual git directory around,
>> such that we do not lose local changes in there and at a later checkout
>> of the submodule we don't need to clone it again.
>>
>> Implement `depopulate_submodule`, that migrates the git directory before
>> deletion of a submodule and afterwards the equivalent of "rm -rf", which
>> is already found in entry.c, so expose that and for clarity add a suffix
>> "_or_dir" to it.
>
> I think you might have meant "_or_die" here.

indeed, will fix in a reroll. Thanks for the review!

RE: [PATCH 6/6] rm: add absorb a submodules git dir before deletion

[David Turner]

> -----Original Message-----
> From: Stefan Beller [mailto:sbeller@google.com]
> Sent: Monday, December 12, 2016 8:41 PM
> To: gitster@pobox.com
> Cc: git@vger.kernel.org; David Turner; bmwill@google.com; Stefan Beller
> Subject: [PATCH 6/6] rm: add absorb a submodules git dir before deletion
> 
> When deleting a submodule we need to keep the actual git directory around,

Nit: comma after submodule.

> -					strbuf_reset(&buf);
> -					strbuf_addstr(&buf, path);
> -					if (!remove_dir_recursively(&buf, 0)) {
> -						removed = 1;
> -						if
> (!remove_path_from_gitmodules(path))
> -							gitmodules_modified = 1;
> -						strbuf_release(&buf);
> -						continue;
> -					} else if (!file_exists(path))
> -						/* Submodule was removed by user */
> -						if
> (!remove_path_from_gitmodules(path))
> -							gitmodules_modified = 1;
> +					if (file_exists(path))
> +						depopulate_submodule(path);
> +					removed = 1;
> +					if (!remove_path_from_gitmodules(path))
> +						gitmodules_modified = 1;
> +					continue;
>  					/* Fallthrough and let remove_path() fail.
> */

It seems odd to have a continue right before a comment that says "Fallthrough".  

Re: [PATCH 0/6] git-rm absorbs submodule git directory before deletion

[Junio C Hamano]

Stefan Beller <sbeller@google.com> writes:

> The "checkout --recurse-submodules" series got too large to comfortably send
> it out for review, so I had to break it up into smaller series'; this is the
> first subseries, but it makes sense on its own.
>
> This series teaches git-rm to absorb the git directory of a submodule instead
> of failing and complaining about the git directory preventing deletion.
>  
> It applies on origin/sb/submodule-embed-gitdir.

Thanks.  I probably should rename the topic again with s/embed/absorb/;

> Any feedback welcome!
>
> Thanks,
> Stefan
>
> Stefan Beller (6):
>   submodule.h: add extern keyword to functions
>   submodule: modernize ok_to_remove_submodule to use argv_array
>   submodule: add flags to ok_to_remove_submodule
>   ok_to_remove_submodule: absorb the submodule git dir
>   t3600: slightly modernize style
>   rm: add absorb a submodules git dir before deletion
>
>  builtin/rm.c  |  21 +++-----
>  cache.h       |   2 +
>  entry.c       |   5 ++
>  submodule.c   |  77 +++++++++++++++++++++++-----
>  submodule.h   |  64 ++++++++++++++---------
>  t/t3600-rm.sh | 159 +++++++++++++++++++++++----------------------------------
>  6 files changed, 182 insertions(+), 146 deletions(-)

Re: [PATCH 0/6] git-rm absorbs submodule git directory before deletion

[Stefan Beller]

On Mon, Dec 12, 2016 at 11:28 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> The "checkout --recurse-submodules" series got too large to comfortably send
>> it out for review, so I had to break it up into smaller series'; this is the
>> first subseries, but it makes sense on its own.
>>
>> This series teaches git-rm to absorb the git directory of a submodule instead
>> of failing and complaining about the git directory preventing deletion.
>>
>> It applies on origin/sb/submodule-embed-gitdir.
>
> Thanks.  I probably should rename the topic again with s/embed/absorb/;

I mostly renamed it in the hope to settle our dispute what embedding means. ;)
So in case we want to further discuss on the name of the function, we should
do that before doing actual work such as renaming.

Note that sb/t3600-cleanup is part of this series now,
(The first commit of that series is in patch 6/6 of this series, and patch 5 is
the modernization effort.)

Thanks,
Stefan

Re: [PATCH 0/6] git-rm absorbs submodule git directory before deletion

[Junio C Hamano]

Stefan Beller <sbeller@google.com> writes:

> On Mon, Dec 12, 2016 at 11:28 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Stefan Beller <sbeller@google.com> writes:
>>
>>> The "checkout --recurse-submodules" series got too large to comfortably send
>>> it out for review, so I had to break it up into smaller series'; this is the
>>> first subseries, but it makes sense on its own.
>>>
>>> This series teaches git-rm to absorb the git directory of a submodule instead
>>> of failing and complaining about the git directory preventing deletion.
>>>
>>> It applies on origin/sb/submodule-embed-gitdir.
>>
>> Thanks.  I probably should rename the topic again with s/embed/absorb/;
>
> I mostly renamed it in the hope to settle our dispute what embedding means. ;)

I do not think there is no dispute about what embedding means.  A
submodule whose .git is inside its working tree has its repository
embedded.

What we had trouble settling on was what to call the operation to
undo the embedding, unentangling its repository out of the working
tree.  I'd still vote for unembed if you want a name to be nominated.

> Note that sb/t3600-cleanup is part of this series now,
> (The first commit of that series is in patch 6/6 of this series, and patch 5 is
> the modernization effort.)

Well, "t3600: remove useless redirect" has been in 'next' already;
do you mean that you want me to queue this series on top of that
topic?

Re: [PATCH 0/6] git-rm absorbs submodule git directory before deletion

[Stefan Beller]

On Tue, Dec 13, 2016 at 10:53 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> On Mon, Dec 12, 2016 at 11:28 PM, Junio C Hamano <gitster@pobox.com> wrote:
>>> Stefan Beller <sbeller@google.com> writes:
>>>
>>>> The "checkout --recurse-submodules" series got too large to comfortably send
>>>> it out for review, so I had to break it up into smaller series'; this is the
>>>> first subseries, but it makes sense on its own.
>>>>
>>>> This series teaches git-rm to absorb the git directory of a submodule instead
>>>> of failing and complaining about the git directory preventing deletion.
>>>>
>>>> It applies on origin/sb/submodule-embed-gitdir.
>>>
>>> Thanks.  I probably should rename the topic again with s/embed/absorb/;
>>
>> I mostly renamed it in the hope to settle our dispute what embedding means. ;)
>
> I do not think there is no dispute about what embedding means.

double negative: You think we have a slight dispute here.

>  A
> submodule whose .git is inside its working tree has its repository
> embedded.
>
> What we had trouble settling on was what to call the operation to
> undo the embedding, unentangling its repository out of the working
> tree.  I'd still vote for unembed if you want a name to be nominated.

So I can redo the series with two commands "git submodule [un]embed".

For me "unembed" == "absorb", such that we could also go with
absorb into superproject <-> embed into worktree


>
>> Note that sb/t3600-cleanup is part of this series now,
>> (The first commit of that series is in patch 6/6 of this series, and patch 5 is
>> the modernization effort.)
>
> Well, "t3600: remove useless redirect" has been in 'next' already;
> do you mean that you want me to queue this series on top of that
> topic?
>

I need to reroll this series any way; the reroll will be on top of that.

Thanks,
Stefan

Re: [PATCH 0/6] git-rm absorbs submodule git directory before deletion

[Junio C Hamano]

Stefan Beller <sbeller@google.com> writes:

>> I do not think there is no dispute about what embedding means.
>
> double negative: You think we have a slight dispute here.

Sorry, I do not think there is any dispute on that.

>>  A
>> submodule whose .git is inside its working tree has its repository
>> embedded.
>>
>> What we had trouble settling on was what to call the operation to
>> undo the embedding, unentangling its repository out of the working
>> tree.  I'd still vote for unembed if you want a name to be nominated.
>
> So I can redo the series with two commands "git submodule [un]embed".
>
> For me "unembed" == "absorb", such that we could also go with
> absorb into superproject <-> embed into worktree

With us agreeing that "embed" is about something is _IN_ submodule
working tree, unembed would naturally be something becomes OUTSIDE
the same thing (i.e. "submodule working tree").  However, if you
introduce "absorb", we suddenly need to talk about a different
thing, i.e. "superproject's .git/modules", that is doing the
absorption.  That is why I suggest "unembed" over "absorb".

Re: [PATCH 0/6] git-rm absorbs submodule git directory before deletion

[Stefan Beller]

On Tue, Dec 13, 2016 at 11:11 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>>> I do not think there is no dispute about what embedding means.
>>
>> double negative: You think we have a slight dispute here.
>
> Sorry, I do not think there is any dispute on that.
>
>>>  A
>>> submodule whose .git is inside its working tree has its repository
>>> embedded.
>>>
>>> What we had trouble settling on was what to call the operation to
>>> undo the embedding, unentangling its repository out of the working
>>> tree.  I'd still vote for unembed if you want a name to be nominated.
>>
>> So I can redo the series with two commands "git submodule [un]embed".
>>
>> For me "unembed" == "absorb", such that we could also go with
>> absorb into superproject <-> embed into worktree
>
> With us agreeing that "embed" is about something is _IN_ submodule
> working tree, unembed would naturally be something becomes OUTSIDE
> the same thing (i.e. "submodule working tree").  However, if you
> introduce "absorb", we suddenly need to talk about a different
> thing, i.e. "superproject's .git/modules", that is doing the
> absorption.  That is why I suggest "unembed" over "absorb".

ok, I will take unembed then.  We could also go with more command line options
such as "embed --reverse" or such, but that is not as nice I'd think.

Re: [PATCH 0/6] git-rm absorbs submodule git directory before deletion

[Stefan Beller]

On Tue, Dec 13, 2016 at 11:13 AM, Stefan Beller <sbeller@google.com> wrote:
> On Tue, Dec 13, 2016 at 11:11 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> Stefan Beller <sbeller@google.com> writes:
>>
>>>> I do not think there is no dispute about what embedding means.
>>>
>>> double negative: You think we have a slight dispute here.
>>
>> Sorry, I do not think there is any dispute on that.
>>
>>>>  A
>>>> submodule whose .git is inside its working tree has its repository
>>>> embedded.
>>>>
>>>> What we had trouble settling on was what to call the operation to
>>>> undo the embedding, unentangling its repository out of the working
>>>> tree.  I'd still vote for unembed if you want a name to be nominated.
>>>
>>> So I can redo the series with two commands "git submodule [un]embed".
>>>
>>> For me "unembed" == "absorb", such that we could also go with
>>> absorb into superproject <-> embed into worktree
>>
>> With us agreeing that "embed" is about something is _IN_ submodule
>> working tree, unembed would naturally be something becomes OUTSIDE
>> the same thing (i.e. "submodule working tree").

I do not agree, yet.
So I thought about this for a while.

The standard in Git is to have the .git directory inside the working tree,
which is why you are convinced that embedded means the .git is in the
working tree, because you approach this discussion as the Git maintainer,
spending only little time on submodule related stuff.

The desired standard for submodules is to have the git dir inside the
superprojects git dir (since  501770e, Aug 2011, Move git-dir for
submodules), which is why I think an "embedded submodule git dir"
is inside the superproject already.

I think both views are legit, and we would want to choose the one that
users find most intuitive (and I think there will be users that find either
viewpoint intuitive).

So when you have typed "git submodule ", I wonder if a user would
assume a submodule-centric mindset of how submodules ought to
work or if they still look at a submodule as its own git repo
that just happens to be embedded into the superproject.

I guess the latter is the case, so embedding is actually inside the working
tree and un-embedding is the relocation to the superproject.

Re: [PATCH 0/6] git-rm absorbs submodule git directory before deletion

[Junio C Hamano]

Stefan Beller <sbeller@google.com> writes:

> The desired standard for submodules is to have the git dir inside the
> superprojects git dir (since  501770e, Aug 2011, Move git-dir for
> submodules), which is why I think an "embedded submodule git dir"
> is inside the superproject already.

Think how you start a new submodule.  What are the steps you take
before you say "git submodule add"?  And where does .git for the
submodule live at that point?

With the current system, you as the submodule originator need to do
something different to make your working tree of the superproject
match what the others who clone from your public repository.

And comparing the two layout, the one originally held by the
submodule originator has .git embedded in the working tree, no?

All of the above is coming from "submodule" centric mindset.  It
just is not centric to those who follow what others originated.

Re: [PATCH 0/6] git-rm absorbs submodule git directory before deletion

[Stefan Beller]

On Tue, Dec 13, 2016 at 11:47 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> The desired standard for submodules is to have the git dir inside the
>> superprojects git dir (since  501770e, Aug 2011, Move git-dir for
>> submodules), which is why I think an "embedded submodule git dir"
>> is inside the superproject already.
>
> Think how you start a new submodule.  What are the steps you take
> before you say "git submodule add"?  And where does .git for the
> submodule live at that point?

Well there is no way to directly start a submodule
(e.g. "git submodule create"), such that there has to be one repo
that actually has the git dir inside the working tree.

If the submodule exists already somewhere, there are 2 ways to do it
("git submodule add <URL>"  or "git clone && git submodule add")
which lead to different outcomes, where the .git resides.

> With the current system, you as the submodule originator need to do
> something different to make your working tree of the superproject
> match what the others who clone from your public repository.
>
> And comparing the two layout, the one originally held by the
> submodule originator has .git embedded in the working tree, no?

When starting a new submodule repo, yes, the git dir is inside
the working tree.

> All of the above is coming from "submodule" centric mindset.  It
> just is not centric to those who follow what others originated.

ok.

> Another reason why I personally see a .git in each submodule working
> tree is "embedded" has nothing to do with Git.  It is an analogy I
> feel (perhaps it is just me) with the word "embedded reporters in
> warzone".  These people are spread around, assigned to units to
> report from places closer to the front line and being closer to the
> leaf of the hierarchy, as opposed to be assigned to a more central
> place like HQ to do their reporting.

I talked to people in the office and got a heavy rejection on the
the work "embedded" here for another reason:

    "Does it put the submodule on an embedded device?
    What does embedded even mean?
    The end user is super confused"

So I think we should not use embed or un-embed one way or the other.
Instead we need to have another name.

I think absorb is ok-ish, as "git submodule absorb" hints that the
superproject absorbs something from the submodule.

So I will reroll it with "absorb" fixing some nits pointed out by David?

Re: [PATCH 0/6] git-rm absorbs submodule git directory before deletion

[Stefan Beller]

On Tue, Dec 13, 2016 at 12:09 PM, Stefan Beller <sbeller@google.com> wrote:
>
> So I will reroll it with "absorb" fixing some nits pointed out by David?

I got confused there, Davids nits are for this series, the absorb series itself
doesn't seem to have nits.

So I'll just reroll this series on top of the currently
sb/submodule-embed-gitdir (which you originally noted to be better renamed to
submodule-absorb-gitdir) merged with t3600-cleanup.

Thanks,
Stefan

Re: [PATCH 0/6] git-rm absorbs submodule git directory before deletion

[Junio C Hamano]

Stefan Beller <sbeller@google.com> writes:

> I guess the latter is the case, so embedding is actually inside
> the working tree and un-embedding is the relocation to the
> superproject.

Another reason why I personally see a .git in each submodule working
tree is "embedded" has nothing to do with Git.  It is an analogy I
feel (perhaps it is just me) with the word "embedded reporters in
warzone".  These people are spread around, assigned to units to
report from places closer to the front line and being closer to the
leaf of the hierarchy, as opposed to be assigned to a more central
place like HQ to do their reporting.