* [PATCHv8 1/6] submodule: use absolute path for computing relative path connecting
2016-12-12 19:04 [PATCHv7 0/6] submodule absorbgitdirs Stefan Beller
@ 2016-12-12 19:04 ` Stefan Beller
2016-12-12 19:04 ` [PATCHv8 2/6] submodule helper: support super prefix Stefan Beller
` (5 subsequent siblings)
6 siblings, 0 replies; 12+ messages in thread
From: Stefan Beller @ 2016-12-12 19:04 UTC (permalink / raw)
To: gitster; +Cc: git, bmwill, pclouds, Stefan Beller
The current caller of connect_work_tree_and_git_dir passes
an absolute path for the `git_dir` parameter. In the future patch
we will also pass in relative path for `git_dir`. Extend the functionality
of connect_work_tree_and_git_dir to take relative paths for parameters.
We could work around this in the future patch by computing the absolute
path for the git_dir in the calling site, however accepting relative
paths for either parameter makes the API for this function much harder
to misuse.
Signed-off-by: Stefan Beller <sbeller@google.com>
---
submodule.c | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/submodule.c b/submodule.c
index 6f7d883de9..d4f7afe2f1 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1223,27 +1223,28 @@ int merge_submodule(unsigned char result[20], const char *path,
}
/* Update gitfile and core.worktree setting to connect work tree and git dir */
-void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir)
+void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)
{
struct strbuf file_name = STRBUF_INIT;
struct strbuf rel_path = STRBUF_INIT;
- const char *real_work_tree = xstrdup(real_path(work_tree));
+ char *git_dir = xstrdup(real_path(git_dir_));
+ char *work_tree = xstrdup(real_path(work_tree_));
/* Update gitfile */
strbuf_addf(&file_name, "%s/.git", work_tree);
write_file(file_name.buf, "gitdir: %s",
- relative_path(git_dir, real_work_tree, &rel_path));
+ relative_path(git_dir, work_tree, &rel_path));
/* Update core.worktree setting */
strbuf_reset(&file_name);
strbuf_addf(&file_name, "%s/config", git_dir);
git_config_set_in_file(file_name.buf, "core.worktree",
- relative_path(real_work_tree, git_dir,
- &rel_path));
+ relative_path(work_tree, git_dir, &rel_path));
strbuf_release(&file_name);
strbuf_release(&rel_path);
- free((void *)real_work_tree);
+ free(work_tree);
+ free(git_dir);
}
int parallel_submodules(void)
--
2.11.0.rc2.49.ge1f3b0c.dirty
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCHv8 2/6] submodule helper: support super prefix
2016-12-12 19:04 [PATCHv7 0/6] submodule absorbgitdirs Stefan Beller
2016-12-12 19:04 ` [PATCHv8 1/6] submodule: use absolute path for computing relative path connecting Stefan Beller
@ 2016-12-12 19:04 ` Stefan Beller
2016-12-12 19:04 ` [PATCHv8 3/6] test-lib-functions.sh: teach test_commit -C <dir> Stefan Beller
` (4 subsequent siblings)
6 siblings, 0 replies; 12+ messages in thread
From: Stefan Beller @ 2016-12-12 19:04 UTC (permalink / raw)
To: gitster; +Cc: git, bmwill, pclouds, Stefan Beller
Just like main commands in Git, the submodule helper needs
access to the superproject prefix. Enable this in the git.c
but have its own fuse in the helper code by having a flag to
turn on the super prefix.
Signed-off-by: Stefan Beller <sbeller@google.com>
---
builtin/submodule--helper.c | 31 ++++++++++++++++++++-----------
git.c | 2 +-
2 files changed, 21 insertions(+), 12 deletions(-)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 4beeda5f9f..5643848667 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1076,21 +1076,24 @@ static int resolve_remote_submodule_branch(int argc, const char **argv,
return 0;
}
+#define SUPPORT_SUPER_PREFIX (1<<0)
+
struct cmd_struct {
const char *cmd;
int (*fn)(int, const char **, const char *);
+ unsigned option;
};
static struct cmd_struct commands[] = {
- {"list", module_list},
- {"name", module_name},
- {"clone", module_clone},
- {"update-clone", update_clone},
- {"relative-path", resolve_relative_path},
- {"resolve-relative-url", resolve_relative_url},
- {"resolve-relative-url-test", resolve_relative_url_test},
- {"init", module_init},
- {"remote-branch", resolve_remote_submodule_branch}
+ {"list", module_list, 0},
+ {"name", module_name, 0},
+ {"clone", module_clone, 0},
+ {"update-clone", update_clone, 0},
+ {"relative-path", resolve_relative_path, 0},
+ {"resolve-relative-url", resolve_relative_url, 0},
+ {"resolve-relative-url-test", resolve_relative_url_test, 0},
+ {"init", module_init, 0},
+ {"remote-branch", resolve_remote_submodule_branch, 0},
};
int cmd_submodule__helper(int argc, const char **argv, const char *prefix)
@@ -1100,9 +1103,15 @@ int cmd_submodule__helper(int argc, const char **argv, const char *prefix)
die(_("submodule--helper subcommand must be "
"called with a subcommand"));
- for (i = 0; i < ARRAY_SIZE(commands); i++)
- if (!strcmp(argv[1], commands[i].cmd))
+ for (i = 0; i < ARRAY_SIZE(commands); i++) {
+ if (!strcmp(argv[1], commands[i].cmd)) {
+ if (get_super_prefix() &&
+ !(commands[i].option & SUPPORT_SUPER_PREFIX))
+ die(_("%s doesn't support --super-prefix"),
+ commands[i].cmd);
return commands[i].fn(argc - 1, argv + 1, prefix);
+ }
+ }
die(_("'%s' is not a valid submodule--helper "
"subcommand"), argv[1]);
diff --git a/git.c b/git.c
index efa1059fe0..98dcf6c518 100644
--- a/git.c
+++ b/git.c
@@ -493,7 +493,7 @@ static struct cmd_struct commands[] = {
{ "stage", cmd_add, RUN_SETUP | NEED_WORK_TREE },
{ "status", cmd_status, RUN_SETUP | NEED_WORK_TREE },
{ "stripspace", cmd_stripspace },
- { "submodule--helper", cmd_submodule__helper, RUN_SETUP },
+ { "submodule--helper", cmd_submodule__helper, RUN_SETUP | SUPPORT_SUPER_PREFIX},
{ "symbolic-ref", cmd_symbolic_ref, RUN_SETUP },
{ "tag", cmd_tag, RUN_SETUP },
{ "unpack-file", cmd_unpack_file, RUN_SETUP },
--
2.11.0.rc2.49.ge1f3b0c.dirty
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCHv8 3/6] test-lib-functions.sh: teach test_commit -C <dir>
2016-12-12 19:04 [PATCHv7 0/6] submodule absorbgitdirs Stefan Beller
2016-12-12 19:04 ` [PATCHv8 1/6] submodule: use absolute path for computing relative path connecting Stefan Beller
2016-12-12 19:04 ` [PATCHv8 2/6] submodule helper: support super prefix Stefan Beller
@ 2016-12-12 19:04 ` Stefan Beller
2016-12-12 19:04 ` [PATCHv8 4/6] worktree: check if a submodule uses worktrees Stefan Beller
` (3 subsequent siblings)
6 siblings, 0 replies; 12+ messages in thread
From: Stefan Beller @ 2016-12-12 19:04 UTC (permalink / raw)
To: gitster; +Cc: git, bmwill, pclouds, Stefan Beller
Specifically when setting up submodule tests, it comes in handy if
we can create commits in repositories that are not at the root of
the tested trash dir. Add "-C <dir>" similar to gits -C parameter
that will perform the operation in the given directory.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
t/test-lib-functions.sh | 20 +++++++++++++++-----
1 file changed, 15 insertions(+), 5 deletions(-)
diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index fdaeb3a96b..579e812506 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -157,16 +157,21 @@ debug () {
GIT_TEST_GDB=1 "$@"
}
-# Call test_commit with the arguments "<message> [<file> [<contents> [<tag>]]]"
+# Call test_commit with the arguments
+# [-C <directory>] <message> [<file> [<contents> [<tag>]]]"
#
# This will commit a file with the given contents and the given commit
# message, and tag the resulting commit with the given tag name.
#
# <file>, <contents>, and <tag> all default to <message>.
+#
+# If the first argument is "-C", the second argument is used as a path for
+# the git invocations.
test_commit () {
notick= &&
signoff= &&
+ indir= &&
while test $# != 0
do
case "$1" in
@@ -176,21 +181,26 @@ test_commit () {
--signoff)
signoff="$1"
;;
+ -C)
+ indir="$2"
+ shift
+ ;;
*)
break
;;
esac
shift
done &&
+ indir=${indir:+"$indir"/} &&
file=${2:-"$1.t"} &&
- echo "${3-$1}" > "$file" &&
- git add "$file" &&
+ echo "${3-$1}" > "$indir$file" &&
+ git ${indir:+ -C "$indir"} add "$file" &&
if test -z "$notick"
then
test_tick
fi &&
- git commit $signoff -m "$1" &&
- git tag "${4:-$1}"
+ git ${indir:+ -C "$indir"} commit $signoff -m "$1" &&
+ git ${indir:+ -C "$indir"} tag "${4:-$1}"
}
# Call test_merge with the arguments "<message> <commit>", where <commit>
--
2.11.0.rc2.49.ge1f3b0c.dirty
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCHv8 4/6] worktree: check if a submodule uses worktrees
2016-12-12 19:04 [PATCHv7 0/6] submodule absorbgitdirs Stefan Beller
` (2 preceding siblings ...)
2016-12-12 19:04 ` [PATCHv8 3/6] test-lib-functions.sh: teach test_commit -C <dir> Stefan Beller
@ 2016-12-12 19:04 ` Stefan Beller
2016-12-12 19:04 ` [PATCHv8 5/6] move connect_work_tree_and_git_dir to dir.h Stefan Beller
` (2 subsequent siblings)
6 siblings, 0 replies; 12+ messages in thread
From: Stefan Beller @ 2016-12-12 19:04 UTC (permalink / raw)
To: gitster; +Cc: git, bmwill, pclouds, Stefan Beller
In a later patch we want to move around the the git directory of
a submodule. Both submodules as well as worktrees are involved in
placing git directories at unusual places, so their functionality
may collide. To react appropriately to situations where worktrees
in submodules are in use, offer a new function to query the
a submodule if it uses the worktree feature.
An earlier approach:
"Implement submodule_get_worktrees and just count them", however:
This can be done cheaply (both in new code to write as well as run time)
by obtaining the list of worktrees based off that submodules git
directory. However as we have loaded the variables for the current
repository, the values in the submodule worktree
can be wrong, e.g.
* core.ignorecase may differ between these two repositories
* the ref resolution is broken (refs/heads/branch in the submodule
resolves to the sha1 value of the `branch` in the current repository
that may not exist or have another sha1)
The implementation here is just checking for any files in
$GIT_COMMON_DIR/worktrees for the submodule, which ought to be sufficient
if the submodule is using the current repository format, which we also
check.
Signed-off-by: Stefan Beller <sbeller@google.com>
---
worktree.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++
worktree.h | 5 +++++
2 files changed, 55 insertions(+)
diff --git a/worktree.c b/worktree.c
index eb6121263b..d4606aa8cd 100644
--- a/worktree.c
+++ b/worktree.c
@@ -380,3 +380,53 @@ const struct worktree *find_shared_symref(const char *symref,
return existing;
}
+
+int submodule_uses_worktrees(const char *path)
+{
+ char *submodule_gitdir;
+ struct strbuf sb = STRBUF_INIT;
+ DIR *dir;
+ struct dirent *d;
+ int ret;
+ struct repository_format format;
+
+ submodule_gitdir = git_pathdup_submodule(path, "%s", "");
+ if (!submodule_gitdir)
+ return 0;
+
+ /* The env would be set for the superproject. */
+ get_common_dir_noenv(&sb, submodule_gitdir);
+
+ /*
+ * The check below is only known to be good for repository format
+ * version 0 at the time of writing this code.
+ */
+ strbuf_addstr(&sb, "/config");
+ read_repository_format(&format, sb.buf);
+ if (format.version != 0) {
+ strbuf_release(&sb);
+ return 1;
+ }
+
+ /* Replace config by worktrees. */
+ strbuf_setlen(&sb, sb.len - strlen("config"));
+ strbuf_addstr(&sb, "worktrees");
+
+ /* See if there is any file inside the worktrees directory. */
+ dir = opendir(sb.buf);
+ strbuf_release(&sb);
+ free(submodule_gitdir);
+
+ if (!dir)
+ return 0;
+
+ while ((d = readdir(dir)) != NULL) {
+ if (is_dot_or_dotdot(d->d_name))
+ continue;
+
+ ret = 1;
+ break;
+ }
+ closedir(dir);
+ return ret;
+}
diff --git a/worktree.h b/worktree.h
index d59ce1fee8..6bfb985203 100644
--- a/worktree.h
+++ b/worktree.h
@@ -27,6 +27,11 @@ struct worktree {
*/
extern struct worktree **get_worktrees(unsigned flags);
+/*
+ * Returns 1 if linked worktrees exist, 0 otherwise.
+ */
+extern int submodule_uses_worktrees(const char *path);
+
/*
* Return git dir of the worktree. Note that the path may be relative.
* If wt is NULL, git dir of current worktree is returned.
--
2.11.0.rc2.49.ge1f3b0c.dirty
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCHv8 5/6] move connect_work_tree_and_git_dir to dir.h
2016-12-12 19:04 [PATCHv7 0/6] submodule absorbgitdirs Stefan Beller
` (3 preceding siblings ...)
2016-12-12 19:04 ` [PATCHv8 4/6] worktree: check if a submodule uses worktrees Stefan Beller
@ 2016-12-12 19:04 ` Stefan Beller
2016-12-12 19:04 ` [PATCHv8 6/6] submodule: add absorb-git-dir function Stefan Beller
2016-12-12 20:35 ` [PATCHv7 0/6] submodule absorbgitdirs Brandon Williams
6 siblings, 0 replies; 12+ messages in thread
From: Stefan Beller @ 2016-12-12 19:04 UTC (permalink / raw)
To: gitster; +Cc: git, bmwill, pclouds, Stefan Beller
That function was primarily used by submodule code, but the function
itself is not inherently about submodules. In the next patch we'll
introduce relocate_git_dir, which can be used by worktrees as well,
so find a neutral middle ground in dir.h.
Signed-off-by: Stefan Beller <sbeller@google.com>
---
dir.c | 25 +++++++++++++++++++++++++
dir.h | 1 +
submodule.c | 25 -------------------------
submodule.h | 1 -
4 files changed, 26 insertions(+), 26 deletions(-)
diff --git a/dir.c b/dir.c
index bfa8c8a9a5..e0efd3c2c3 100644
--- a/dir.c
+++ b/dir.c
@@ -2748,3 +2748,28 @@ void untracked_cache_add_to_index(struct index_state *istate,
{
untracked_cache_invalidate_path(istate, path);
}
+
+/* Update gitfile and core.worktree setting to connect work tree and git dir */
+void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)
+{
+ struct strbuf file_name = STRBUF_INIT;
+ struct strbuf rel_path = STRBUF_INIT;
+ char *git_dir = xstrdup(real_path(git_dir_));
+ char *work_tree = xstrdup(real_path(work_tree_));
+
+ /* Update gitfile */
+ strbuf_addf(&file_name, "%s/.git", work_tree);
+ write_file(file_name.buf, "gitdir: %s",
+ relative_path(git_dir, work_tree, &rel_path));
+
+ /* Update core.worktree setting */
+ strbuf_reset(&file_name);
+ strbuf_addf(&file_name, "%s/config", git_dir);
+ git_config_set_in_file(file_name.buf, "core.worktree",
+ relative_path(work_tree, git_dir, &rel_path));
+
+ strbuf_release(&file_name);
+ strbuf_release(&rel_path);
+ free(work_tree);
+ free(git_dir);
+}
diff --git a/dir.h b/dir.h
index 97c83bb383..051674a431 100644
--- a/dir.h
+++ b/dir.h
@@ -335,4 +335,5 @@ struct untracked_cache *read_untracked_extension(const void *data, unsigned long
void write_untracked_extension(struct strbuf *out, struct untracked_cache *untracked);
void add_untracked_cache(struct index_state *istate);
void remove_untracked_cache(struct index_state *istate);
+extern void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);
#endif
diff --git a/submodule.c b/submodule.c
index d4f7afe2f1..0bb50b4b62 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1222,31 +1222,6 @@ int merge_submodule(unsigned char result[20], const char *path,
return 0;
}
-/* Update gitfile and core.worktree setting to connect work tree and git dir */
-void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)
-{
- struct strbuf file_name = STRBUF_INIT;
- struct strbuf rel_path = STRBUF_INIT;
- char *git_dir = xstrdup(real_path(git_dir_));
- char *work_tree = xstrdup(real_path(work_tree_));
-
- /* Update gitfile */
- strbuf_addf(&file_name, "%s/.git", work_tree);
- write_file(file_name.buf, "gitdir: %s",
- relative_path(git_dir, work_tree, &rel_path));
-
- /* Update core.worktree setting */
- strbuf_reset(&file_name);
- strbuf_addf(&file_name, "%s/config", git_dir);
- git_config_set_in_file(file_name.buf, "core.worktree",
- relative_path(work_tree, git_dir, &rel_path));
-
- strbuf_release(&file_name);
- strbuf_release(&rel_path);
- free(work_tree);
- free(git_dir);
-}
-
int parallel_submodules(void)
{
return parallel_jobs;
diff --git a/submodule.h b/submodule.h
index d9e197a948..4e3bf469b4 100644
--- a/submodule.h
+++ b/submodule.h
@@ -65,7 +65,6 @@ int merge_submodule(unsigned char result[20], const char *path, const unsigned c
int find_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_name,
struct string_list *needs_pushing);
int push_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_name);
-void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);
int parallel_submodules(void);
/*
--
2.11.0.rc2.49.ge1f3b0c.dirty
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCHv8 6/6] submodule: add absorb-git-dir function
2016-12-12 19:04 [PATCHv7 0/6] submodule absorbgitdirs Stefan Beller
` (4 preceding siblings ...)
2016-12-12 19:04 ` [PATCHv8 5/6] move connect_work_tree_and_git_dir to dir.h Stefan Beller
@ 2016-12-12 19:04 ` Stefan Beller
2016-12-19 5:35 ` Duy Nguyen
2016-12-12 20:35 ` [PATCHv7 0/6] submodule absorbgitdirs Brandon Williams
6 siblings, 1 reply; 12+ messages in thread
From: Stefan Beller @ 2016-12-12 19:04 UTC (permalink / raw)
To: gitster; +Cc: git, bmwill, pclouds, Stefan Beller
When a submodule has its git dir inside the working dir, the submodule
support for checkout that we plan to add in a later patch will fail.
Add functionality to migrate the git directory to be absorbed
into the superprojects git directory.
The newly added code in this patch is structured such that other areas of
Git can also make use of it. The code in the submodule--helper is a mere
wrapper and option parser for the function
`absorb_git_dir_into_superproject`, that takes care of embedding the
submodules git directory into the superprojects git dir. That function
makes use of the more abstract function for this use case
`relocate_gitdir`, which can be used by e.g. the worktree code eventually
to move around a git directory.
Signed-off-by: Stefan Beller <sbeller@google.com>
---
Documentation/git-submodule.txt | 15 ++++++
builtin/submodule--helper.c | 38 ++++++++++++++
dir.c | 12 +++++
dir.h | 3 ++
git-submodule.sh | 7 ++-
submodule.c | 103 +++++++++++++++++++++++++++++++++++++
submodule.h | 4 ++
t/t7412-submodule-absorbgitdirs.sh | 101 ++++++++++++++++++++++++++++++++++++
8 files changed, 282 insertions(+), 1 deletion(-)
create mode 100755 t/t7412-submodule-absorbgitdirs.sh
diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index d841573475..918bd1d1bd 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -22,6 +22,7 @@ SYNOPSIS
[commit] [--] [<path>...]
'git submodule' [--quiet] foreach [--recursive] <command>
'git submodule' [--quiet] sync [--recursive] [--] [<path>...]
+'git submodule' [--quiet] absorbgitdirs [--] [<path>...]
DESCRIPTION
@@ -245,6 +246,20 @@ sync::
If `--recursive` is specified, this command will recurse into the
registered submodules, and sync any nested submodules within.
+absorbgitdirs::
+ If a git directory of a submodule is inside the submodule,
+ move the git directory of the submodule into its superprojects
+ `$GIT_DIR/modules` path and then connect the git directory and
+ its working directory by setting the `core.worktree` and adding
+ a .git file pointing to the git directory embedded in the
+ superprojects git directory.
++
+A repository that was cloned independently and later added as a submodule or
+old setups have the submodules git directory inside the submodule instead of
+embedded into the superprojects git directory.
++
+This command is recursive by default.
+
OPTIONS
-------
-q::
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 5643848667..242d9911a6 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1076,6 +1076,43 @@ static int resolve_remote_submodule_branch(int argc, const char **argv,
return 0;
}
+static int absorb_git_dirs(int argc, const char **argv, const char *prefix)
+{
+ int i;
+ struct pathspec pathspec;
+ struct module_list list = MODULE_LIST_INIT;
+ unsigned flags = ABSORB_GITDIR_RECURSE_SUBMODULES;
+
+ struct option embed_gitdir_options[] = {
+ OPT_STRING(0, "prefix", &prefix,
+ N_("path"),
+ N_("path into the working tree")),
+ OPT_BIT(0, "--recursive", &flags, N_("recurse into submodules"),
+ ABSORB_GITDIR_RECURSE_SUBMODULES),
+ OPT_END()
+ };
+
+ const char *const git_submodule_helper_usage[] = {
+ N_("git submodule--helper embed-git-dir [<path>...]"),
+ NULL
+ };
+
+ argc = parse_options(argc, argv, prefix, embed_gitdir_options,
+ git_submodule_helper_usage, 0);
+
+ gitmodules_config();
+ git_config(submodule_config, NULL);
+
+ if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
+ return 1;
+
+ for (i = 0; i < list.nr; i++)
+ absorb_git_dir_into_superproject(prefix,
+ list.entries[i]->name, flags);
+
+ return 0;
+}
+
#define SUPPORT_SUPER_PREFIX (1<<0)
struct cmd_struct {
@@ -1094,6 +1131,7 @@ static struct cmd_struct commands[] = {
{"resolve-relative-url-test", resolve_relative_url_test, 0},
{"init", module_init, 0},
{"remote-branch", resolve_remote_submodule_branch, 0},
+ {"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
};
int cmd_submodule__helper(int argc, const char **argv, const char *prefix)
diff --git a/dir.c b/dir.c
index e0efd3c2c3..d872cc1570 100644
--- a/dir.c
+++ b/dir.c
@@ -2773,3 +2773,15 @@ void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)
free(work_tree);
free(git_dir);
}
+
+/*
+ * Migrate the git directory of the given path from old_git_dir to new_git_dir.
+ */
+void relocate_gitdir(const char *path, const char *old_git_dir, const char *new_git_dir)
+{
+ if (rename(old_git_dir, new_git_dir) < 0)
+ die_errno(_("could not migrate git directory from '%s' to '%s'"),
+ old_git_dir, new_git_dir);
+
+ connect_work_tree_and_git_dir(path, new_git_dir);
+}
diff --git a/dir.h b/dir.h
index 051674a431..bf23a470af 100644
--- a/dir.h
+++ b/dir.h
@@ -336,4 +336,7 @@ void write_untracked_extension(struct strbuf *out, struct untracked_cache *untra
void add_untracked_cache(struct index_state *istate);
void remove_untracked_cache(struct index_state *istate);
extern void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);
+extern void relocate_gitdir(const char *path,
+ const char *old_git_dir,
+ const char *new_git_dir);
#endif
diff --git a/git-submodule.sh b/git-submodule.sh
index a024a135d6..9285b5c43d 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -1131,6 +1131,11 @@ cmd_sync()
done
}
+cmd_absorbgitdirs()
+{
+ git submodule--helper absorb-git-dirs --prefix "$wt_prefix" "$@"
+}
+
# This loop parses the command line arguments to find the
# subcommand name to dispatch. Parsing of the subcommand specific
# options are primarily done by the subcommand implementations.
@@ -1140,7 +1145,7 @@ cmd_sync()
while test $# != 0 && test -z "$command"
do
case "$1" in
- add | foreach | init | deinit | update | status | summary | sync)
+ add | foreach | init | deinit | update | status | summary | sync | absorbgitdirs)
command=$1
;;
-q|--quiet)
diff --git a/submodule.c b/submodule.c
index 0bb50b4b62..45ccfb7ab4 100644
--- a/submodule.c
+++ b/submodule.c
@@ -14,6 +14,7 @@
#include "blob.h"
#include "thread-utils.h"
#include "quote.h"
+#include "worktree.h"
static int config_fetch_recurse_submodules = RECURSE_SUBMODULES_ON_DEMAND;
static int parallel_jobs = 1;
@@ -1237,3 +1238,105 @@ void prepare_submodule_repo_env(struct argv_array *out)
}
argv_array_push(out, "GIT_DIR=.git");
}
+
+/*
+ * Embeds a single submodules git directory into the superprojects git dir,
+ * non recursively.
+ */
+static void relocate_single_git_dir_into_superproject(const char *prefix,
+ const char *path)
+{
+ char *old_git_dir = NULL, *real_old_git_dir = NULL, *real_new_git_dir = NULL;
+ const char *new_git_dir;
+ const struct submodule *sub;
+
+ if (submodule_uses_worktrees(path))
+ die(_("relocate_gitdir for submodule '%s' with "
+ "more than one worktree not supported"), path);
+
+ old_git_dir = xstrfmt("%s/.git", path);
+ if (read_gitfile(old_git_dir))
+ /* If it is an actual gitfile, it doesn't need migration. */
+ return;
+
+ real_old_git_dir = xstrdup(real_path(old_git_dir));
+
+ sub = submodule_from_path(null_sha1, path);
+ if (!sub)
+ die(_("could not lookup name for submodule '%s'"), path);
+
+ new_git_dir = git_path("modules/%s", sub->name);
+ if (safe_create_leading_directories_const(new_git_dir) < 0)
+ die(_("could not create directory '%s'"), new_git_dir);
+ real_new_git_dir = xstrdup(real_path(new_git_dir));
+
+ if (!prefix)
+ prefix = get_super_prefix();
+
+ fprintf(stderr, _("Migrating git directory of '%s%s' from\n'%s' to\n'%s'\n"),
+ prefix ? prefix : "", path,
+ real_old_git_dir, real_new_git_dir);
+
+ relocate_gitdir(path, real_old_git_dir, real_new_git_dir);
+
+ free(old_git_dir);
+ free(real_old_git_dir);
+ free(real_new_git_dir);
+}
+
+/*
+ * Migrate the git directory of the submodule given by path from
+ * having its git directory within the working tree to the git dir nested
+ * in its superprojects git dir under modules/.
+ */
+void absorb_git_dir_into_superproject(const char *prefix,
+ const char *path,
+ unsigned flags)
+{
+ const char *sub_git_dir, *v;
+ char *real_sub_git_dir = NULL, *real_common_git_dir = NULL;
+ struct strbuf gitdir = STRBUF_INIT;
+
+ strbuf_addf(&gitdir, "%s/.git", path);
+ sub_git_dir = resolve_gitdir(gitdir.buf);
+
+ /* Not populated? */
+ if (!sub_git_dir)
+ goto out;
+
+ /* Is it already absorbed into the superprojects git dir? */
+ real_sub_git_dir = xstrdup(real_path(sub_git_dir));
+ real_common_git_dir = xstrdup(real_path(get_git_common_dir()));
+ if (!skip_prefix(real_sub_git_dir, real_common_git_dir, &v))
+ relocate_single_git_dir_into_superproject(prefix, path);
+
+ if (flags & ABSORB_GITDIR_RECURSE_SUBMODULES) {
+ struct child_process cp = CHILD_PROCESS_INIT;
+ struct strbuf sb = STRBUF_INIT;
+
+ if (flags & ~ABSORB_GITDIR_RECURSE_SUBMODULES)
+ die("BUG: we don't know how to pass the flags down?");
+
+ if (get_super_prefix())
+ strbuf_addstr(&sb, get_super_prefix());
+ strbuf_addstr(&sb, path);
+ strbuf_addch(&sb, '/');
+
+ cp.dir = path;
+ cp.git_cmd = 1;
+ cp.no_stdin = 1;
+ argv_array_pushl(&cp.args, "--super-prefix", sb.buf,
+ "submodule--helper",
+ "absorb-git-dirs", NULL);
+ prepare_submodule_repo_env(&cp.env_array);
+ if (run_command(&cp))
+ die(_("could not recurse into submodule '%s'"), path);
+
+ strbuf_release(&sb);
+ }
+
+out:
+ strbuf_release(&gitdir);
+ free(real_sub_git_dir);
+ free(real_common_git_dir);
+}
diff --git a/submodule.h b/submodule.h
index 4e3bf469b4..6229054b99 100644
--- a/submodule.h
+++ b/submodule.h
@@ -74,4 +74,8 @@ int parallel_submodules(void);
*/
void prepare_submodule_repo_env(struct argv_array *out);
+#define ABSORB_GITDIR_RECURSE_SUBMODULES (1<<0)
+extern void absorb_git_dir_into_superproject(const char *prefix,
+ const char *path,
+ unsigned flags);
#endif
diff --git a/t/t7412-submodule-absorbgitdirs.sh b/t/t7412-submodule-absorbgitdirs.sh
new file mode 100755
index 0000000000..1c47780e2b
--- /dev/null
+++ b/t/t7412-submodule-absorbgitdirs.sh
@@ -0,0 +1,101 @@
+#!/bin/sh
+
+test_description='Test submodule absorbgitdirs
+
+This test verifies that `git submodue absorbgitdirs` moves a submodules git
+directory into the superproject.
+'
+
+. ./test-lib.sh
+
+test_expect_success 'setup a real submodule' '
+ git init sub1 &&
+ test_commit -C sub1 first &&
+ git submodule add ./sub1 &&
+ test_tick &&
+ git commit -m superproject
+'
+
+test_expect_success 'absorb the git dir' '
+ >expect.1 &&
+ >expect.2 &&
+ >actual.1 &&
+ >actual.2 &&
+ git status >expect.1 &&
+ git -C sub1 rev-parse HEAD >expect.2 &&
+ git submodule absorbgitdirs &&
+ git fsck &&
+ test -f sub1/.git &&
+ test -d .git/modules/sub1 &&
+ git status >actual.1 &&
+ git -C sub1 rev-parse HEAD >actual.2 &&
+ test_cmp expect.1 actual.1 &&
+ test_cmp expect.2 actual.2
+'
+
+test_expect_success 'absorbing does not fail for deinitalized submodules' '
+ test_when_finished "git submodule update --init" &&
+ git submodule deinit --all &&
+ git submodule absorbgitdirs &&
+ test -d .git/modules/sub1 &&
+ test -d sub1 &&
+ ! test -e sub1/.git
+'
+
+test_expect_success 'setup nested submodule' '
+ git init sub1/nested &&
+ test_commit -C sub1/nested first_nested &&
+ git -C sub1 submodule add ./nested &&
+ test_tick &&
+ git -C sub1 commit -m "add nested" &&
+ git add sub1 &&
+ git commit -m "sub1 to include nested submodule"
+'
+
+test_expect_success 'absorb the git dir in a nested submodule' '
+ git status >expect.1 &&
+ git -C sub1/nested rev-parse HEAD >expect.2 &&
+ git submodule absorbgitdirs &&
+ test -f sub1/nested/.git &&
+ test -d .git/modules/sub1/modules/nested &&
+ git status >actual.1 &&
+ git -C sub1/nested rev-parse HEAD >actual.2 &&
+ test_cmp expect.1 actual.1 &&
+ test_cmp expect.2 actual.2
+'
+
+test_expect_success 'setup a gitlink with missing .gitmodules entry' '
+ git init sub2 &&
+ test_commit -C sub2 first &&
+ git add sub2 &&
+ git commit -m superproject
+'
+
+test_expect_success 'absorbing the git dir fails for incomplete submodules' '
+ git status >expect.1 &&
+ git -C sub2 rev-parse HEAD >expect.2 &&
+ test_must_fail git submodule absorbgitdirs &&
+ git -C sub2 fsck &&
+ test -d sub2/.git &&
+ git status >actual &&
+ git -C sub2 rev-parse HEAD >actual.2 &&
+ test_cmp expect.1 actual.1 &&
+ test_cmp expect.2 actual.2
+'
+
+test_expect_success 'setup a submodule with multiple worktrees' '
+ # first create another unembedded git dir in a new submodule
+ git init sub3 &&
+ test_commit -C sub3 first &&
+ git submodule add ./sub3 &&
+ test_tick &&
+ git commit -m "add another submodule" &&
+ git -C sub3 worktree add ../sub3_second_work_tree
+'
+
+test_expect_success 'absorbing fails for a submodule with multiple worktrees' '
+ test_must_fail git submodule absorbgitdirs sub3 2>error &&
+ test_i18ngrep "not supported" error
+'
+
+test_done
--
2.11.0.rc2.49.ge1f3b0c.dirty
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: [PATCHv8 6/6] submodule: add absorb-git-dir function
2016-12-12 19:04 ` [PATCHv8 6/6] submodule: add absorb-git-dir function Stefan Beller
@ 2016-12-19 5:35 ` Duy Nguyen
2016-12-19 18:15 ` Stefan Beller
0 siblings, 1 reply; 12+ messages in thread
From: Duy Nguyen @ 2016-12-19 5:35 UTC (permalink / raw)
To: Stefan Beller; +Cc: gitster, git, bmwill
On Mon, Dec 12, 2016 at 11:04:35AM -0800, Stefan Beller wrote:
> diff --git a/dir.c b/dir.c
> index e0efd3c2c3..d872cc1570 100644
> --- a/dir.c
> +++ b/dir.c
> @@ -2773,3 +2773,15 @@ void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)
> free(work_tree);
> free(git_dir);
> }
> +
> +/*
> + * Migrate the git directory of the given path from old_git_dir to new_git_dir.
> + */
> +void relocate_gitdir(const char *path, const char *old_git_dir, const char *new_git_dir)
> +{
> + if (rename(old_git_dir, new_git_dir) < 0)
> + die_errno(_("could not migrate git directory from '%s' to '%s'"),
> + old_git_dir, new_git_dir);
> +
> + connect_work_tree_and_git_dir(path, new_git_dir);
Should we worry about recovering (e.g. maybe move new_git_dir back to
old_git_dir) if this connect_work_tree_and_git_dir() fails?
Both write_file() and git_config_set_.. in this function may die(). In
such a case the repo is in broken state and the user needs pretty good
submodule understanding to recover from it, I think.
Recovering is not easy (nor entirely safe) either, though I suppose if
we keep original copies for modified files, then we could restore them
after moving the directory back and pray the UNIX gods that all
operations succeed.
--
Duy
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCHv8 6/6] submodule: add absorb-git-dir function
2016-12-19 5:35 ` Duy Nguyen
@ 2016-12-19 18:15 ` Stefan Beller
2016-12-20 1:39 ` Duy Nguyen
0 siblings, 1 reply; 12+ messages in thread
From: Stefan Beller @ 2016-12-19 18:15 UTC (permalink / raw)
To: Duy Nguyen; +Cc: Junio C Hamano, git@vger.kernel.org, Brandon Williams
On Sun, Dec 18, 2016 at 9:35 PM, Duy Nguyen <pclouds@gmail.com> wrote:
> On Mon, Dec 12, 2016 at 11:04:35AM -0800, Stefan Beller wrote:
>> diff --git a/dir.c b/dir.c
>> index e0efd3c2c3..d872cc1570 100644
>> --- a/dir.c
>> +++ b/dir.c
>> @@ -2773,3 +2773,15 @@ void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)
>> free(work_tree);
>> free(git_dir);
>> }
>> +
>> +/*
>> + * Migrate the git directory of the given path from old_git_dir to new_git_dir.
>> + */
>> +void relocate_gitdir(const char *path, const char *old_git_dir, const char *new_git_dir)
>> +{
>> + if (rename(old_git_dir, new_git_dir) < 0)
>> + die_errno(_("could not migrate git directory from '%s' to '%s'"),
>> + old_git_dir, new_git_dir);
>> +
>> + connect_work_tree_and_git_dir(path, new_git_dir);
>
> Should we worry about recovering (e.g. maybe move new_git_dir back to
> old_git_dir) if this connect_work_tree_and_git_dir() fails?
What if the move back fails?
>
> Both write_file() and git_config_set_.. in this function may die(). In
> such a case the repo is in broken state and the user needs pretty good
> submodule understanding to recover from it, I think.
>
> Recovering is not easy (nor entirely safe) either, though I suppose if
> we keep original copies for modified files, then we could restore them
> after moving the directory back and pray the UNIX gods that all
> operations succeed.
There are different levels of brokenness available.
I just tried what happens if core.worktree is unset in a submodule
and that seems to not impact git operations (I only tested git-status
both in the superproject as well as in the submodule).
So I wonder why we set core.worktree at all here as it doesn't
seem to be needed.
Which means that the move of the git directory as well as the .git file
update to point at that moved directory are the important things
to get right.
So maybe:
1) rename the git dir or die
2) write the new gitlink
If that fails remove the .git file (if it exists partially or empty)
and undo 1) by calling rename again with swapped arguments
and then die
3) set core.worktree
If that fails, just warn the user
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCHv8 6/6] submodule: add absorb-git-dir function
2016-12-19 18:15 ` Stefan Beller
@ 2016-12-20 1:39 ` Duy Nguyen
0 siblings, 0 replies; 12+ messages in thread
From: Duy Nguyen @ 2016-12-20 1:39 UTC (permalink / raw)
To: Stefan Beller; +Cc: Junio C Hamano, git@vger.kernel.org, Brandon Williams
On Tue, Dec 20, 2016 at 1:15 AM, Stefan Beller <sbeller@google.com> wrote:
> On Sun, Dec 18, 2016 at 9:35 PM, Duy Nguyen <pclouds@gmail.com> wrote:
>> On Mon, Dec 12, 2016 at 11:04:35AM -0800, Stefan Beller wrote:
>>> diff --git a/dir.c b/dir.c
>>> index e0efd3c2c3..d872cc1570 100644
>>> --- a/dir.c
>>> +++ b/dir.c
>>> @@ -2773,3 +2773,15 @@ void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)
>>> free(work_tree);
>>> free(git_dir);
>>> }
>>> +
>>> +/*
>>> + * Migrate the git directory of the given path from old_git_dir to new_git_dir.
>>> + */
>>> +void relocate_gitdir(const char *path, const char *old_git_dir, const char *new_git_dir)
>>> +{
>>> + if (rename(old_git_dir, new_git_dir) < 0)
>>> + die_errno(_("could not migrate git directory from '%s' to '%s'"),
>>> + old_git_dir, new_git_dir);
>>> +
>>> + connect_work_tree_and_git_dir(path, new_git_dir);
>>
>> Should we worry about recovering (e.g. maybe move new_git_dir back to
>> old_git_dir) if this connect_work_tree_and_git_dir() fails?
>
> What if the move back fails?
That's when you pray the UNIX gods that recovery steps don't fail :-)
This is why I don't _suggest_ to do things but just wonder about it.
In theory though, if we keep recovery to dead simple operations (e.g.
a series of rename() and nothing else) then it's less likely to fail.
I'll look at the new patches when I get home.
--
Duy
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCHv7 0/6] submodule absorbgitdirs
2016-12-12 19:04 [PATCHv7 0/6] submodule absorbgitdirs Stefan Beller
` (5 preceding siblings ...)
2016-12-12 19:04 ` [PATCHv8 6/6] submodule: add absorb-git-dir function Stefan Beller
@ 2016-12-12 20:35 ` Brandon Williams
2016-12-12 20:37 ` Stefan Beller
6 siblings, 1 reply; 12+ messages in thread
From: Brandon Williams @ 2016-12-12 20:35 UTC (permalink / raw)
To: Stefan Beller; +Cc: gitster, git, pclouds
On 12/12, Stefan Beller wrote:
> v8:
> * Change the worktree implementation: do not have an internal
> submodule_get_worktrees, and count the number of worktrees, instead
> directly look if there is a worktrees dir and check if there are any files
> in there.
> * reworded one test title slightly.
> * interdiff to v7 (that is queued as origin/sb/submodule-embed-gitdir) below.
>
not important but your cover letter's subject is v7 instead of v8 :)
--
Brandon Williams
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCHv7 0/6] submodule absorbgitdirs
2016-12-12 20:35 ` [PATCHv7 0/6] submodule absorbgitdirs Brandon Williams
@ 2016-12-12 20:37 ` Stefan Beller
0 siblings, 0 replies; 12+ messages in thread
From: Stefan Beller @ 2016-12-12 20:37 UTC (permalink / raw)
To: Brandon Williams; +Cc: Junio C Hamano, git@vger.kernel.org, Duy Nguyen
On Mon, Dec 12, 2016 at 12:35 PM, Brandon Williams <bmwill@google.com> wrote:
> On 12/12, Stefan Beller wrote:
>> v8:
>> * Change the worktree implementation: do not have an internal
>> submodule_get_worktrees, and count the number of worktrees, instead
>> directly look if there is a worktrees dir and check if there are any files
>> in there.
>> * reworded one test title slightly.
>> * interdiff to v7 (that is queued as origin/sb/submodule-embed-gitdir) below.
>>
>
> not important but your cover letter's subject is v7 instead of v8 :)
>
I realized that, after sending out.
When working on just one series on the mailing list,
I tend to reuse the coverletter and just adapt it, so I
would presume the diff stat is also broken.
^ permalink raw reply [flat|nested] 12+ messages in thread