git init --separate-git-dir does not update symbolic .git links for submodules

git init --separate-git-dir does not update symbolic .git links for submodules

[Sven Strickroth]

Hi,

clone or have a repository with submodules.

Now issue "git init --separate-git-dir ../test" which moves the content
of the .git direcotry to the ../test directory and replaces the .git
directory with a .git file "link".

However, for all submodules the .git-file is not updated and still
points to the .git directory of the parent repository which now is a file.

This issue is present for me in Git 2.12.0.

-- 
Best regards,
 Sven Strickroth
 PGP key id F5A9D4C4 @ any key-server

Re: git init --separate-git-dir does not update symbolic .git links for submodules

[Valery Tolstov]

Looking for microproject ideas for GSoC.
Would this issue be suitable as the microproject?

Thanks,
  Valery Tolstov

Re: git init --separate-git-dir does not update symbolic .git links for submodules

[Stefan Beller]

On Sat, Mar 4, 2017 at 6:15 AM, Valery Tolstov <me@vtolstov.org> wrote:
> Looking for microproject ideas for GSoC.
> Would this issue be suitable as the microproject?

It would be a good project, but not as 'micro' I would assume. ;)
Why it is not a micro project:

To fix this issue we'd want to fix the .git link files recursively, i.e.
nested submodules would also need fixing. And to recurse into
submodules we normally spawn a child process.

So maybe the plan is as follow:
1) Add the functionality to builtin/submodule--helper.c
    it fixes just one submodule, given by name.
2) add an externally visible command in the same file
  See e.g. module_name (the function and the table
  at the very end of the file)
  (1) needs to call this command (see clone_submodule() in
    that file how to call further commands)

2) "git init --separate-git-dir ../test" would call the function
  in (1), which then recursively uses (2) to call (1) in nested
  submodules.

After thinking about it, it may be good for a micro project,
as it is smaller than originally assumed.
Specifically if you plan to look into submodules later on.

Thanks,
Stefan

Re: git init --separate-git-dir does not update symbolic .git links for submodules

[Valery Tolstov]

I think we can reuse code from module_clone that writes .git link.
Possibly this code fragment needs to be factored out from module_clone

Also, to fix all the links, we need to obtain the list of submodules
and then iterate over them. module_list command iterates
and prints the result to stdout. Maybe we can reuse this output.
Or create separate function that returns the list of submodules.

Can we call these functions from submodule--helper inside init command,
or run them thorugh internal command interface?

In my opinion, make submodule--helper fully responsible for link fixes
would be a good solution. Then we create two additional function, one
that fixes all submodules on the current level, and another that
fixes individual submodule.
Although it looks good, I'm not quite sure that it's really good.

So, maybe we can do link fixes like this:

    1. Start fixing from init command using submodule--helper, with
       subcommand that fixes all submodules on current level
    2. Each submodule processed with another subcommand/function in
       submodule--helper individually
    3. Repeat for current submodule recursively

Glad to see your advices.

Regards,
  Valery Tolstov

Re: git init --separate-git-dir does not update symbolic .git links for submodules

[Valery Tolstov]

Just noticed that there already is function that gives module list
module_list_compute. But purpose of it's arguments is not quite clear
for me at this moment.

Re: git init --separate-git-dir does not update symbolic .git links for submodules

[Stefan Beller]

On Tue, Mar 7, 2017 at 10:59 AM, Valery Tolstov <me@vtolstov.org> wrote:
> I think we can reuse code from module_clone that writes .git link.
> Possibly this code fragment needs to be factored out from module_clone

That fragment already exists, see dir.h:
connect_work_tree_and_git_dir(work_tree, git_dir);
Maybe another good microproject is to use that in module_clone.

>
> Also, to fix all the links, we need to obtain the list of submodules
> and then iterate over them.

Right, but a submodule may have a nested submodule.
So we need to fix each submodule from that list recursively,
i.e. not just the submodule itself, but any potential nested
submodule in that submodule, too.
(the listing doesn't list these nested submodules)

So we would call

    fix_gitlink(sub)
    {
        fix_locally(sub);

        // have a child process that calls
        // this function on any submodule inside sub.
    }

> module_list command iterates
> and prints the result to stdout. Maybe we can reuse this output.
> Or create separate function that returns the list of submodules.

yeah you can make use of module_list_compute to just produce
the list internally.

>
> Can we call these functions from submodule--helper inside init command,
> or run them thorugh internal command interface?

Both sounds fine, though calling internally is preferable for
performance reasons.

>
> In my opinion, make submodule--helper fully responsible for link fixes
> would be a good solution. Then we create two additional function, one
> that fixes all submodules on the current level, and another that
> fixes individual submodule.
> Although it looks good, I'm not quite sure that it's really good.

That sounds good to me.

So "git init --separate-git-dir" calls internally a new function
in the submodule--helper to

>
> So, maybe we can do link fixes like this:
>
>    1. Start fixing from init command using submodule--helper, with
>       subcommand that fixes all submodules on current level

for this step we do not change the repository we are in, so there
is no need to call a new process, but we rather want to
call it internally.

>    2. Each submodule processed with another subcommand/function in
>       submodule--helper individually

sounds good.

>    3. Repeat for current submodule recursively

and this recursive action needs to have its own process in the submodule,
e.g. "git submodule--helper --recursive fix-git-links"

>
> Glad to see your advices.
>
> Regards,
>  Valery Tolstov

Regards,
Stefan

Re: git init --separate-git-dir does not update symbolic .git links for submodules

[Stefan Beller]

On Tue, Mar 7, 2017 at 11:52 AM, Valery Tolstov <me@vtolstov.org> wrote:
> Just noticed that there already is function that gives module list
> module_list_compute. But purpose of it's arguments is not quite clear
> for me at this moment.
>

static int module_list_compute(int argc, const char **argv,
                               const char *prefix,
                               struct pathspec *pathspec,
                               struct module_list *list)


argc, argv and prefix are just passed through from each caller
argc is the number of arguments on the command line,
argv is an array of said arguments on the command line,
prefix is the position inside the repository.

e.g. If in git.git in Documentation/, you run a git command
"git submodule--helper module_list *"
then first "git" and "submodule--helper" are removed,
and then cmd_submodule__helper is called (at the end of
the submodule helper, which then sees
    argc=2, argv=["module_list", "*"],
    prefix="Documentation/"
which then proceeds to into module_list().

That parses these arguments, but there is no
argument it knows about, so it does nothing.

Then it just passes these three (argc, argv, prefix)
to module_list_compute, which then makes up the list,
to be stored in the last parameter 'list'.

The 'pathspec' parameter seems weird.
Internally the arguments mentioned above are converted to a pathspec,
such that we can go through all files and call match_pathspec inside
of module_list_compute.

In all but one cases we do not care about the pathspec, but in one
case (in update_clone) we want to issue a warning if the pathspec
was empty (i.e. the user just said "git submodule--helper update_clone"
with no further path arguments)