Re: Re: [PATCH] submodule : Add --no-separate-git-dir option to add and update command.

[Heiko Voigt <hvoigt@hvoigt.net>]

On Tue, Mar 11, 2014 at 10:55:03AM +0100, Henri GEIST wrote:
> Le lundi 10 mars 2014 à 21:32 +0100, Heiko Voigt a écrit :
> > On Mon, Mar 10, 2014 at 10:08:06AM +0100, Henri GEIST wrote:
> > > Le samedi 08 mars 2014 à 00:00 +0100, Jens Lehmann a écrit :
> > > > Am 06.03.2014 23:20, schrieb Henri GEIST:
> > > > >> What is the use case you are trying to solve and why can that
> > > > >> not be handled by adding "subsubmodule" inside "submodule"?
> > > > > 
> > > > > The problem is access rights.
> > > > > 
> > > > > Imagine you have 2 people Pierre and Paul.
> > > > > Each with different access write on the server.
> > > > > Pierre has full access on every things.
> > > > > Paul has full access on superproject and subsubmodule but no read/write
> > > > > access to submodule only execution on the directory.
> > > > 
> > > > Ok, I think I'm slowly beginning to understand your setup.
> > > > 
> > > > > I want all user to get every things they are allowed to have with the
> > > > > command 'git submodule update --init --recursive'.
> > > > > Then as Paul can not clone submodule he can not get subsubmodule
> > > > > recursively through it.
> > > > 
> > > > Sure, that's how it should work. Paul could only work on a branch
> > > > where "submodule" is an empty directory containing "subsubmodule",
> > > > as he doesn't have the rights to clone "submodule".
> > > 
> > > I will not redundantly create a branch for each user on the server.
> > > When users clone the server it already create a special branch for them
> > > 'master' which track 'origin/master'. And if each user have its own branch
> > > on the server it will completely defeat the goal of the server "collaboration".
> > > And transform the git server in simple rsync server.
> > 
> > I do not think that is what Jens was suggesting. It does not matter in
> > which branch they work, they can directly use master if you like. What
> > he was suggesting is that they create their repository structure like
> > this:
> > 
> > git clone git@somewhere.net:superproject.git
> > cd superproject/submodule
> > git clone git@somehwere.net:subsubmodule.git
> > cd subsubmodule
> > ... work, commit, work, commit ...
> > 
> > The same applies for the superproject. Now only someone with access to
> > the submodule has to update the registered sha1 once the work is pushed
> > to submodule.
> 
> I am not sure to understand everything.
> But if you suggest to clone manually subsubmodule because it could
> not be clone recursively by submodule due to the lake of access write
> to get submodule.
> 
> It is not practical in my use cases.
> Two of the superprojects I have in charge contains hundreds of submodules
> or subsubmodules and I have too much users with disparate computer skills.
> 
> Getting all what a user has access on should be just a recursive clone.

Then I would think about getting rid of the recursion part as it seems
you have interdependencies which can only be solved by a package
management system. I would see the superproject as this package
management system, but it requires you to have all the submodules next
to each other instead of contained in each other.

I think in terms of combining libraries that is actually the correct
solution because there can be modules that need each other. Some
submodule A might evolve and add a dependency to a subsubmodule B that
is itself contained in another submodule C. Then it just does not feel
correct anymore that B is contained in C. You want to have one instance
that is in charge of all the dependencies, that is IMO directly the
superproject and not something that reaches through another submodule
to record a dependency to a subsubmodule.

> > > > > And I need superproject to add also submodule/subsubmodule.
> > > > 
> > > > No. Never let the same file/directory be tracked by two git
> > > > repositories at the same time. Give Paul a branch to work on
> > > > where "submodule" is just an empty directory, and everything
> > > > will be fine. Or move "subsubmodule" outside of "submodule"
> > > > (and let a symbolic link point to the new location if the
> > > > path cannot be easily changed). Would that work for you?
> > > 
> > > If I use symbolic links it will just as gitlink enable to use the
> > > same subsubmodule clone by more than one superproject but with two
> > > major problems :
> > >   - symbolic links do not work under Windows and some of my users do
> > >     not even know something else could exist.
> > >   - symbolic links will not store the SHA-1 of the subsubmodule.
> > >     And a 'git status' in the repository containing the symbolic link
> > >     will say nothing about subsubmodule state.
> > 
> > Here you are also missing something. What Jens was suggesting was that
> > you move your subsubmodule directly underneath the superproject and from
> > the old location you create a link to the new location for a quick
> > transition. But you can also change all paths in your project to point
> > to the new location. But in the new location you will have subsubmodule
> > registered as a submodule only that it is now directly linked (as
> > submodule) from the superproject instead of the submodule.
> > 
> 
> Ok but in this case what happen to someone cloning only submodule but
> not superproject ? He will not get subsubmodule which is part of it.
> Just a dead symbolic link with no hint on what is missing behind.
> 
> Each of my submodules (at any level) should be usable superprojects by
> them self having a gitlink to each subsubmodules they needs.

Then you will end up duplicating many submodules if everything should
contain everything it needs. Think about the example I described above.
In case of libraries, that will become a management nightmare.

> > > I think where we diverge is in the way we are looking gitlinks.
> > > Where you see a hierarchic tree, I see a web.
> > > And I use gitlinks just like multiplatform symbolic links storing
> > > the SHA-1 of there destination and pointing exclusively on git repositories.
> > 
> > Well but the problem with a web is that it will introduce a lot of
> > problems that need to be solved. Some repository has to have the
> > authority about a file (or link). If you have a file in multiple
> > repositories overlayed how do you know who is in charge and when?
> > 
> 
> It will not introduce a lot of problems.
> Me and my teams are using gitlinks this way every days for 2 years know.
> With a web far more complex than the example I give above.
> And the problem you are speaking about and which we solve with the
> --no-separate-git-dir option is the only one we encounter until now.
> 
> This solve the question of who is in charge ? and when ?
> subsubmodule is in charge of itself. Always.
> 
> I know there is some good reasons for the separate gitdir.
> But none of them bother me in my day to day use.
> That is why I came with this simple solution
> 
> Another solution to combine all advantages.
> Is deciding to always make superproject in charge and place a link
> $SUBMODULE_GIT_DIR/modules/link_to_subsubmodule_gitdir
> pointing to $SUPERPROJECT_GIT_DIR/modules/submodule/subsubmodule.
> 
> I think I can write a 3 step patch doing just that :
> 
>   1) A little change in the $GIT_DIR/modules layout making the
> $SUBMODULE_GIT_DIR=$SUPERPROJECT_GIT_DIR/modules/submodule/.git instead of
> $SUBMODULE_GIT_DIR=$SUPERPROJECT_GIT_DIR/modules/submodule
> 
> Then it will be possible to have
> $SUBSUBMODULE_GIT_DIR=$SUPERPROJECT_GIT_DIR/modules/submodule/subsubmodule/.git
> and so on without risk of name collision in case subsubmodule have a name like
> 'branches', 'hooks', 'refs' or anything like that.
> 
>   2) Making submodules aware that they have been cloned has submodule and
> where is at least one of their superproject that is the non trivial part.
> Maybe choosing the one which actually clone it.
> Another solution is to simply found it by the fact the $SUBMODULE_GIT_DIR
> is supposed to be in the $SUPERPROJECT_GIT_DIR.
> 
>   3) Making 'git submodule <add|update>' search recursively through its
> superprojects or directly for the top one and place adequately its own
> $SUBSUBMODULE_GIT_DIR.
> 
> 
> > There is a reason why it is designed like this: simplicity. I currently
> > do not see how your web idea can be simple without introducing a lot of
> > user interface questions.
> > 
> 
> Working with the web idea for several time now I can ensure you that
> Git is so well designed that it is ready to use with this concept.
> I have no user interface problem except the one we are speaking about.

If you have multiple superprojects for a submodule in which superproject
do you commit a change on a gitlink?

Don't you end up having a lot of commits if you have to commit in
multiple superprojects?

Suppose you have a directory layout like this:

super\
   submoduleA\
   	submoduleB

What happens if submoduleA has no knowledge about submoduleB but super
decides to track it? Then on one point submoduleA decides to put files
into the directory named 'submoduleB'. What happens?

Just a few questions that immediately pop into my mind.

Cheers Heiko