Re: Re: [PATCH] submodule : Add --no-separate-git-dir option to add and update command.

[Henri GEIST <geist.henri@laposte.net>]

[-- Attachment #1: Type: text/plain, Size: 12002 bytes --]

Le mardi 11 mars 2014 à 21:11 +0100, Heiko Voigt a écrit :
> On Tue, Mar 11, 2014 at 10:55:03AM +0100, Henri GEIST wrote:
> > Le lundi 10 mars 2014 à 21:32 +0100, Heiko Voigt a écrit :
> > > On Mon, Mar 10, 2014 at 10:08:06AM +0100, Henri GEIST wrote:
> > > > Le samedi 08 mars 2014 à 00:00 +0100, Jens Lehmann a écrit :
> > > > > Am 06.03.2014 23:20, schrieb Henri GEIST:
> > > > > >> What is the use case you are trying to solve and why can that
> > > > > >> not be handled by adding "subsubmodule" inside "submodule"?
> > > > > > 
> > > > > > The problem is access rights.
> > > > > > 
> > > > > > Imagine you have 2 people Pierre and Paul.
> > > > > > Each with different access write on the server.
> > > > > > Pierre has full access on every things.
> > > > > > Paul has full access on superproject and subsubmodule but no read/write
> > > > > > access to submodule only execution on the directory.
> > > > > 
> > > > > Ok, I think I'm slowly beginning to understand your setup.
> > > > > 
> > > > > > I want all user to get every things they are allowed to have with the
> > > > > > command 'git submodule update --init --recursive'.
> > > > > > Then as Paul can not clone submodule he can not get subsubmodule
> > > > > > recursively through it.
> > > > > 
> > > > > Sure, that's how it should work. Paul could only work on a branch
> > > > > where "submodule" is an empty directory containing "subsubmodule",
> > > > > as he doesn't have the rights to clone "submodule".
> > > > 
> > > > I will not redundantly create a branch for each user on the server.
> > > > When users clone the server it already create a special branch for them
> > > > 'master' which track 'origin/master'. And if each user have its own branch
> > > > on the server it will completely defeat the goal of the server "collaboration".
> > > > And transform the git server in simple rsync server.
> > > 
> > > I do not think that is what Jens was suggesting. It does not matter in
> > > which branch they work, they can directly use master if you like. What
> > > he was suggesting is that they create their repository structure like
> > > this:
> > > 
> > > git clone git@somewhere.net:superproject.git
> > > cd superproject/submodule
> > > git clone git@somehwere.net:subsubmodule.git
> > > cd subsubmodule
> > > ... work, commit, work, commit ...
> > > 
> > > The same applies for the superproject. Now only someone with access to
> > > the submodule has to update the registered sha1 once the work is pushed
> > > to submodule.
> > 
> > I am not sure to understand everything.
> > But if you suggest to clone manually subsubmodule because it could
> > not be clone recursively by submodule due to the lake of access write
> > to get submodule.
> > 
> > It is not practical in my use cases.
> > Two of the superprojects I have in charge contains hundreds of submodules
> > or subsubmodules and I have too much users with disparate computer skills.
> > 
> > Getting all what a user has access on should be just a recursive clone.
> 
> Then I would think about getting rid of the recursion part as it seems
> you have interdependencies which can only be solved by a package
> management system. I would see the superproject as this package
> management system, but it requires you to have all the submodules next
> to each other instead of contained in each other.
>

You put the finger on a key point.

I use the submodule system exactly as a package management system.
It is even the only use I have of it. I am not able to imagine another use.
(My imagination is limited).
I really use 'git clone --recursive' as 'apt-get install'.
And I am pretty sure you also.

And in fact for the case where the submodules/packages should be side by
side, I have a third patch witch enable just this by enabling '../' to be
part of a gitlink.
Much of my submodules/packages make use of this feature but I also have the
case where the dependency make them contained in each others.
 
> I think in terms of combining libraries that is actually the correct
> solution because there can be modules that need each other. Some
> submodule A might evolve and add a dependency to a subsubmodule B that
> is itself contained in another submodule C. Then it just does not feel
> correct anymore that B is contained in C. You want to have one instance
> that is in charge of all the dependencies, that is IMO directly the
> superproject and not something that reaches through another submodule
> to record a dependency to a subsubmodule.

Right.
But each module need to know by its own gitlinks which are its
dependency to be able to track version compatibility and not rely on
an hypothetic superproject which may or may not do it as a submodule
do not even know if it is part of a superproject.
And could be include in totally different superprojects.

> 
> > > > > > And I need superproject to add also submodule/subsubmodule.
> > > > > 
> > > > > No. Never let the same file/directory be tracked by two git
> > > > > repositories at the same time. Give Paul a branch to work on
> > > > > where "submodule" is just an empty directory, and everything
> > > > > will be fine. Or move "subsubmodule" outside of "submodule"
> > > > > (and let a symbolic link point to the new location if the
> > > > > path cannot be easily changed). Would that work for you?
> > > > 
> > > > If I use symbolic links it will just as gitlink enable to use the
> > > > same subsubmodule clone by more than one superproject but with two
> > > > major problems :
> > > >   - symbolic links do not work under Windows and some of my users do
> > > >     not even know something else could exist.
> > > >   - symbolic links will not store the SHA-1 of the subsubmodule.
> > > >     And a 'git status' in the repository containing the symbolic link
> > > >     will say nothing about subsubmodule state.
> > > 
> > > Here you are also missing something. What Jens was suggesting was that
> > > you move your subsubmodule directly underneath the superproject and from
> > > the old location you create a link to the new location for a quick
> > > transition. But you can also change all paths in your project to point
> > > to the new location. But in the new location you will have subsubmodule
> > > registered as a submodule only that it is now directly linked (as
> > > submodule) from the superproject instead of the submodule.
> > > 
> > 
> > Ok but in this case what happen to someone cloning only submodule but
> > not superproject ? He will not get subsubmodule which is part of it.
> > Just a dead symbolic link with no hint on what is missing behind.
> > 
> > Each of my submodules (at any level) should be usable superprojects by
> > them self having a gitlink to each subsubmodules they needs.
> 
> Then you will end up duplicating many submodules if everything should
> contain everything it needs. Think about the example I described above.
> In case of libraries, that will become a management nightmare.
> 

As stated above in this case my other patch enable me to put the submodules
sides by sides and address this issue. Then I have no duplication.

> > > > I think where we diverge is in the way we are looking gitlinks.
> > > > Where you see a hierarchic tree, I see a web.
> > > > And I use gitlinks just like multiplatform symbolic links storing
> > > > the SHA-1 of there destination and pointing exclusively on git repositories.
> > > 
> > > Well but the problem with a web is that it will introduce a lot of
> > > problems that need to be solved. Some repository has to have the
> > > authority about a file (or link). If you have a file in multiple
> > > repositories overlayed how do you know who is in charge and when?
> > > 
> > 
> > It will not introduce a lot of problems.
> > Me and my teams are using gitlinks this way every days for 2 years know.
> > With a web far more complex than the example I give above.
> > And the problem you are speaking about and which we solve with the
> > --no-separate-git-dir option is the only one we encounter until now.
> > 
> > This solve the question of who is in charge ? and when ?
> > subsubmodule is in charge of itself. Always.
> > 
> > I know there is some good reasons for the separate gitdir.
> > But none of them bother me in my day to day use.
> > That is why I came with this simple solution
> > 
> > Another solution to combine all advantages.
> > Is deciding to always make superproject in charge and place a link
> > $SUBMODULE_GIT_DIR/modules/link_to_subsubmodule_gitdir
> > pointing to $SUPERPROJECT_GIT_DIR/modules/submodule/subsubmodule.
> > 
> > I think I can write a 3 step patch doing just that :
> > 
> >   1) A little change in the $GIT_DIR/modules layout making the
> > $SUBMODULE_GIT_DIR=$SUPERPROJECT_GIT_DIR/modules/submodule/.git instead of
> > $SUBMODULE_GIT_DIR=$SUPERPROJECT_GIT_DIR/modules/submodule
> > 
> > Then it will be possible to have
> > $SUBSUBMODULE_GIT_DIR=$SUPERPROJECT_GIT_DIR/modules/submodule/subsubmodule/.git
> > and so on without risk of name collision in case subsubmodule have a name like
> > 'branches', 'hooks', 'refs' or anything like that.
> > 
> >   2) Making submodules aware that they have been cloned has submodule and
> > where is at least one of their superproject that is the non trivial part.
> > Maybe choosing the one which actually clone it.
> > Another solution is to simply found it by the fact the $SUBMODULE_GIT_DIR
> > is supposed to be in the $SUPERPROJECT_GIT_DIR.
> > 
> >   3) Making 'git submodule <add|update>' search recursively through its
> > superprojects or directly for the top one and place adequately its own
> > $SUBSUBMODULE_GIT_DIR.
> > 
> > 
> > > There is a reason why it is designed like this: simplicity. I currently
> > > do not see how your web idea can be simple without introducing a lot of
> > > user interface questions.
> > > 
> > 
> > Working with the web idea for several time now I can ensure you that
> > Git is so well designed that it is ready to use with this concept.
> > I have no user interface problem except the one we are speaking about.
> 
> If you have multiple superprojects for a submodule in which superproject
> do you commit a change on a gitlink?

Each superproject impacted by the changes.
But only when I need it to be updated.

> 
> Don't you end up having a lot of commits if you have to commit in
> multiple superprojects?
> 

I have only commit in project which have a real change which conceptually
need a commit.
A careful design of projects modularization and dependency will not
lead to unneeded commits. But if you create unneeded dependency between
your submodules you can have a lot of meaningless commit.
It is only a project design problem not a Git problem.

But in counterpart it add some capital value (at least to my eyes),
traceability of the dependency between submodules/packages through their SHA-1.

> Suppose you have a directory layout like this:
> 
> super\
>    submoduleA\
>    	submoduleB
> 
> What happens if submoduleA has no knowledge about submoduleB but super
> decides to track it? Then on one point submoduleA decides to put files
> into the directory named 'submoduleB'. What happens?
> 

It is a general design question not a Git question.
If super have put a submoduleB in the submoduleA directory it is intentional
and should be sound in your design.
And as designer I see no good reason to place submoduleB in a submoduleA already containing
a submoduleA directory or track file in submoduleB by submoduleA.
And in fact Git prevent this.
My patch only enable it for gitlinks.

But maybe you have in mind a case where the maintainer of super will do this
for a good reason in this case I can enable it.

> Just a few questions that immediately pop into my mind.
> 
> Cheers Heiko



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 230 bytes --]