[PATCH/RFC v2] git-submodule: multi-level module definition

[PATCH/RFC v2] git-submodule: multi-level module definition

[Ping Yin]

This patch allows multi-level module definition in .gitmodules as
Linus and Sven Verdoolaege etc. have suggested in mails
"Let .git/config specify the url for submodules"
(http://article.gmane.org/gmane.comp.version-control.git/48939).

Following shows an example of such a .gitmodules.

.gitmodules with with multiple level of indirection
------------------------------------------------------
[submodule "service"]
   submodule = crawler
   submodule = search
[submodule "crawler"]
   submodule = util
   submodule = imcrawter
[submodule "search"]
   submodule = util
   submodule = imsearch
[submodule "util"]
   url = git://xyzzy/util.git
[submodule "imsearch"]
   path = search/imsearch
   url = git://xyzzy/imsearch.git
[submodule "imcrawler"]
   path = crawler/imcrawter
   url = git://xyzzy/imcrawter.git
------------------------------------------------------

To simplify the case, submodule sections with submodule attribute should
have neither path attribute nor url attribute (if have, should be ignored).

An option '-m|--module-name' is introduced to designate submodules
by logical module names instead of module paths. So follwoing
commands pairs (1,2), (3,4) will be equivalent.

--------------------------------------
$ git submodule a b c           (1)
$ git submodule -m all          (2)
$ git submodule init a b        (3)
$ git submodule -m init all1    (4)
--------------------------------------

"-m|--module-name" options is introduced with which modules are designated
 by logical names instead of real paths as following example shows.

Identical commands forms with/without "--module-name"
---------------------------------------------------
$ git submodule XXX util imcrawler              (1)
$ git submodule XXX -n crawler                  (2)
$ git submodule XXX util imcrawler imsearch     (3)
$ git submodule XXX -n service                  (4)
$ git submodule XXX -n crawler search           (5)

* XXX represents status, update or init, but not add
* (1) and (2) are idetical conditionally (explained below)
* (3), (4) and (5) are identical conditionally
---------------------------------------------------

There are still minor difference between these two forms.

In the no "--module-name" form, the path parameter may be not the real submodule
path, and it just acts as the filter for real submodule paths. While In the
 "--module-name" form, name parameter must be the logical name, and the real
paths corresponding to the logical name may be neither a submodule path nor
even existent.

How to handle such a path depends on the subcommand.
 - status: Output 0{40} as the sha1. Doing this can remind the user to
   add the path as submodule or delete the path from .gitmodules.
 - update: Skip that path and issue a "Not a submodule" warning
 - init: Also init for that path

So in the example above, commands (1) and (2) are identical only when util and
imcrawler are already submodules.

With "--module-name" option, now we can add submodules in batch.

The former workflow to add submodules is adding one by one with
"git submodule add url path" which then modifies .gitmodules. However,
sometimes it may be more convenient to work in the reverse way: edit
.gitmodules first and then add submodules in batch.

Now "git submodule add --module-name modulename" can help us to do that.
It will find all submodules corresponding to the logical name and add them
in batch by using the path and url from .gitmodules. Of course, it will skip
those paths which have already been submodules.

Signed-off-by: Ping Yin <pkufranky@gmail.com>
---
 git-submodule.sh |  131 +++++++++++++++++++++++++++++++++++++++++++++++-------
 1 files changed, 114 insertions(+), 17 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index a6aaf40..13517b2 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -13,6 +13,7 @@ command=
 branch=
 quiet=
 cached=
+use_module_name=
 
 #
 # print stuff on stdout unless -q was specified
@@ -81,6 +82,75 @@ module_name()
 }
 
 #
+# List all submodule info line by line by given names (path\tname\turl)
+# $@ = module names
+#
+module_info_by_name() {
+	(
+	export GIT_CONFIG=.gitmodules
+	# When names is not given, list all module info
+	if test $# = 0
+	then
+		names=$(git config --get-regexp 'submodule.*.url' |
+			sed 's/submodule.\(.*\).url .*$/\1/' 2>/dev/null
+			)
+		for name in $names
+		do
+			if git config submodule.$name.submodule >/dev/null 2>&1
+			then
+				continue
+			fi
+			url=$(git config submodule.$name.url)
+			path=$(git config submodule.$name.path) || path="$name"
+			sha1=$(git ls-files --stage $path | grep -e '^160000' |
+				awk '{print $2}')
+			test -z "$sha1" && sha1=0000000000000000000000000000000000000000
+			echo "$sha1	$path	$name	$url"
+		done
+		exit
+	fi
+
+	for name
+	do
+		local names=$(git config --get-all submodule.$name.submodule)
+		if test -n "$names"
+		then
+			module_info_by_name $names
+		else
+			url=$(git config submodule.$name.url)
+			test -z "$url" && continue
+			path=$(git config submodule.$name.path) || path="$name"
+			sha1=$(git ls-files --stage $path | grep -e '^160000' |
+				awk '{print $2}')
+			test -z "$sha1" && sha1=0000000000000000000000000000000000000000
+			echo "$sha1	$path	$name	$url"
+		fi
+	done
+	) | LC_ALL=C sort -u
+}
+
+module_info() {
+	if test -n "$use_module_name"
+	then
+		module_info_by_name "$@"
+	else
+		git ls-files --stage -- "$@" | grep -e '^160000 ' |
+		while read mode sha1 stage path
+		do
+			name=$(module_name "$path")
+			if test -n "$name"
+			then
+				url=$(git config submodule."$name".url)
+				echo "$sha1	$path	$name	$url"
+			else
+				echo "$sha1	$path		"
+			fi
+		done | LC_ALL=C sort -u
+	fi
+
+}
+
+#
 # Clone a submodule
 #
 # Prior to calling, cmd_update checks that a possibly existing
@@ -146,6 +216,17 @@ cmd_add()
 		shift
 	done
 
+	if test -n "$use_module_name"
+	then
+		module_info "$@" |
+		while read sha1 path name url
+		do
+			use_module_name=
+			cmd_add "$url" "$path"
+		done
+		return
+	fi
+
 	repo=$1
 	path=$2
 
@@ -220,15 +301,16 @@ cmd_init()
 		shift
 	done
 
-	git ls-files --stage -- "$@" | grep -e '^160000 ' |
-	while read mode sha1 stage path
+	module_info "$@" |
+	while read sha1 path name url
 	do
+		test -z "$name" &&
+		say "No submodule mapping found in .gitmodules for path '$path'" &&
+		continue
 		# Skip already registered paths
-		name=$(module_name "$path") || exit
-		url=$(git config submodule."$name".url)
-		test -z "$url" || continue
+		git config submodule."$name".url >/dev/null 2>&1 &&
+		continue
 
-		url=$(GIT_CONFIG=.gitmodules git config submodule."$name".url)
 		test -z "$url" &&
 		die "No url found for submodule path '$path' in .gitmodules"
 
@@ -274,12 +356,18 @@ cmd_update()
 		shift
 	done
 
-	git ls-files --stage -- "$@" | grep -e '^160000 ' |
-	while read mode sha1 stage path
+	module_info "$@" |
+	while read sha1 path name url
 	do
-		name=$(module_name "$path") || exit
-		url=$(git config submodule."$name".url)
-		if test -z "$url"
+		if test $sha1 = 0000000000000000000000000000000000000000
+		then
+			say "Not a submodule: $name @ $path"
+			continue
+		elif test -z "$name"
+		then
+			say "No submodule mapping found in .gitmodules for path '$path'"
+			continue
+		elif test -z "$url"
 		then
 			# Only mention uninitialized submodules when its
 			# path have been specified
@@ -357,15 +445,21 @@ cmd_status()
 		shift
 	done
 
-	git ls-files --stage -- "$@" | grep -e '^160000 ' |
-	while read mode sha1 stage path
+	module_info "$@" |
+	while read sha1 path name url
 	do
-		name=$(module_name "$path") || exit
-		url=$(git config submodule."$name".url)
-		if test -z "url" || ! test -d "$path"/.git
+		if test $sha1 = 0000000000000000000000000000000000000000
+		then
+			say "*$sha1 $path"
+			continue
+		elif test -z "$name"
+		then
+			say "No submodule mapping found in .gitmodules for path '$path'"
+			continue
+		elif test -z "$url" || ! test -d "$path"/.git
 		then
 			say "-$sha1 $path"
-			continue;
+			continue
 		fi
 		set_name_rev "$path" "$sha1"
 		if git diff-files --quiet -- "$path"
@@ -408,6 +502,9 @@ do
 	--cached)
 		cached=1
 		;;
+	-m|--module-name)
+		use_module_name=1
+		;;
 	--)
 		break
 		;;
-- 
1.5.4.3.347.g5314c

Re: [PATCH/RFC v2] git-submodule: multi-level module definition

[Johannes Schindelin]

Hi,

On Wed, 5 Mar 2008, Ping Yin wrote:

> [submodule "service"]
>    submodule = crawler
>    submodule = search
> [submodule "crawler"]
>    submodule = util
>    submodule = imcrawter
> [submodule "search"]
>    submodule = util
>    submodule = imsearch
> [submodule "util"]
>    url = git://xyzzy/util.git
> [submodule "imsearch"]
>    path = search/imsearch
>    url = git://xyzzy/imsearch.git
> [submodule "imcrawler"]
>    path = crawler/imcrawter
>    url = git://xyzzy/imcrawter.git

In "git remote", we also have a notion of groups.  It looks much nicer, 
IMO.  So why not do this?

[submodules]
	service = crawler search
	crawler = util imcrawler
	search = util imsearch
[submodule "util"]
    url = git://xyzzy/util.git
[submodule "imsearch"]
   path = search/imsearch
   url = git://xyzzy/imsearch.git
[submodule "imcrawler"]
   path = crawler/imcrawter
   url = git://xyzzy/imcrawter.git

However, I think that being able to specify submodule groups recursively 
is only adding to confusion, so I would even prefer this:

[submodules]
	service = util imcrawler imsearch
	crawler = util imcrawler
	search = util imsearch
[...]

Ciao,
Dscho

Re: [PATCH/RFC v2] git-submodule: multi-level module definition

[Ping Yin]

On Wed, Mar 5, 2008 at 12:12 AM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
>  In "git remote", we also have a notion of groups.  It looks much nicer,
>  IMO.  So why not do this?
>
>  [submodules]
>         service = crawler search
>         crawler = util imcrawler
>         search = util imsearch
>
Hmm, it seems more brief.
>
>  However, I think that being able to specify submodule groups recursively
>  is only adding to confusion, so I would even prefer this:
>
>  [submodules]
>         service = util imcrawler imsearch
>         crawler = util imcrawler
>         search = util imsearch
>  [...]
>
This is fine when number of modules is not much, or user may feel
inconvenient since they have to type so many module names.
A hierarchy of modules can help much when having a group of common
modules which should be included in most of role-related logical
moduels.
>  Ciao,
>  Dscho
>



-- 
Ping Yin

Re: [PATCH/RFC v2] git-submodule: multi-level module definition

[Ping Yin]

On Wed, Mar 5, 2008 at 11:17 AM, Imran M Yousuf <imyousuf@gmail.com> wrote:
 > On Tue, Mar 4, 2008 at 10:04 PM, Ping Yin <pkufranky@gmail.com> wrote:

>  >  .gitmodules with with multiple level of indirection
 >  >  ------------------------------------------------------
 >  >  [submodule "service"]
 >  >    submodule = crawler
 >  >    submodule = search
 >  >  [submodule "crawler"]
 >  >    submodule = util
 >  >    submodule = imcrawter
 >  >  [submodule "search"]
 >  >    submodule = util
 >  >    submodule = imsearch

 >

>  At this point I have few questions - Will Service have its own tree
 >  besides its submodules?
 Now, service itself is not a submodule, and service is just a logical
 name for a group of moduels.


 >
 >  For your above example of .gitmodules hierarchy if I issue "git
 >  submodule init -m util", can you please explain what will be the
 >  outcome, i.e. which modules will be initialized.
 >
 Only the util module will be initialized using the path and url in
 submodule.util.{path,url}

 >
 >  --

>  Imran M Yousuf
 >  Entrepreneur & Software Engineer
 >  Smart IT Engineering
 >  Dhaka, Bangladesh
 >  Email: imran@smartitengineering.com
 >  Mobile: +880-1711402557
 >


-- 
Ping Yin

Re: [PATCH/RFC v2] git-submodule: multi-level module definition

[Imran M Yousuf]

On Wed, Mar 5, 2008 at 1:16 PM, Ping Yin <pkufranky@gmail.com> wrote:
>
> On Wed, Mar 5, 2008 at 11:17 AM, Imran M Yousuf <imyousuf@gmail.com> wrote:
>   > On Tue, Mar 4, 2008 at 10:04 PM, Ping Yin <pkufranky@gmail.com> wrote:
>
>  >  >  .gitmodules with with multiple level of indirection
>   >  >  ------------------------------------------------------
>   >  >  [submodule "service"]
>   >  >    submodule = crawler
>   >  >    submodule = search
>   >  >  [submodule "crawler"]
>   >  >    submodule = util
>   >  >    submodule = imcrawter
>   >  >  [submodule "search"]
>   >  >    submodule = util
>   >  >    submodule = imsearch
>
>   >
>
>  >  At this point I have few questions - Will Service have its own tree
>   >  besides its submodules?
>   Now, service itself is not a submodule, and service is just a logical
>   name for a group of moduels.
>
If I am not mistaken I can do it now as well, by "git submodule add
repo path". What would be the difference with this and your approach?
>
>   >
>   >  For your above example of .gitmodules hierarchy if I issue "git
>   >  submodule init -m util", can you please explain what will be the
>   >  outcome, i.e. which modules will be initialized.
>   >
>   Only the util module will be initialized using the path and url in
>   submodule.util.{path,url}
>
I got that part as thats how git-submodules currently work; Actually
what I wanted to know is as util is referenced to several logical
submodules, will they be initialized as well? that means wherever util
is referenced it will be initialized; is that what it means?
>   >
>   >  --
>
>   >
>
>
>  --
>  Ping Yin
>
>
> --
>  To unsubscribe from this list: send the line "unsubscribe git" in
>  the body of a message to majordomo@vger.kernel.org
>  More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Imran M Yousuf
Entrepreneur & Software Engineer
Smart IT Engineering
Dhaka, Bangladesh
Email: imran@smartitengineering.com
Mobile: +880-1711402557

Re: [PATCH/RFC v2] git-submodule: multi-level module definition

[Ping Yin]

On Wed, Mar 5, 2008 at 3:40 PM, Imran M Yousuf <imyousuf@gmail.com> wrote:
> On Wed, Mar 5, 2008 at 1:16 PM, Ping Yin <pkufranky@gmail.com> wrote:
>  >
>  > On Wed, Mar 5, 2008 at 11:17 AM, Imran M Yousuf <imyousuf@gmail.com> wrote:
>  >   > On Tue, Mar 4, 2008 at 10:04 PM, Ping Yin <pkufranky@gmail.com> wrote:
>  >
>  >  >  >  .gitmodules with with multiple level of indirection
>  >   >  >  ------------------------------------------------------
>  >   >  >  [submodule "service"]
>  >   >  >    submodule = crawler
>  >   >  >    submodule = search
>  >   >  >  [submodule "crawler"]
>  >   >  >    submodule = util
>  >   >  >    submodule = imcrawter
>  >   >  >  [submodule "search"]
>  >   >  >    submodule = util
>  >   >  >    submodule = imsearch
>  >
>  >   >
>  >
>  >  >  At this point I have few questions - Will Service have its own tree
>  >   >  besides its submodules?
>  >   Now, service itself is not a submodule, and service is just a logical
>  >   name for a group of moduels.
>  >
>  If I am not mistaken I can do it now as well, by "git submodule add
>  repo path". What would be the difference with this and your approach?
No difference at this point. But using logical name can help add
modules in batch such as "git submodule add -m service"
>
> >
>  >   >
>  >   >  For your above example of .gitmodules hierarchy if I issue "git
>  >   >  submodule init -m util", can you please explain what will be the
>  >   >  outcome, i.e. which modules will be initialized.
>  >   >
>  >   Only the util module will be initialized using the path and url in
>  >   submodule.util.{path,url}
>  >
>  I got that part as thats how git-submodules currently work; Actually
>  what I wanted to know is as util is referenced to several logical
>  submodules, will they be initialized as well? that means wherever util
>  is referenced it will be initialized; is that what it means?
>  >   >
Since the user designate only util module, we needn't automatically
deduce the related modules for them. And as you have said, we can't
deduce well since util may be referenced by several logical modules.
>  >   >  --
>  >
>  >   >
>  >
>  >
>  >  --
>  >  Ping Yin
>
> >
>  >
>  > --
>  >  To unsubscribe from this list: send the line "unsubscribe git" in
>  >  the body of a message to majordomo@vger.kernel.org
>  >  More majordomo info at  http://vger.kernel.org/majordomo-info.html
>  >
>
>
>
>  --
>
>
> Imran M Yousuf
>  Entrepreneur & Software Engineer
>  Smart IT Engineering
>  Dhaka, Bangladesh
>  Email: imran@smartitengineering.com
>  Mobile: +880-1711402557
>



-- 
Ping Yin

Re: [PATCH/RFC v2] git-submodule: multi-level module definition

[Imran M Yousuf]

On Wed, Mar 5, 2008 at 1:53 PM, Ping Yin <pkufranky@gmail.com> wrote:
> On Wed, Mar 5, 2008 at 3:40 PM, Imran M Yousuf <imyousuf@gmail.com> wrote:
>  > On Wed, Mar 5, 2008 at 1:16 PM, Ping Yin <pkufranky@gmail.com> wrote:
>  >  >
>  >  > On Wed, Mar 5, 2008 at 11:17 AM, Imran M Yousuf <imyousuf@gmail.com> wrote:
>  >  >   > On Tue, Mar 4, 2008 at 10:04 PM, Ping Yin <pkufranky@gmail.com> wrote:
>  >  >
>  >  >  >  >  .gitmodules with with multiple level of indirection
>  >  >   >  >  ------------------------------------------------------
>  >  >   >  >  [submodule "service"]
>  >  >   >  >    submodule = crawler
>  >  >   >  >    submodule = search
>  >  >   >  >  [submodule "crawler"]
>  >  >   >  >    submodule = util
>  >  >   >  >    submodule = imcrawter
>  >  >   >  >  [submodule "search"]
>  >  >   >  >    submodule = util
>  >  >   >  >    submodule = imsearch
>  >  >
>  >  >   >
>  >  >
>  >  >  >  At this point I have few questions - Will Service have its own tree
>  >  >   >  besides its submodules?
>  >  >   Now, service itself is not a submodule, and service is just a logical
>  >  >   name for a group of moduels.
>  >  >
>  >  If I am not mistaken I can do it now as well, by "git submodule add
>  >  repo path". What would be the difference with this and your approach?
>  No difference at this point. But using logical name can help add
>  modules in batch such as "git submodule add -m service"
>

Ok I am working on something similar a recurse command for git
submodule; thats why I asked it. Hopefully will appear in mailing list
tomorrow :).

> >
>  > >
>  >  >   >
>  >  >   >  For your above example of .gitmodules hierarchy if I issue "git
>  >  >   >  submodule init -m util", can you please explain what will be the
>  >  >   >  outcome, i.e. which modules will be initialized.
>  >  >   >
>  >  >   Only the util module will be initialized using the path and url in
>  >  >   submodule.util.{path,url}
>  >  >
>  >  I got that part as thats how git-submodules currently work; Actually
>  >  what I wanted to know is as util is referenced to several logical
>  >  submodules, will they be initialized as well? that means wherever util
>  >  is referenced it will be initialized; is that what it means?
>  >  >   >
>  Since the user designate only util module, we needn't automatically
>  deduce the related modules for them. And as you have said, we can't
>  deduce well since util may be referenced by several logical modules.
>
> >  >   >  --
>  >  >
>  >  >   >
>  >  >
>  >  >
>  >  >  --
>  >  >  Ping Yin
>  >
>  > >
>  >  >
>  >  > --
>  >  >  To unsubscribe from this list: send the line "unsubscribe git" in
>  >  >  the body of a message to majordomo@vger.kernel.org
>  >  >  More majordomo info at  http://vger.kernel.org/majordomo-info.html
>  >  >
>  >
>  >
>  >
>  >  --
>  >
>  >
>  > Imran M Yousuf
>  >  Entrepreneur & Software Engineer
>  >  Smart IT Engineering
>  >  Dhaka, Bangladesh
>  >  Email: imran@smartitengineering.com
>  >  Mobile: +880-1711402557
>  >
>
>
>
>  --
>  Ping Yin
>



-- 
Imran M Yousuf
Entrepreneur & Software Engineer
Smart IT Engineering
Dhaka, Bangladesh
Email: imran@smartitengineering.com
Mobile: +880-1711402557

Re: [PATCH/RFC v2] git-submodule: multi-level module definition

[Junio C Hamano]

Ping Yin <pkufranky@gmail.com> writes:

> This patch allows multi-level module definition in .gitmodules as
> Linus and Sven Verdoolaege etc. have suggested in mails
> "Let .git/config specify the url for submodules"
> (http://article.gmane.org/gmane.comp.version-control.git/48939).
> 
> Following shows an example of such a .gitmodules.
>
> .gitmodules with with multiple level of indirection
> ------------------------------------------------------
> [submodule "service"]
>    submodule = crawler
>    submodule = search
> ...
> [submodule "util"]
>    url = git://xyzzy/util.git
> [submodule "imsearch"]
>    path = search/imsearch
>    url = git://xyzzy/imsearch.git
> [submodule "imcrawler"]
>    path = crawler/imcrawter
>    url = git://xyzzy/imcrawter.git
> ------------------------------------------------------

I would agree that allowing the user to use a short-hand to name a group
of modules the user is interested in would be a good idea, but I think
.gitmodules is a wrong place to do so.  The grouping is a user preference,
isn't it?

The place the owner of the repository (not the project) expresses which
modules are of interest, what transports she wants to use to access it,
etc. is $GIT_DIR/config, and .gitmodules is a vehicle to supply hints to
be used when the user populates that information.

Re: [PATCH/RFC v2] git-submodule: multi-level module definition

[Ping Yin]

On Thu, Mar 6, 2008 at 7:18 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Ping Yin <pkufranky@gmail.com> writes:
>
>  > This patch allows multi-level module definition in .gitmodules as
>  > Linus and Sven Verdoolaege etc. have suggested in mails
>  > "Let .git/config specify the url for submodules"
>  > (http://article.gmane.org/gmane.comp.version-control.git/48939).
>  >
>  > Following shows an example of such a .gitmodules.
>  >
>  > .gitmodules with with multiple level of indirection
>  > ------------------------------------------------------
>  > [submodule "service"]
>  >    submodule = crawler
>  >    submodule = search
>  > ...
>
> > [submodule "util"]
>  >    url = git://xyzzy/util.git
>  > [submodule "imsearch"]
>  >    path = search/imsearch
>  >    url = git://xyzzy/imsearch.git
>  > [submodule "imcrawler"]
>  >    path = crawler/imcrawter
>  >    url = git://xyzzy/imcrawter.git
>  > ------------------------------------------------------
>
>  I would agree that allowing the user to use a short-hand to name a group
>  of modules the user is interested in would be a good idea, but I think
>  .gitmodules is a wrong place to do so.  The grouping is a user preference,
>  isn't it?
>
>  The place the owner of the repository (not the project) expresses which
>  modules are of interest, what transports she wants to use to access it,
>  etc. is $GIT_DIR/config, and .gitmodules is a vehicle to supply hints to
>  be used when the user populates that information.
>
Not always the case. In my company environment, we have many
submodules and have a unified hierachy of modules, and we use the same
transports ssh. So after top maintainer give the basic module hierachy
config in .gitmodules, other people needn't and even shouldn't
consider changing these common config. If other people have special
needs, they can of course edit .git/config to add new hierachy or
override existing ones in .gitmodules. However, it's better to put the
common config in .gitmodules to pass to every one.

I think this may not be a special requirement for my company, so
letting .gitmodule record the common module hierachy should be a
common requirement in a company environment.


-- 
Ping Yin

Re: [PATCH/RFC v2] git-submodule: multi-level module definition

[Imran M Yousuf]

On Thu, Mar 6, 2008 at 7:54 AM, Ping Yin <pkufranky@gmail.com> wrote:
>
> On Thu, Mar 6, 2008 at 7:18 AM, Junio C Hamano <gitster@pobox.com> wrote:
>  > Ping Yin <pkufranky@gmail.com> writes:
>  >
>  >  > This patch allows multi-level module definition in .gitmodules as
>  >  > Linus and Sven Verdoolaege etc. have suggested in mails
>  >  > "Let .git/config specify the url for submodules"
>  >  > (http://article.gmane.org/gmane.comp.version-control.git/48939).
>  >  >
>  >  > Following shows an example of such a .gitmodules.
>  >  >
>  >  > .gitmodules with with multiple level of indirection
>  >  > ------------------------------------------------------
>  >  > [submodule "service"]
>  >  >    submodule = crawler
>  >  >    submodule = search
>  >  > ...
>  >
>  > > [submodule "util"]
>  >  >    url = git://xyzzy/util.git
>  >  > [submodule "imsearch"]
>  >  >    path = search/imsearch
>  >  >    url = git://xyzzy/imsearch.git
>  >  > [submodule "imcrawler"]
>  >  >    path = crawler/imcrawter
>  >  >    url = git://xyzzy/imcrawter.git
>  >  > ------------------------------------------------------
>  >
>  >  I would agree that allowing the user to use a short-hand to name a group
>  >  of modules the user is interested in would be a good idea, but I think
>  >  .gitmodules is a wrong place to do so.  The grouping is a user preference,
>  >  isn't it?
>  >
>  >  The place the owner of the repository (not the project) expresses which
>  >  modules are of interest, what transports she wants to use to access it,
>  >  etc. is $GIT_DIR/config, and .gitmodules is a vehicle to supply hints to
>  >  be used when the user populates that information.
>  >
>  Not always the case. In my company environment, we have many
>  submodules and have a unified hierachy of modules, and we use the same
>  transports ssh. So after top maintainer give the basic module hierachy
>  config in .gitmodules, other people needn't and even shouldn't
>  consider changing these common config. If other people have special
>  needs, they can of course edit .git/config to add new hierachy or
>  override existing ones in .gitmodules. However, it's better to put the
>  common config in .gitmodules to pass to every one.
>

I am a bit confused about why we need it as from discussion earlier we
see that there isn't much difference with the currently available
mechanism. I still do not see any advantage over the currently
available system other than having an alternate way to do a thing
already can be done.

>  I think this may not be a special requirement for my company, so
>  letting .gitmodule record the common module hierachy should be a
>  common requirement in a company environment.
>

Actually our company has a high modular structure, using maven makes
it easier to main and link modules. I would find an in place submodule
much more handy for such requirement.

>
>  --
>  Ping Yin
>
>
> --
>  To unsubscribe from this list: send the line "unsubscribe git" in
>  the body of a message to majordomo@vger.kernel.org
>  More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Imran M Yousuf
Entrepreneur & Software Engineer
Smart IT Engineering
Dhaka, Bangladesh
Email: imran@smartitengineering.com
Mobile: +880-1711402557

Re: [PATCH/RFC v2] git-submodule: multi-level module definition

[Johannes Schindelin]

Hi,

On Thu, 6 Mar 2008, Ping Yin wrote:

> On Thu, Mar 6, 2008 at 7:18 AM, Junio C Hamano <gitster@pobox.com> wrote:
> > Ping Yin <pkufranky@gmail.com> writes:
> >
> >  > This patch allows multi-level module definition in .gitmodules as
> >  > Linus and Sven Verdoolaege etc. have suggested in mails
> >  > "Let .git/config specify the url for submodules"
> >  > (http://article.gmane.org/gmane.comp.version-control.git/48939).
> >  >
> >  > Following shows an example of such a .gitmodules.
> >  >
> >  > .gitmodules with with multiple level of indirection
> >  > ------------------------------------------------------
> >  > [submodule "service"]
> >  >    submodule = crawler
> >  >    submodule = search
> >  > ...
> >
> > > [submodule "util"]
> >  >    url = git://xyzzy/util.git
> >  > [submodule "imsearch"]
> >  >    path = search/imsearch
> >  >    url = git://xyzzy/imsearch.git
> >  > [submodule "imcrawler"]
> >  >    path = crawler/imcrawter
> >  >    url = git://xyzzy/imcrawter.git
> >  > ------------------------------------------------------
> >
> >  I would agree that allowing the user to use a short-hand to name a group
> >  of modules the user is interested in would be a good idea, but I think
> >  .gitmodules is a wrong place to do so.  The grouping is a user preference,
> >  isn't it?
> >
> >  The place the owner of the repository (not the project) expresses which
> >  modules are of interest, what transports she wants to use to access it,
> >  etc. is $GIT_DIR/config, and .gitmodules is a vehicle to supply hints to
> >  be used when the user populates that information.
> >
> Not always the case.

If it is _not_ always the case, .gitmodules is definitely the wrong place, 
and $GIT_DIR/config is.

Just like we need "init && update", and not have "init" update implicitly, 
like some people wish (who forget that other people might have other 
wishes), we need to allow for different options here.

And as .gitmodules is _meant_ to be tracked, it is not the place to 
express individual wishes differing from the colleagues' wishes.

Ciao,
Dscho

Re: [PATCH/RFC v2] git-submodule: multi-level module definition

[Junio C Hamano]

"Ping Yin" <pkufranky@gmail.com> writes:

>>  I would agree that allowing the user to use a short-hand to name a group
>>  of modules the user is interested in would be a good idea, but I think
>>  .gitmodules is a wrong place to do so.  The grouping is a user preference,
>>  isn't it?
>>
>>  The place the owner of the repository (not the project) expresses which
>>  modules are of interest, what transports she wants to use to access it,
>>  etc. is $GIT_DIR/config, and .gitmodules is a vehicle to supply hints to
>>  be used when the user populates that information.
>>
> Not always the case. In my company environment, we have many
> submodules and have a unified hierachy of modules,...
> ..., it's better to put the
> common config in .gitmodules to pass to every one.

I think we are saying the same thing, and I do not understand why you
sound so upset about my comments.  I never said it is bad to copy the
"hints" supplied by .gitmodules verbatim to $GIT_DIR/config.

See how the current cmd_init() uses the information in .gitmodules.  It
could later become interactive to allow users to override what's in there,
but by default it simply uses the information from .gitmodules verbatim.

In the special case of everybody using exactly the same settings, you can
have "all" submodule group alias in .gitmodules, and the user can say "git
submodule init all", which would set up the set of the modules the user
would work with, using the "hints" in the .gitmodules distributed company
wide verbatim.

One of the goal of the overall design of submodules is _not forcing_ but
still allowing such uniformity, and I think using $GIT_DIR/config as
authoritative and treating in-tree .gitmodules as hint to prime
$GIT_DIR/config is quite fundamental to that design.  Let's make sure we
do not needlessly deviate from it.

Because names to name things are fundamental part of the communication, I
do not think we would want to allow users to call A what the project as a
whole calls B.  The use of .gitmodules by the current module_name() that
maps a specific name to a path is not just fine but is preferred for this
reason (i.e. if we copied them to $GIT_DIR/config it would only introduce
tower-of-babel confusion).

However, a short-hand to name groups of submodules is not just about
naming one thing with one name, but has user convenience aspect too.
There should be per-user customizability built into the design.  You could
probably fix this by making the code first consult $GIT_DIR/config and
then fall back on in-tree .gitmodules file.

By the way, in your patch, I notice that quoting of $name is often very
loosely done.  I do not think we have officially defined what the allowed
repertoire of module name characters are, but we probably should declare
the rules and make sure the code actively enforces it to prevent future
grief.  I'd say the same restriction as we have for refnames, except
that we do not allow slashes either, is good enough?

Re: [PATCH/RFC v2] git-submodule: multi-level module definition

[Ping Yin]

On Thu, Mar 6, 2008 at 10:32 AM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
>
>  > >  The place the owner of the repository (not the project) expresses which
>  > >  modules are of interest, what transports she wants to use to access it,
>  > >  etc. is $GIT_DIR/config, and .gitmodules is a vehicle to supply hints to
>  > >  be used when the user populates that information.
>  > >
>  > Not always the case.
>
>  If it is _not_ always the case, .gitmodules is definitely the wrong place,
>  and $GIT_DIR/config is.
>
>  Just like we need "init && update", and not have "init" update implicitly,
>  like some people wish (who forget that other people might have other
>  wishes), we need to allow for different options here.
>
>  And as .gitmodules is _meant_ to be tracked, it is not the place to
>  express individual wishes differing from the colleagues' wishes.
>
As an analogy, .gitignore is for colleagues' wishes and
.git/info/exluces is for individual wishes.
The same, .gitmodules is for colleagues' wishes and .git/config is for
individual wishes.

There are always common cases and special cases, so we need a machinsm
to support both of them.
>  Ciao,
>  Dscho
>
>



-- 
Ping Yin

Re: [PATCH/RFC v2] git-submodule: multi-level module definition

[Ping Yin]

On Thu, Mar 6, 2008 at 11:48 AM, Junio C Hamano <gitster@pobox.com> wrote:
> "Ping Yin" <pkufranky@gmail.com> writes:
>
>

>  I think we are saying the same thing, and I do not understand why you
>  sound so upset about my comments.  I never said it is bad to copy the
>  "hints" supplied by .gitmodules verbatim to $GIT_DIR/config.
>
>  See how the current cmd_init() uses the information in .gitmodules.  It
>  could later become interactive to allow users to override what's in there,
>  but by default it simply uses the information from .gitmodules verbatim.
>
>  However, a short-hand to name groups of submodules is not just about
>  naming one thing with one name, but has user convenience aspect too.
>  There should be per-user customizability built into the design.  You could
>  probably fix this by making the code first consult $GIT_DIR/config and
>  then fall back on in-tree .gitmodules file.

We agree on this point and 'fall back' is what i will try to do. Our
difference is the
 way to implement it. Since finally we can fall back to .gitmodules
when information
 is not found in .git/config, why always use 'git init' at the first.

'git init' should be useful when a user has many special requirements which can
help the user copy info from .gitmodules to .git/config and then edit
.git/config.
But in many cases (at least in my company), users may not have special
requirements,
so he needn't use 'git init'.

So now 'git init' is not the first class citizen, it's just a util to
help users when they need it.

And there may be inconsistence between .git/config and .gitmodules
when always using 'git init'
at first. If .gitmodules has changed in the upstream, what should user
do? run 'git init' again? That's
not so easy.  Since .git/config has a whole copy of .gitmodules, the
user has no easy way to know
which entries he has changed which shouldn't follow the upstream changes.

>
>  By the way, in your patch, I notice that quoting of $name is often very
>  loosely done.  I do not think we have officially defined what the allowed
>  repertoire of module name characters are, but we probably should declare
>  the rules and make sure the code actively enforces it to prevent future
>  grief.  I'd say the same restriction as we have for refnames, except
>  that we do not allow slashes either, is good enough?
>
I think it's enough.

In this bash script, there are many places which don't/can't handle
names containing whitespace
well. So the decaring a naming rule for the modue name sounds good.



-- 
Ping Yin

Re: [PATCH/RFC v2] git-submodule: multi-level module definition

[Ping Yin]

On Thu, Mar 6, 2008 at 1:48 PM, Ping Yin <pkufranky@gmail.com> wrote:
> On Thu, Mar 6, 2008 at 11:48 AM, Junio C Hamano <gitster@pobox.com> wrote:
>  > "Ping Yin" <pkufranky@gmail.com> writes:
>  >
>  >
>
>
> >  I think we are saying the same thing, and I do not understand why you
>  >  sound so upset about my comments.  I never said it is bad to copy the
>  >  "hints" supplied by .gitmodules verbatim to $GIT_DIR/config.
>  >
>  >  See how the current cmd_init() uses the information in .gitmodules.  It
>  >  could later become interactive to allow users to override what's in there,
>  >  but by default it simply uses the information from .gitmodules verbatim.
>  >
>
> >  However, a short-hand to name groups of submodules is not just about
>  >  naming one thing with one name, but has user convenience aspect too.
>  >  There should be per-user customizability built into the design.  You could
>  >  probably fix this by making the code first consult $GIT_DIR/config and
>  >  then fall back on in-tree .gitmodules file.
>
>  We agree on this point and 'fall back' is what i will try to do. Our
>  difference is the
>   way to implement it. Since finally we can fall back to .gitmodules
>  when information
>   is not found in .git/config, why always use 'git init' at the first.
>
>  'git init' should be useful when a user has many special requirements which can
>  help the user copy info from .gitmodules to .git/config and then edit
>  .git/config.
>  But in many cases (at least in my company), users may not have special
>  requirements,
>  so he needn't use 'git init'.
>
>  So now 'git init' is not the first class citizen, it's just a util to
>  help users when they need it.
>
>  And there may be inconsistence between .git/config and .gitmodules
>  when always using 'git init'
>  at first. If .gitmodules has changed in the upstream, what should user
>  do? run 'git init' again? That's
>  not so easy.  Since .git/config has a whole copy of .gitmodules, the
>  user has no easy way to know
>  which entries he has changed which shouldn't follow the upstream changes.
>
>
>  >
>  >  By the way, in your patch, I notice that quoting of $name is often very
>  >  loosely done.  I do not think we have officially defined what the allowed
>  >  repertoire of module name characters are, but we probably should declare
>  >  the rules and make sure the code actively enforces it to prevent future
>  >  grief.  I'd say the same restriction as we have for refnames, except
>  >  that we do not allow slashes either, is good enough?
>  >
>  I think it's enough.
It may be not enough. There are many places using code like 'while
read name path',
and path and name are the same by default, so name and path should use the
same name restriction. But path will have slashes and multibyte
characters. So how
about just limiting name and path not containing whitespace characters?

And this should be an implementation restriction. After git-module
becomes a builtin,
we should cancel the limit. So it should be better to put the naming
restriction into
git-submodule.sh instead of the document.

>
>  In this bash script, there are many places which don't/can't handle
>  names containing whitespace
>  well. So the decaring a naming rule for the modue name sounds good.
>
>
>
>  --
>  Ping Yin
>



-- 
Ping Yin