[PATCH 0/3] Update submodule documentation

[PATCH 0/3] Update submodule documentation

[Stefan Beller]

I think I sent out the first 2 patches once upon a time, but the review
or me resending them stalled. They improve the man page for the git-submodule
command.

The third patch is RFC-ish and adds a background story to submodules
Similar to gitattributes, gitnamespaces, gittutorial it doesn't describe
a particular command but e.g. explains design.

Thanks,
Stefan

Stefan Beller (3):
  submodule documentation: add options to the subcommand
  submodule update documentation: don't repeat ourselves
  submodules: add a background story

 Documentation/git-submodule.txt |  93 +++++++--------------
 Documentation/gitsubmodules.txt | 176 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 206 insertions(+), 63 deletions(-)
 create mode 100644 Documentation/gitsubmodules.txt

-- 
2.11.0.196.gee862f456e.dirty

[PATCH 1/3] submodule documentation: add options to the subcommand

[Stefan Beller]

When reading up on a subcommand of `git submodule <subcommand>`,
it is convenient to have its options nearby and not just at the
top of the man page.  Add the options to each subcommand.

While at it, also document the `--checkout` option for `update`.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 Documentation/git-submodule.txt | 27 +++++++++++----------------
 1 file changed, 11 insertions(+), 16 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index d841573475..f9da0d9e91 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -9,17 +9,12 @@ git-submodule - Initialize, update or inspect submodules
 SYNOPSIS
 --------
 [verse]
-'git submodule' [--quiet] add [-b <branch>] [-f|--force] [--name <name>]
-	      [--reference <repository>] [--depth <depth>] [--] <repository> [<path>]
+'git submodule' [--quiet] add [<options>] [--] <repository> [<path>]
 'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] init [--] [<path>...]
 'git submodule' [--quiet] deinit [-f|--force] (--all|[--] <path>...)
-'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch]
-	      [--[no-]recommend-shallow] [-f|--force] [--rebase|--merge]
-	      [--reference <repository>] [--depth <depth>] [--recursive]
-	      [--jobs <n>] [--] [<path>...]
-'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
-	      [commit] [--] [<path>...]
+'git submodule' [--quiet] update [<options>] [--] [<path>...]
+'git submodule' [--quiet] summary [<options>] [--] [<path>...]
 'git submodule' [--quiet] foreach [--recursive] <command>
 'git submodule' [--quiet] sync [--recursive] [--] [<path>...]
 
@@ -62,7 +57,7 @@ if you choose to go that route.
 
 COMMANDS
 --------
-add::
+add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--depth <depth>] [--] <repository> [<path>]::
 	Add the given repository as a submodule at the given path
 	to the changeset to be committed next to the current
 	project: the current project is termed the "superproject".
@@ -103,7 +98,7 @@ together in the same relative location, and only the
 superproject's URL needs to be provided: git-submodule will correctly
 locate the submodule using the relative URL in .gitmodules.
 
-status::
+status [--cached] [--recursive] [--] [<path>...]::
 	Show the status of the submodules. This will print the SHA-1 of the
 	currently checked out commit for each submodule, along with the
 	submodule path and the output of 'git describe' for the
@@ -120,7 +115,7 @@ submodules with respect to the commit recorded in the index or the HEAD,
 linkgit:git-status[1] and linkgit:git-diff[1] will provide that information
 too (and can also report changes to a submodule's work tree).
 
-init::
+init [--] [<path>...]::
 	Initialize the submodules recorded in the index (which were
 	added and committed elsewhere) by copying submodule
 	names and urls from .gitmodules to .git/config.
@@ -135,7 +130,7 @@ init::
 	the explicit 'init' step if you do not intend to customize
 	any submodule locations.
 
-deinit::
+deinit [-f|--force] (--all|[--] <path>...)::
 	Unregister the given submodules, i.e. remove the whole
 	`submodule.$name` section from .git/config together with their work
 	tree. Further calls to `git submodule update`, `git submodule foreach`
@@ -151,7 +146,7 @@ instead of deinit-ing everything, to prevent mistakes.
 If `--force` is specified, the submodule's working tree will
 be removed even if it contains local modifications.
 
-update::
+update [--init] [--remote] [-N|--no-fetch] [--[no-]recommend-shallow] [-f|--force] [--checkout|--rebase|--merge] [--reference <repository>] [--depth <depth>] [--recursive] [--jobs <n>] [--] [<path>...]::
 +
 --
 Update the registered submodules to match what the superproject
@@ -197,7 +192,7 @@ submodule with the `--init` option.
 If `--recursive` is specified, this command will recurse into the
 registered submodules, and update any nested submodules within.
 --
-summary::
+summary [--cached|--files] [(-n|--summary-limit) <n>] [commit] [--] [<path>...]::
 	Show commit summary between the given commit (defaults to HEAD) and
 	working tree/index. For a submodule in question, a series of commits
 	in the submodule between the given super project commit and the
@@ -210,7 +205,7 @@ summary::
 Using the `--submodule=log` option with linkgit:git-diff[1] will provide that
 information too.
 
-foreach::
+foreach [--recursive] <command>::
 	Evaluates an arbitrary shell command in each checked out submodule.
 	The command has access to the variables $name, $path, $sha1 and
 	$toplevel:
@@ -231,7 +226,7 @@ As an example, +git submodule foreach \'echo $path {backtick}git
 rev-parse HEAD{backtick}'+ will show the path and currently checked out
 commit for each submodule.
 
-sync::
+sync [--recursive] [--] [<path>...]::
 	Synchronizes submodules' remote URL configuration setting
 	to the value specified in .gitmodules. It will only affect those
 	submodules which already have a URL entry in .git/config (that is the
-- 
2.11.0.196.gee862f456e.dirty

[PATCH 2/3] submodule update documentation: don't repeat ourselves

[Stefan Beller]

The documentation for the `git submodule update` command, repeats itself
for each update option, "This is done when <option> is given, or no
option is given and `submodule.<name>.update` is set to <string>.

Avoid these repetitive clauses by stating the command line options take
precedence over configured options.

Also add 'none' to the list of options instead of mentioning it in the
following running text and split the list into two parts, one that is
accessible via the command line and one that is only reachable via the
configuration variables.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 Documentation/git-submodule.txt | 30 ++++++++++++++----------------
 1 file changed, 14 insertions(+), 16 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index f9da0d9e91..7e03ccc444 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -153,13 +153,13 @@ Update the registered submodules to match what the superproject
 expects by cloning missing submodules and updating the working tree of
 the submodules. The "updating" can be done in several ways depending
 on command line options and the value of `submodule.<name>.update`
-configuration variable. Supported update procedures are:
+configuration variable. The command line option takes precedence over
+the configuration variable. if neither is given, a checkout is performed.
+update procedures supported both from the command line as well as setting
+`submodule.<name>.update`:
 
 	checkout;; the commit recorded in the superproject will be
-	    checked out in the submodule on a detached HEAD. This is
-	    done when `--checkout` option is given, or no option is
-	    given, and `submodule.<name>.update` is unset, or if it is
-	    set to 'checkout'.
+	    checked out in the submodule on a detached HEAD.
 +
 If `--force` is specified, the submodule will be checked out (using
 `git checkout --force` if appropriate), even if the commit specified
@@ -167,23 +167,21 @@ in the index of the containing repository already matches the commit
 checked out in the submodule.
 
 	rebase;; the current branch of the submodule will be rebased
-	    onto the commit recorded in the superproject. This is done
-	    when `--rebase` option is given, or no option is given, and
-	    `submodule.<name>.update` is set to 'rebase'.
+	    onto the commit recorded in the superproject.
 
 	merge;; the commit recorded in the superproject will be merged
-	    into the current branch in the submodule. This is done
-	    when `--merge` option is given, or no option is given, and
-	    `submodule.<name>.update` is set to 'merge'.
+	    into the current branch in the submodule.
+
+The following procedures are only available via the `submodule.<name>.update`
+configuration variable:
 
 	custom command;; arbitrary shell command that takes a single
 	    argument (the sha1 of the commit recorded in the
-	    superproject) is executed. This is done when no option is
-	    given, and `submodule.<name>.update` has the form of
-	    '!command'.
+	    superproject) is executed. When `submodule.<name>.update`
+	    is set to '!command', the remainder after the exclamation mark
+	    is the custom command.
 
-When no option is given and `submodule.<name>.update` is set to 'none',
-the submodule is not updated.
+	none;; the submodule is not updated.
 
 If the submodule is not yet initialized, and you just want to use the
 setting as stored in .gitmodules, you can automatically initialize the
-- 
2.11.0.196.gee862f456e.dirty

[PATCH 3/3] submodules: add a background story

[Stefan Beller]

Just like gitattributes, gitcredentials, gitnamespaces, gittutorial,
gitmodules, we'd like to provide some background on submodules,
which is not specific to the `submodule` command, but elaborates
on the background and its intended usage.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 Documentation/git-submodule.txt |  36 ++------
 Documentation/gitsubmodules.txt | 176 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 181 insertions(+), 31 deletions(-)
 create mode 100644 Documentation/gitsubmodules.txt

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 7e03ccc444..c99920aceb 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -23,37 +23,7 @@ DESCRIPTION
 -----------
 Inspects, updates and manages submodules.
 
-A submodule allows you to keep another Git repository in a subdirectory
-of your repository. The other repository has its own history, which does not
-interfere with the history of the current repository. This can be used to
-have external dependencies such as third party libraries for example.
-
-When cloning or pulling a repository containing submodules however,
-these will not be checked out by default; the 'init' and 'update'
-subcommands will maintain submodules checked out and at
-appropriate revision in your working tree.
-
-Submodules are composed from a so-called `gitlink` tree entry
-in the main repository that refers to a particular commit object
-within the inner repository that is completely separate.
-A record in the `.gitmodules` (see linkgit:gitmodules[5]) file at the
-root of the source tree assigns a logical name to the submodule and
-describes the default URL the submodule shall be cloned from.
-The logical name can be used for overriding this URL within your
-local repository configuration (see 'submodule init').
-
-Submodules are not to be confused with remotes, which are other
-repositories of the same project; submodules are meant for
-different projects you would like to make part of your source tree,
-while the history of the two projects still stays completely
-independent and you cannot modify the contents of the submodule
-from within the main project.
-If you want to merge the project histories and want to treat the
-aggregated whole as a single project from then on, you may want to
-add a remote for the other project and use the 'subtree' merge strategy,
-instead of treating the other project as a submodule. Directories
-that come from both projects can be cloned and checked out as a whole
-if you choose to go that route.
+For more information about submodules, see linkgit:gitsubmodules[5]
 
 COMMANDS
 --------
@@ -405,6 +375,10 @@ This file should be formatted in the same way as `$GIT_DIR/config`. The key
 to each submodule url is "submodule.$name.url".  See linkgit:gitmodules[5]
 for details.
 
+SEE ALSO
+--------
+linkgit:gitsubmodules[1], linkgit:gitmodules[1].
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/Documentation/gitsubmodules.txt b/Documentation/gitsubmodules.txt
new file mode 100644
index 0000000000..aa5fa25608
--- /dev/null
+++ b/Documentation/gitsubmodules.txt
@@ -0,0 +1,176 @@
+gitsubmodules(5)
+================
+
+NAME
+----
+gitsubmodules - information about submodules
+
+SYNOPSIS
+--------
+$GIT_DIR/config, .gitmodules
+
+------------------
+git submodule
+------------------
+
+DESCRIPTION
+-----------
+
+A submodule allows you to keep another Git repository in a subdirectory
+of your repository. The other repository has its own history, which does not
+interfere with the history of the current repository. This can be used to
+have external dependencies such as third party libraries for example.
+
+Submodules are composed from a so-called `gitlink` tree entry
+in the main repository that refers to a particular commit object
+within the inner repository that is completely separate.
+A record in the `.gitmodules` (see linkgit:gitmodules[5]) file at the
+root of the source tree assigns a logical name to the submodule and
+describes the default URL the submodule shall be cloned from.
+The logical name can be used for overriding this URL within your
+local repository configuration (see 'submodule init').
+
+Submodules are not to be confused with remotes, which are other
+repositories of the same project; submodules are meant for
+different projects you would like to make part of your source tree,
+while the history of the two projects still stays completely
+independent and you cannot modify the contents of the submodule
+from within the main project.
+If you want to merge the project histories and want to treat the
+aggregated whole as a single project from then on, you may want to
+add a remote for the other project and use the 'subtree' merge strategy,
+instead of treating the other project as a submodule. Directories
+that come from both projects can be cloned and checked out as a whole
+if you choose to go that route.
+
+When cloning or pulling a repository containing submodules however,
+the submodules will not be checked out by default; You need to instruct
+'clone' to recurse into submodules. The 'init' and 'update' subcommands
+of 'git submodule' will maintain submodules checked out and at an
+appropriate revision in your working tree.
+
+WHEN TO USE
+-----------
+
+Submodules, repositories in other repositories, can be used for different
+use cases:
+
+* To have finer grained access control.
+  The design principles of Git  do not allow for partial repositories to be
+  checked out or transferred. A repository is the smallest unit that a user
+  can be given access to. Submodules are separate repositories, such that
+  you can restrict access to parts of your project via the use of submodules.
+
+* To decouple Git histories.
+  Decoupling histories has different benefits.
+
+  When you want to use a (third party) library tied to a specific version.
+  Using submodules for a library allows you to have a clean history for
+  your own project and only updating the library in the submodule when needed.
+
+  In its current form Git scales up poorly for very large repositories that
+  change a lot, as the history grows very large. For that you may want to look
+  at shallow clone, sparse checkout or git-lfs.
+  However you can also use submodules to e.g. hold large binary assets
+  and these repositories are then shallowly cloned such that you do not
+  have a large history locally.
+
+STATES
+------
+
+Submodules can be in different states, the following indicators are used:
+
+* the existence of the setting of 'submodule.<name>.url' in the
+  superprojects configuration
+* the existence of the submodules working tree within the
+  working tree of the superproject
+* the existence of the submodules git directory within the superprojects
+  git directory at $GIT_DIR/modules/<name> or within the submodules working
+  tree
+
+      State      URL config        working tree     git dir
+      -----------------------------------------------------
+      uninitialized    no               no           no
+      initialized     yes               no           no
+      populated       yes              yes          yes
+      depopulated     yes               no          yes
+      deinitialized    no               no          yes
+      uninteresting    no              yes          yes
+
+      invalid          no              yes           no
+      invalid         yes              yes           no
+      -----------------------------------------------------
+
+The first six states can be reached by normal git usage, the latter two are
+only shown for completeness. The states in detail:
+
+* uninitialized
+  The uninitialized state is the default state if no
+  '--recurse-submodules' / '--recursive'. An empty directory will be put in
+  the working tree as a place holder, such that you are reminded of the
+  existence of the submodule.
+
+  To transition into the initialized state
+  you can use 'git submodule init', which copies the presets from the
+  .gitmodules file into the config.
+
+* initialized
+  Users transitioned from the uninitialized state to this state via
+  'git submodule init', which preset the URL configuration. As these URLs
+  may not be desired in certain scenarios, this state allows to change the
+  URLs.  For example in a corporate environment you may want to run
+
+    sed -i s/github.com/$internal-mirror/ .git/config
+
+  before proceeding to populate the submodules.
+
+* populated
+  In the populated state you have the submodule fully available.
+
+* depopulated
+  In this state you still have the git directory around, but the working tree
+  is gone.  For example when the superproject checks out a revision that doesn't
+  have the submodule, the state changes to depopulated.
+
+* deinitialized
+  The git directory is still there, but the user is no longer interested in the
+  submodule as inidicated by the missing URL configuration.
+
+
+INNER WORKINGS
+--------------
+
+On the location of the git directory
+------------------------------------
+
+Since v1.7.7 of Git, the git directory of submodules is stored inside the
+superprojects git directory at $GIT_DIR/modules/<submodule-name>
+This location allows for the working tree to be non existent while keeping
+the history around. So we can use git-rm on a submodule without loosing
+information that may only be local.
+
+In the future we may see git-checkout that can checkout submodules and
+revisions that do not contain the submodule can still be checked out without
+having to drop the submodules git directory.
+
+It is also possbile to imagine a future in which a bare repository still
+contains its submodules inside the modules sub directory, such that you can
+get a full clone including submodules from that bare repository, the URLs
+as configured or given in the .gitmodules would only be used as a backup.
+
+SAMPLE WORKFLOWS (RFC/TODO)
+----------------
+Do we need
+* an opinionated way to check for a specific state of a submodule
+* (submodule helper to be plumbing?)
+* expose the design mistake of having the (name->path) mapping inside the
+  working tree, i.e. never remove a name from the submodule config even when
+  the submodule doesn't exist any more.
+
+SEE ALSO
+--------
+linkgit:git-submodule[1], linkgit:gitmodules[1].
+
+GIT
+---
+Part of the linkgit:git[1] suite
-- 
2.11.0.196.gee862f456e.dirty